Method, system and device for improving image resolution

Abstract

The invention provides a method, a system and a device for improving image resolution, and belongs to the technical field of image processing. The method comprises the following steps: acquiring an original image obtained based on biomolecule microscopic detection; inputting the original image into an image resolution improving model to obtain a target image; wherein the image resolution improving model is obtained by the following method: obtaining a plurality of groups of image pairs, the plurality of groups of image pairs being image pairs obtained by images of at least one visual field collected by a microscopic imaging system at at least two time points, the visual field comprising a plurality of optically detectable biomolecules, one group of image pairs comprises a first image and a second image which are collected at the same time point and have the same view field, and the peak signal-to-noise ratio of the first image is higher than that of the second image; and inputting the plurality of groups of image pairs into a machine learning model, extracting features of each second image, and training the machine learning model by taking the first image as a label. According to the method, the image resolution can be effectively improved, and the image quality is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a method, system and device for improving image resolution. More specifically, the present application relates to a method and system for improving image resolution, a computer device, a computer-readable storage medium and a computer program product. Background Art

[0002] With the rapid development of high-throughput sequencing technology, the accuracy and quality of sequencing data have become important factors affecting the results of bioinformatics analysis. In sequencing technology, the resolution of sequencing images directly affects the sequencing results.

[0003] At present, the main methods to improve the resolution of sequencing fluorescence images include: improving the optical system, optimizing the imaging algorithm or using traditional image deblurring technology. Among them, improving the optical system requires higher-precision hardware, which is costly and constrained by physical limits; traditional image deblurring methods (such as Wiener filtering, non-blind deconvolution, etc.) are difficult to adaptively handle complex blurring caused by various factors (such as sample table deformation, motion blur, and focus error, etc.) during the sequencing process. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one purpose of the present application is to propose a method that can effectively improve the resolution of sequencing images.

[0005] In the first aspect of the present application, the present application proposes a method for improving image resolution. According to an embodiment of the present application, the method includes: obtaining an original image obtained based on biomolecule microscopic detection; inputting the original image into an image resolution improvement model to obtain a target image; wherein the image resolution improvement model is obtained by the following method: obtaining multiple groups of image pairs, the multiple groups of image pairs are image pairs obtained by images of at least one field of view acquired by a microscopic imaging system at at least two time points, the field of view contains multiple optically detectable biomolecules, and a group of image pairs includes a first image and a second image of the same field of view acquired at the same time point, and the peak signal-to-noise ratio of the first image is higher than the peak signal-to-noise ratio of the second image; inputting the multiple groups of image pairs into a machine learning model, extracting the features of each of the second images, and using the first image as a label to train the machine learning model to obtain the image resolution improvement model.

[0006] The above method can effectively improve image resolution and image quality by inputting the original image into an image resolution enhancement model trained based on machine learning. This method uses multiple sets of high peak signal-to-noise ratio and low peak signal-to-noise ratio images to train the model, adaptively handles the diverse blur problems introduced by factors such as out-of-focus blur, motion blur, and deformation blur, reduces dependence on hardware improvements, and reduces costs and technical barriers. At the same time, in the scenario of biomolecular microscopy, high peak signal-to-noise ratio images help reduce crosstalk between fluorescent signal points and improve the accuracy of downstream algorithms in reading the brightness of bright spots such as fluorescent spots, thereby improving the reliability of base recognition and sequencing results.

[0007] In the second aspect of the present application, the present application proposes a system for improving image resolution. According to an embodiment of the present application, the system includes: an original image acquisition module for acquiring an original image obtained based on biomolecule microscopic detection; a target image acquisition module for inputting the original image into an image resolution improvement model to obtain a target image; wherein the image resolution improvement model is obtained by the following method: acquiring multiple groups of image pairs, the multiple groups of image pairs are image pairs obtained from images of at least one field of view acquired by a microscopic imaging system at at least two time points, the field of view contains multiple optically detectable biomolecules, and a group of image pairs includes a first image and a second image of the same field of view acquired at the same time point, and the image resolution of the first image is higher than the image resolution of the second image; inputting the multiple groups of image pairs into a machine learning model, extracting the features of each of the second images, and training the machine learning model with the first image as a label to obtain the image resolution improvement model.

[0008] The image resolution enhancement system uses a machine learning model to output a higher-resolution target image based on a low-resolution original image, effectively breaking through the diffraction limit and other blur factors, and improving the clarity and details of the image. The system is trained using image pairs with multiple blur types, allowing the model to adapt to different blur conditions, enhancing its flexibility and accuracy in practical applications.

[0009] In a third aspect of the present application, the present application proposes an electronic device. According to an embodiment of the present application, the device comprises: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program to implement the method described in the first aspect.

[0010] In a fourth aspect of the present application, the present application proposes a computer-readable storage medium. According to an embodiment of the present application, the computer-readable storage medium includes computer instructions, and when the instructions are executed by a computer, the computer implements the method described in the first aspect.

[0011] In a fifth aspect of the present application, the present application proposes a computer program product. According to an embodiment of the present application, the computer program product includes computer instructions, and when part or all of the computer instructions are run on a computer, the model training method described in the first aspect is executed. The electronic device, computer-readable storage medium and computer program product described above achieve efficient automation by automatically executing computer instructions to improve image resolution. In addition, the instruction-based characteristics make it more stable in different environments.

[0012] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 This is a flow chart of a method for improving image resolution provided by an embodiment of the present application; Figure 2 This is a schematic diagram of a system for improving sequencing image resolution provided by an embodiment of the present application; Figure 3 is a schematic diagram of an electronic device provided by an embodiment of the present application; Figure 4 This is a schematic diagram of the comparison results of the upper left corner of the original image before and after processing provided by an embodiment of the present application; Figure 5 This is a schematic diagram of the comparison results of the upper right corner of the original image before and after processing provided by an embodiment of the present application; Figure 6 This is a schematic diagram of the comparison results of the lower right corner of the original image before and after processing provided by an embodiment of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0016] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0017] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0018] In this article, unless otherwise specified, the term "machine learning model" refers to a computing model or algorithm that can automatically perform tasks such as prediction, classification, recognition or decision-making by learning and analyzing input data. The learning process of the model is based on statistical principles and data pattern recognition, and uses training data sets to adjust model parameters and optimize the model to improve its prediction or reasoning ability. Machine learning models can use various algorithms and techniques, such as neural networks, support vector machines, decision trees, random forests, deep learning, etc. These models can be trained and optimized by supervised learning, unsupervised learning or reinforcement learning. In practical applications, machine learning models can be used in various fields, such as natural language processing, image recognition, pattern recognition, data mining, recommendation systems, predictive analysis, etc. It has important application potential in processing large-scale data, automated decision-making and intelligent systems. However, it should be noted that in the specific application of machine learning models, it is necessary to conduct in-depth research on the features and models used for prediction in order to obtain a more satisfactory prediction effect, otherwise various problems will arise, such as overfitting problems, underfitting problems, and poor generalization problems. After a large number of experimental verifications, the inventors of the present application found that by using a machine learning model to train multiple types of second images (fuzzy sequencing images) and their corresponding first images (clear sequencing images), a machine learning model that can adaptively process fuzzy sequencing images is obtained.

[0019] In this article, unless otherwise specified, the term "sequencing image" refers to the image data generated during the DNA / RNA sequencing process. This process aims to determine the exact order of bases (adenine, cytosine, guanine and thymine) in the DNA molecule. The generation of sequencing images goes through several key steps: First, the DNA molecule is usually divided into short fragments, and then these fragments are introduced into the sequencing reaction. In the sequencing reaction, the bases are marked with special markers, and the markers of each base are distinguished using different fluorescent colors for subsequent identification. Next, the fluorescent signals generated by the sequencing reaction are captured and recorded in the image, usually by a camera or laser scanner. The position and intensity of these signals reflect the sequence and number of bases. In order to obtain accurate sequence information of DNA, complex image processing and signal analysis are required to associate each signal with its corresponding base. The signals contained in each sequencing image are processed by a computer program or algorithm. Finally, the exact sequence of the DNA molecule is obtained. The inventors found that during the sequencing process, the sequencing image may cause blurring due to various reasons during the image processing and signal analysis process, which makes the analysis of the sequencing signal difficult and the accuracy of the sequencing results low.

[0020] In this article, unless otherwise specified, the so-called sequencing images can be obtained through different sequencing platforms, and there is no special limitation in this application. In some examples of this application, the optional sequencing platforms include but are not limited to Illumina's Hiseq, Miseq, Nextseq and Novasek sequencing platforms, Thermo Fisher / Life Technologies' Ion Torrent platform, BGISEQ and MGISEQ / DNBSEQ platforms of BGI, and single-molecule sequencing platforms; the sequencing method can be selected from single-end sequencing, double-end sequencing, or a sequencing method supported by the selected automated sequencing platform, etc.

[0021] In this article, unless otherwise specified, the so-called "spots" (spots or peaks), also called "bright spots" or "light spots", refer to locations on an image where the signal is relatively strong. For example, the signal at this location is stronger than the surrounding area, and it appears as a relatively bright spot or point on the image. A bright spot or a so-called location occupies one or more pixels.

[0022] In this article, unless otherwise specified, the term "Training Set" refers to the sequenced images used to train the machine learning model. It contains a series of input samples (blurred images) and corresponding known outputs (clear images). During the training phase, the machine learning model uses the samples in the training set to learn and adjust the parameters of the model to minimize the error between the predicted results and the true labels. By constantly trying different parameters and algorithms, the model gradually learns the rules and characteristics between samples, thereby obtaining more accurate prediction capabilities.

[0023] In this article, unless otherwise specified, the term "test set" is a dataset used to evaluate the performance of a machine learning model. It also contains input samples (blurred images) and the corresponding true outputs (clear images), but the samples in the test set have not been used in the training phase of the model, which ensures that the data in the test set is unknown to the model. After training, the test set is used to evaluate the performance of the model, that is, the model predicts the samples in the test set and compares them with the true labels. By comparing the predicted results with the true labels, the performance of the model on unknown data is evaluated to determine its generalization ability and prediction accuracy.

[0024] In the field of gene sequencing, the resolution of sequencing images has a direct impact on the accuracy of sequencing results. In this context, there may be a variety of factors that cause the resolution of sequencing images to decrease, such as motion blur that may occur during image acquisition, out-of-focus blur caused by inaccurate focusing, and out-of-focus blur caused by random deformation of the sample platform (biochip, also known as sequencing chip). In addition, the factors that cause out-of-focus blur present randomness and diversity, such as existing in different sequencing rounds, or between different base images in the same round and different regions of the same base image. In response to the above problems, some technologies have been used to improve the resolution of sequencing images, but because the training data of the machine learning model for improving the resolution of sequencing images recorded in the prior art is single, it only contains image data taken at reduced power and normal power (WO2023158804A1) or image training data sets taken at different mobile stabilization times (without reduction and reduction) (WO2023158809A1), it is difficult to effectively cope with the challenge of low resolution of sequencing images caused by various reasons.

[0025] In response to the above-mentioned deficiencies, the inventors have made many optimizations, such as increasing the diversity of machine learning model training data, introducing a spatial attention mechanism, and segmenting and reconstructing the original image.

[0026] In one aspect of the present application, the present application proposes a method for improving image resolution, referring to Figure 1 , the method comprising: S110, acquiring an original image obtained based on biomolecule microscopic detection; In this article, "biomolecule microscopic detection" refers to the process of observing, analyzing and detecting biomolecules using microscopic imaging technology. Specifically, it involves imaging biomolecules (such as nucleic acids, proteins, etc.) with optically detectable labels (such as fluorescent labels) through a microscopic imaging system (such as a fluorescence microscope) to obtain information such as their type and spatial distribution. In the embodiments of the present application, biomolecules refer to organic molecules with specific structures and functions in organisms, including but not limited to proteins, nucleic acids, polysaccharides and lipids. In some specific examples of the present application, biomolecule microscopic detection includes imaging fluorescently labeled nucleic acid molecules using a microscopic imaging system.

[0027] S120, inputting the original image into an image resolution enhancement model to obtain a target image; The image resolution enhancement model is obtained by the following method: Acquire a plurality of sets of image pairs, wherein the plurality of sets of image pairs are image pairs obtained from images of at least one field of view acquired by a microscopic imaging system at at least two time points, wherein the field of view includes a plurality of optically detectable biomolecules, and a set of the image pairs includes a first image and a second image of the same field of view acquired at the same time point, and a peak signal-to-noise ratio of the first image is higher than a peak signal-to-noise ratio of the second image; Input multiple groups of image pairs into the machine learning model, extract the features of each second image, and train the machine learning model with the first image as a label to obtain the image resolution enhancement model.

[0028] In the embodiment of the present application, Peak Signal-to-Noise Ratio (PSNR) is an indicator for measuring image quality, which represents the ratio of the maximum possible power of the image signal to the noise power that affects observation. The embodiment of the present application uses PSNR to evaluate image quality.

[0029] The calculation formula for PSNR is:

[0030] Among them, P max Indicates the maximum possible power of the signal. For an 8-bit image, P max =255; MSE is the Mean Squared Error, which represents the difference between the original image and the distorted image.

[0031] In some examples of the present application, the peak signal-to-noise ratio of the first image is greater than or equal to 30 dB, the peak signal-to-noise ratio of the second image is less than 30 dB, and the difference between the peak signal-to-noise ratios of the two is greater than or equal to 5 dB. By ensuring that there is a certain peak signal-to-noise ratio difference between the first image and the second image, the adaptability of the model when processing images with different peak signal-to-noise ratios can be enhanced. This peak signal-to-noise ratio gap provides sufficient training samples, allowing the model to learn the restoration of image details more accurately, thereby improving the image resolution effect.

[0032] In some examples of the present application, the aforementioned first image is obtained by capturing the image of the field of view, and the aforementioned second image is obtained by blurring the aforementioned first image. That is, the first image is an image actually taken, and the second image is obtained by blurring the image actually taken. By artificially creating low-resolution images, the training data is richer and more diverse. This training method enables the model to have a stronger ability to handle multiple types of blur, which can effectively improve the clarity and details of low-resolution images, thereby enhancing the effect of improving image resolution.

[0033] Among them, blur processing includes various types, such as convolution kernels of different directions, lengths, and shapes, including Gaussian, mean, radial, and rotational convolution kernels, as well as various blur levels and angles. Since the training data of this application covers a variety of blur types, levels, and directions, the trained model has the ability to improve the resolution of images limited by the diffraction limit and handle motion, out-of-focus, and deformation blur.

[0034] It is understandable that in the multiple sets of image pairs in the training data, the first images may be the same. For example, there are three sets of image pairs, the first images are the same, and the second images are images obtained by processing the first images in different blur processing methods.

[0035] Since the image actually captured may be blurred due to a variety of reasons, such as diffraction limit, artificial chip deformation, motion blur and focus problems, etc.; therefore, the second image obtained by blurring should also include the types of the aforementioned blurred images. In some examples of the present application, the blurring processing method adopted includes: convolving the aforementioned first image and the point spread function. By convolving the first image and the point spread function, the blurring phenomenon in the image can be effectively simulated and reproduced, providing more realistic training data for the subsequent image resolution enhancement model. This makes the model more accurate in processing blurred images, improves the image resolution effect, and enhances the model's ability to cope with different blur situations in practical applications.

[0036] The convolution is performed by the following steps: A. adapting the point spread function to the first image; It should be noted that the term “adaptation” means that each pixel in the first image corresponds to a specific value in the point spread function.

[0037] B. Performing matrix conversion on the pixel values ​​of each point in each area of ​​the first image and the corresponding values ​​of the point spread function to obtain the second image.

[0038] In some examples of the present application, the point spread function is selected from at least one of a Gaussian elliptic function and a rectangular function.

[0039] In some specific examples of this application, the Gaussian elliptic function formula applied is as follows: ; in, ;

[0040] In the formula, psf represents the spatial response characteristics of the optical system to a point light source. Ideally, a point light source should form a perfect point on the imaging plane after passing through the optical system. However, due to physical effects such as aberration and diffraction of the optical system, in actual imaging, the point light source will form an extended bright spot on the imaging plane; psf1 represents the degree of blur; x and y represent the pixel coordinates; x_rot and y_rot represent the coordinates obtained by rotating the original coordinates (x, y) counterclockwise around the center of the first image (center[0], center[1]); sigma_x and sigma_y represent the standard deviation of the Gaussian distribution of psf in the x and y directions; angle represents the rotation angle.

[0041] In some other specific examples of the present application, the applied rectangular function formula is as follows: psf (x, y) = 1 / L,if (x * cos(θ) + y * sin(θ)) / L<= 1; psf (x, y) =0, if else 0; Where x and y represent the pixel coordinates; θ represents the direction and angle of movement; and L represents the length of blur. At this time, 1 / L represents a numerical value, and θ can take values ​​in the range of [0, 360°]. The length of blur is used to characterize the range of the blur area, and can be represented by the horizontal, vertical or radial dimensions of the blurred part. For example, L can take values ​​in the range of [0,10]. Based on the above-mentioned Gaussian elliptic function or rectangular function, the psf matrix is ​​obtained, and the psf matrix is ​​further multiplied and summed with the first image matrix (matrix conversion) to obtain the second image pixel matrix, that is, the second image.

[0042] In some examples of this application, matrix conversion is achieved by the following formula:

[0043] Among them, lr represents the second image pixel matrix; f(m,n) represents the first image pixel matrix; g(m,n) represents the PSF matrix.

[0044] In other examples of the present application, the inventors also improve the resolution of the second image by adjusting the photographing processing conditions, wherein the photographing condition processing includes but is not limited to adjusting the scanning speed, adjusting the exposure time, or adjusting the light source intensity.

[0045] In some examples of the present application, the aforementioned first image and the second image are both obtained by capturing images of the field of view. That is, the first image and the second image are both actually captured images. The use of two actually captured images can reflect a variety of situations in actual applications, such as fluctuations in device performance, changes in the shooting environment, and the like. This method can improve the adaptability and practicality of the image resolution enhancement model, thereby enabling it to better cope with various image quality issues in the real world.

[0046] In some examples of the present application, the second image is obtained by capturing the image of the field of view by adjusting at least one of the scanning angle, the focus mode, the scanning speed, the exposure time, or the light source intensity. By adjusting the imaging parameters to obtain the second image, images with different peak signal-to-noise ratios or blur levels are directly obtained, so that the training data is more in line with the actual imaging conditions and the generalization ability of the model is improved. At the same time, this method can cover the problem of image quality degradation caused by different imaging conditions (such as insufficient light, focal length offset, scanning speed changes, etc.), so that the model can better handle various types of blurred and low-resolution images in practical applications, and improve the quality and detail retention of the final restored image.

[0047] In some examples of the present application, multiple groups of the image pairs are input into a machine learning model, the features of each second image are extracted, and the machine learning model is trained with the first image as a label to obtain the image resolution improvement model, including: according to the same image segmentation method, the first image and the second image in the image pair are segmented to form multiple first sub-regions and multiple second sub-regions, wherein the image segmentation methods of different image pairs are the same or different; for each group of the image pairs, multiple first sub-regions and multiple second sub-regions are input into the machine learning model, the features of the second sub-regions are extracted, and the machine learning model is trained with the first sub-region as a label to obtain the image resolution improvement model. By performing image segmentation on the first image and the second image, the model's adaptive ability to the blur diversity of different regions in the same image can be enhanced, so that it can effectively handle local blur problems caused by factors such as defocus, motion blur or optical distortion, thereby improving the overall image quality. In addition, image segmentation can also improve the effective utilization of training data, avoid information redundancy or uneven weight distribution when the entire image is input, make the model more efficient during training, and have better generalization ability and robustness.

[0048] It can be understood that the aforementioned same image segmentation method means that in a set of image pairs, the same image segmentation method (such as regular grid segmentation, edge detection-based segmentation, adaptive threshold segmentation, etc.) is used to segment the first image and the second image.

[0049] In some examples of the present application, the original image is input into the image resolution enhancement model to obtain a target image, including: segmenting the original image according to the image segmentation method to obtain multiple original sub-regions; inputting multiple original sub-regions into the image resolution enhancement model to output multiple target sub-regions; based on the mapping relationship before and after the image is cut in the image segmentation method, the multiple target sub-regions are reorganized into the target image. By segmenting the original image and processing multiple small regions separately, the resolution of local details can be more effectively improved, and the model's adaptive ability to the degree of blur in different regions can be enhanced. Compared with the method of inputting the entire image into the model, this method can reduce the computational burden, improve the reasoning efficiency, and avoid the degradation of model performance due to excessive image size. In addition, based on the mapping relationship of the image segmentation method, the target sub-region is reorganized into a complete target image, which can ensure the integrity of the image structure and avoid edge artifacts or information loss caused by block processing, thereby improving the overall image quality and visual consistency.

[0050] In some examples of the present application, the optically detectable biomolecule comprises an optically detectable label, wherein the optically detectable label is at least one of a fluorescent label, a chemiluminescent label, and a light scattering label.

[0051] In some examples of the present application, the features of the optically detectable mark include at least one of bright spot hue information, bright spot saturation information, bright spot pixel information, bright spot intensity information, and bright spot position information.

[0052] It can be understood that the aforementioned bright spot hue information includes the color information of the bright spot, such as the bright spot hue of the first image in one set of image pairs is green, and the bright spot hue of the first image in another set of image pairs is red; the aforementioned bright spot saturation information indicates the relative ratio of the signal intensity of the detectable mark to the background noise, such as high saturation indicates that the mark signal is strong and the background noise is low, while low saturation may indicate photobleaching.

[0053] The aforementioned bright spot pixel information may include one or more of the bright spot pixel value, pixel size, pixel spacing, pixel resolution, color mode and channel, pixel grayscale or fluorescence intensity, and pixel statistical information such as the mean and standard deviation of the pixel value.

[0054] In some examples of the present application, the machine learning model adopts a spatial attention mechanism architecture. By introducing the spatial attention mechanism architecture, it is helpful to retain the relative brightness relationship between bright spots such as fluorescent spots, thereby avoiding the relative brightness loss between bright spots such as fluorescent spots, resulting in base recognition errors. Among them, the spatial attention mechanism architecture includes a convolution layer, an attention layer, and an upsampling layer.

[0055] In some examples of the present application, the convolution layer includes multiple convolution processes. The convolution layer referred to herein includes one convolution layer or multiple convolution layers. Multiple convolution processes include performing multiple convolution processes in one convolution layer or performing multiple convolution processes in multiple convolution layers. Among them, in the multiple convolution processes performed in one or more convolution layers, at least one of the convolution processes uses at least one convolution kernel to obtain the features of the training set image, and at least one of the convolution processes uses at least one convolution kernel to reduce the number of input channels.

[0056] For example, if there are two convolutional layers, each convolutional layer performs a convolution process. In the first convolutional layer, the convolution process is used to obtain the features of the training set image using at least one convolution kernel; in the second convolutional layer, the convolution process is used to reduce the number of input channels using at least one convolution kernel.

[0057] In some examples of this application, the attention layer is used to generate a spatial attention map, and the input feature map is mapped to the attention map through a convolution operation to dynamically adjust the importance of different regions in the input image. The generated spatial attention map contains the weight information of each pixel position. The weight information is obtained through a softmax operation.

[0058] Exemplarily, the attention layer is used to generate a spatial attention map including the following steps: First, the input feature map is convolved through a convolutional layer to generate an attention map of the same size as the input feature map.

[0059] Secondly, the generated attention map is normalized by applying the "softmax" function to ensure that the sum of the weights at each position is 1. This step generates an attention map of the same size as the input feature map, and the value of each position (pixel) represents the weight of the feature at that position in terms of importance.

[0060] Finally, the input feature map is multiplied by the generated attention map to focus on the important areas in the input feature map, thereby increasing the image resolution and improving the image quality.

[0061] In some examples of the present application, an upsampling layer is used for first image reconstruction.

[0062] In some examples of the present application, the training is performed in the following manner: a. using the convolution layer to extract multiple features of the second image; b. using the attention layer to perform convolution processing on the multiple features of the second image to determine the weights of the multiple features of the second image; c. based on the weights of the multiple features, using the upsampling layer to reconstruct the second image; d. training the machine learning model based on the similarity between the reconstructed image and the first image to obtain an image resolution enhancement model.

[0063] Exemplarily, the following describes in detail the steps of segmenting-mapping-reconstructing an image using a spatial attention mechanism architecture in the machine learning model of the present application.

[0064] First convolution layer: Number of input channels: 1 (indicates grayscale image input); Number of output channels: 64; Convolution kernel size: 9x9; Padding: 4; Purpose: This convolution layer is designed for feature extraction and captures local features of the input image by applying a 9x9 convolution kernel.

[0065] Second convolution layer: number of input channels: 64; number of output channels: 32; convolution kernel size: 1x1; purpose: This convolution layer is used to reduce the number of channels of the feature map from 64 channels to 32 channels to reduce the computational complexity of the model.

[0066] Attention layer: Number of input channels: 1; Number of output channels: 1; Convolution kernel size: 1x1; Purpose: This layer is used to generate a spatial attention map, which maps the input feature map to the attention map through convolution operations to dynamically adjust the importance of different areas in the input image.

[0067] Upsampling layer: Number of input channels: 32; Number of output channels: 1; Convolution kernel size: 5x5; Padding: 2; Purpose: This layer is used for final image reconstruction, converting the 32-channel feature map into a 1-channel high-resolution image.

[0068] The spatial attention mechanism architecture first extracts features through the first and second convolutional layers and applies the ReLu activation function respectively. Then, the attention layer dynamically adjusts the importance of different regions in the feature map, and finally the upsampling layer performs the final image reconstruction, converting the feature map into a 1-channel high-resolution image.

[0069] In some examples of the present application, the machine learning model is selected from at least one of SRCNN, SRGAN and EDSR.

[0070] In some examples of the present application, training the machine learning model includes: using at least one of mean square error loss, L1 loss, and structural similarity loss to determine when to terminate the training.

[0071] Exemplarily, taking the SRCNN model as an example, the loss functions used include: mean square error loss, L1 loss and structural similarity loss.

[0072] The so-called mean square error (MSE) is used to measure the gap between the model's predicted output and the true high-resolution image. The calculation formula is as follows: MSE = (1 / N) * Σ (I_true - I_pred)^2; Among them, MSE: mean square error loss; N: number of samples; Σ: summation sign; I_true: pixel value of the real first image; I_pred: pixel value of the first image generated by the model.

[0073] The so-called L1 loss (MAE) is used to measure the absolute difference between the predicted value and the true value. The calculation formula is as follows: L1 loss = (1 / N) * Σ |I_true - I_pred|; Among them, L1 loss: mean absolute error loss; N: number of samples; Σ: summation sign; I_true: pixel value of the true high-resolution image; I_pred: pixel value of the high-resolution image generated by the model.

[0074] The so-called structural similarity loss (SSIM) is used to measure the structural similarity between two images, which takes into account three aspects of brightness, contrast and structure. The calculation formula of SSIM is as follows: SSIM = (2 * μ_true * μ_pred + C1) * (2 * σ_true * σ_pred + C2) / ((μ_true^2 + μ_pred^2 + C1) * (σ_true^2 + σ_pred^2 + C2)); Among them, SSIM: structural similarity index; μ_true, μ_pred: mean brightness of the real image and the predicted image; σ_true, σ_pred: standard deviation of brightness of the real image and the predicted image; C1, C2: stability constants, usually used to avoid the situation where the denominator is zero.

[0075] Those skilled in the art know that a high SSIM value indicates that two images are more similar in structure. Therefore, the goal of SSIM loss is to maximize the SSIM value and minimize 1-SSIM.

[0076] In addition to using the above loss function, the inventors also added perceptual loss to solve the problem of losing the brightness difference between points. The main purpose of introducing perceptual loss in image generation tasks is to guide the generated images to be closer to the real images by using a pre-trained deep learning model (usually a convolutional neural network such as VGG16) to measure the feature similarity between the generated images and the real images.

[0077] Exemplarily, the specific steps of introducing perceptual loss into the loss function include: Create a pre-trained VGG model: First, choose a suitable deep learning model, usually a model pre-trained on a large-scale image dataset, such as VGG16. These models have powerful feature extraction capabilities for tasks such as image classification. The pre-trained model contains multiple convolutional layers and pooling layers to extract features of the input image hierarchically.

[0078] Feature extraction: In the forward propagation of the loss function, both the generated image (usually referred to as x) and the real image (usually referred to as y) are passed through the pre-trained VGG model to extract their feature representations. Usually one or more layers of the pre-trained model (usually the first 16 layers are selected, which can also be adjusted according to the task) are selected as feature extractors. These layers are usually located at the top of the model and can capture higher-level feature information.

[0079] Calculate MSE loss (mean squared error loss): The mean squared error (MSE) between the feature representations of the generated image and the real image is calculated as the perceptual loss. MSE loss measures the difference between the two in the feature space. The feature representation is usually a high-dimensional tensor, and the MSE loss is calculated by squaring the difference on each feature channel and summing over all channels. The goal is to make the generated image more similar to the real image in the feature space by minimizing the MSE loss.

[0080] Adding perceptual loss: Perceptual loss is added to the overall loss function. The overall loss function usually includes other loss terms, such as pixel-level mean squared error loss or adversarial loss. In some examples of this application, the introduction of perceptual loss helps to generate images with better visual quality and perceptual consistency.

[0081] In some examples of the present application, the determination of the termination timing includes: obtaining the loss function of N consecutive model trainings, and comparing the obtained loss functions. If, in the N model trainings, the loss function obtained in the latter model training does not decrease or even increases compared to the loss function obtained in the previous model training, then the training of the machine learning model is terminated, where N is a natural number greater than or equal to 5. By monitoring the changes in the loss function, overtraining can be effectively avoided, computing resources can be saved, and training efficiency can be improved. At the same time, this judgment method based on the trend of the loss function helps the model avoid falling into a local minimum, thereby ensuring that the training process is more robust and the final model performance is more optimized.

[0082] In some examples of the present application, the optically detectable biomolecule is a nucleic acid molecule with an optically detectable label. That is, the image resolution enhancement method of the present application is selected from image processing in the field of gene sequencing. It is understandable that the method of the present application can also be applied to medical image processing, etc.

[0083] In some examples of the present application, the data set used for model training of the present application includes a test set image, and the test set image also includes a first image and a second image, and the second image is obtained by blurring the first image and / or by capturing the image of the field of view. The test set image is used to verify the performance of the trained machine learning model. It should be noted that the test set image is independent of the training set image.

[0084] In some examples of the present application, performance verification of the above-mentioned trained machine learning model is further included.

[0085] Exemplarily, at least one of the methods (1) to (2) is used to evaluate the performance of the model: (1) Perform visual comparison to compare the differences between the image processed by the model and the original image to visually evaluate the image restoration quality.

[0086] (2) Perform image processing-based evaluations, such as base recognition and alignment, and calculate quantitative indicators such as alignment rate and error rate to evaluate the performance of the model on specific tasks.

[0087] In another aspect of the present application, the present application proposes a system for improving image resolution. Figure 2 The system 200 includes: an original image acquisition module 210 and a target image acquisition module 220, wherein: The original image acquisition module 210 is used to acquire the original image obtained based on the biomolecule microscopic detection; The target image acquisition module 220 is used to input the original image into the image resolution enhancement model to obtain the target image; wherein the image resolution enhancement model is obtained by the following method: obtaining multiple groups of image pairs, the multiple groups of image pairs are image pairs obtained from images of at least one field of view acquired by a microscopic imaging system at at least two time points, the field of view contains multiple optically detectable biological molecules, and one group of image pairs includes a first image and a second image of the same field of view acquired at the same time point, and the peak signal-to-noise ratio of the first image is higher than the peak signal-to-noise ratio of the second image; inputting the multiple groups of image pairs into the machine learning model, extracting the features of each of the second images, and training the machine learning model with the first image as a label to obtain the image resolution enhancement model.

[0088] In some examples of the present application, the peak signal-to-noise ratio of the first image is greater than or equal to 30 dB, the peak signal-to-noise ratio of the second image is less than 30 dB, and the difference between the peak signal-to-noise ratios of the two is greater than or equal to 5 dB.

[0089] In some examples of the present application, the first image is obtained by capturing the field of view, and the second image is obtained by blurring the first image.

[0090] In some examples of the present application, the blurring process includes: convolving the first image with a point spread function.

[0091] In some examples of the present application, the convolution is performed by the following steps: A. adapting the point spread function to the first image; B. performing matrix conversion between the pixel values ​​of each point in each target area of ​​the first image and the corresponding values ​​of the point spread function to obtain a second image.

[0092] In some examples of the present application, the point spread function is selected from at least one of a Gaussian elliptic function and a rectangular function.

[0093] In some examples of the present application, the Gaussian elliptic function formula is as follows: ; in, ;

[0094] In the formula, psf represents the spatial response characteristics of the optical system to a point light source; psf1 represents the degree of blur; x and y represent the pixel coordinates; x_rot and y_rot represent the coordinates obtained by rotating the original coordinates (x, y) counterclockwise around the center of the first image (center[0], center[1]); sigma_x and sigma_y represent the standard deviation of the Gaussian distribution of psf in the x and y directions; angle represents the rotation angle.

[0095] In some examples of this application, the rectangular function formula is as follows: psf (x, y) = 1 / L,if (x * cos(θ) + y * sin(θ)) / L<= 1; psf (x, y) =0, if else; Where x and y represent the pixel coordinates; θ represents the direction and angle of motion; and L represents the length of blur. At this time, 1 / L represents a value, and θ can be in the range of [0, 360°]. The length of blur is used to characterize the range of the blur area, and can be represented by the horizontal, vertical, or radial dimensions of the blurred part. For example, L can be in the range of [0,10].

[0096] In some examples of the present application, the matrix conversion is implemented by the following formula:

[0097] Among them, lr represents the second image pixel matrix; f(m,n) represents the first image pixel matrix; g(m,n) represents the PSF matrix.

[0098] In some examples of the present application, both the first image and the second image are obtained by capturing images of the field of view.

[0099] In some examples of the present application, the second image is obtained by capturing the image of the field of view by adjusting at least one of a scanning angle, a focusing method, a scanning speed, an exposure time, or a light source intensity.

[0100] In some examples of the present application, multiple groups of image pairs are input into a machine learning model, features of each second image are extracted, and the machine learning model is trained with the first image as a label to obtain the image resolution improvement model, including: according to the same image segmentation method, the first image and the second image in the image pair are segmented to form multiple first sub-regions and multiple second sub-regions, wherein the image segmentation methods of different image pairs are the same or different; for each group of image pairs, multiple first sub-regions and multiple second sub-regions are input into the machine learning model, features of the second sub-regions are extracted, and the machine learning model is trained with the first sub-region as a label to obtain the image resolution improvement model.

[0101] In some examples of the present application, the original image is input into the image resolution enhancement model to obtain a target image, including: segmenting the original image according to the image segmentation method to obtain multiple original sub-regions; inputting the multiple original sub-regions into the image resolution enhancement model to output multiple target sub-regions; and recombining the multiple target sub-regions into the target image based on the mapping relationship before and after cutting the image in the image segmentation method.

[0102] In some examples of the present application, the optically detectable biomolecule comprises an optically detectable label, wherein the optically detectable label is at least one of a fluorescent label, a chemiluminescent label, and a light scattering label.

[0103] In some examples of the present application, the features of the optically detectable mark include at least one of bright spot hue information, bright spot saturation information, bright spot pixel information, bright spot intensity information, and bright spot position information.

[0104] In some examples of the present application, the machine learning model adopts a spatial attention mechanism architecture.

[0105] In some examples of the present application, the spatial attention mechanism architecture includes a convolution layer, an attention layer and an upsampling layer, and the training is performed in the following manner: a. using the convolution layer to extract multiple features of the second image; b. using the attention layer to perform convolution processing on the multiple features of the second image to determine the weights of the multiple features of the second image; c. based on the weights of the multiple features, using the upsampling layer to reconstruct the second image; d. training the machine learning model based on the similarity between the reconstructed image and the first image to obtain an image resolution enhancement model.

[0106] In some examples of the present application, the machine learning model is selected from at least one of SRCNN, SRGAN and EDSR.

[0107] In some examples of the present application, training the machine learning model includes: using at least one of mean square error loss, L1 loss, and structural similarity loss to determine when to terminate the training.

[0108] In some examples of the present application, the determination of the termination timing includes: obtaining the loss function of N consecutive model trainings, and comparing the obtained loss functions. If, among the N model trainings, the loss function obtained from the latter model training does not decrease or even increases compared with the loss function obtained from the previous model training, then the training of the machine learning model is terminated, where N is a natural number greater than or equal to 5.

[0109] In some examples of the present application, the optically detectable biomolecule is a nucleic acid molecule with an optically detectable label.

[0110] It should be understood that the system embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 2 The system 200 shown may perform Figure 1 The corresponding method embodiments, and the aforementioned and other operations and / or functions of each module in the system 200 are respectively to implement Figure 1 For the sake of brevity, the corresponding processes in each method are not repeated here.

[0111] The system 200 of the embodiment of the present application is described above from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0112] In yet another aspect of the present application, the present application provides an electronic device. Figure 3 The electronic device 300 may be a device for executing the above method, but is not limited thereto. Figure 3 As shown, the electronic device 300 may include: The memory 310 and the processor 320, the memory 310 is used to store the computer program 330 and transmit the computer program 330 to the processor 320. In other words, the processor 320 can call and run the computer program 330 from the memory 310 to implement the method in the embodiment of the present application.

[0113] For example, the processor 320 may be configured to execute the steps in the above method according to the instructions in the computer program 330 .

[0114] In some embodiments of the present application, the processor 320 may include but is not limited to: General-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0115] In some embodiments of the present application, the memory 310 includes but is not limited to: Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0116] In some embodiments of the present application, the computer program 330 may be divided into one or more modules, which are stored in the memory 310 and executed by the processor 320 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 330 in the electronic device.

[0117] like Figure 3 As shown, the electronic device 300 may further include: The transceiver 340 may be connected to the processor 320 or the memory 310 .

[0118] The processor 320 may control the transceiver 340 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 340 may include a transmitter and a receiver. The transceiver 340 may further include an antenna, and the number of antennas may be one or more.

[0119] It should be understood that the various components in the electronic device 300 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0120] According to one aspect of the present application, a computer-readable storage medium is provided, on which a computer instruction or program is stored, and when the computer instruction or program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instruction is executed by a computer, the computer can perform the method of the above method embodiment.

[0121] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of the above method embodiment.

[0122] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0123] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0124] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0125] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0126] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0127] The scheme of the present application will be explained below in conjunction with the examples. Those skilled in the art will understand that the following examples are only used to illustrate the present application and should not be regarded as limiting the scope of the present application.

[0128] Example 1: Model effect evaluation Data source: The training data image size is 580*650 pixels. A total of 5000 pairs of training data are used, which are the first and second image pairs obtained from sequencing. The images in the image pairs are all images containing fluorescent spots, of which 3000 pairs are from actual photography, and 2000 pairs are from constructing different PSF blur functions to blur the first image.

[0129] The model training was performed using the method in the specific implementation method section of this application, and the model evaluation results obtained were as follows: 1. Visually compare the original image and the processed image with different blur levels Depend on Figure 4~Figure 6 It can be seen that compared with the original image, the resolution of the processed image is significantly improved, the points are significantly smaller, and the separation effect between bright spots is significantly improved.

[0130] 2. The effect of base recognition is shown in Table 1. It can be seen that for images with different blur levels and blur directions, the processed images have improved alignment rates and significantly reduced error rates compared to the original images, especially in the upper right and lower right corners where the deformation is more obvious. The error rate is still greatly reduced when the alignment rate is significantly improved. Among them, the error rate is used to evaluate the quality value of the accuracy of the base recognition results, reflecting the credibility of base recognition during the sequencing process. The smaller the error rate, the higher the credibility of the base recognition results; the larger the error rate, the lower the credibility of the base recognition results. The alignment rate refers to the percentage of sequencing sequences that are aligned with the sample standard sequence.

[0131] Table 1 Comparison results of base recognition effects

[0132] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0133] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in the field can change, modify, replace and modify the above embodiments within the scope of the present application without departing from the principles and purpose of the present application.

Claims

1. A method for improving image resolution, characterized in that: The steps include: Acquire original images based on biomolecular microscopic detection; Inputting the original image into an image resolution enhancement model to obtain a target image; The image resolution enhancement model is obtained by the following method: Acquire a plurality of sets of image pairs, wherein the plurality of sets of image pairs are image pairs obtained from images of at least one field of view acquired by a microscopic imaging system at at least two time points, wherein the field of view includes a plurality of optically detectable biomolecules, and a set of the image pairs includes a first image and a second image of the same field of view acquired at the same time point, and a peak signal-to-noise ratio of the first image is higher than a peak signal-to-noise ratio of the second image; Input multiple groups of image pairs into the machine learning model, extract the features of each second image, and train the machine learning model with the first image as a label to obtain the image resolution enhancement model.

2. The method according to claim 1, characterized in that The peak signal-to-noise ratio of the first image is greater than or equal to 30 dB, the peak signal-to-noise ratio of the second image is less than 30 dB, and a difference between the peak signal-to-noise ratios of the two is greater than or equal to 5 dB.

3. The method according to claim 2, characterized in that The first image is obtained by collecting images of the field of view, and the second image is obtained by blurring the first image.

4. The method according to claim 3, characterized in that The blurring process includes: convolving the first image with a point spread function.

5. The method according to claim 4, characterized in that The convolution is performed by the following steps: A. adapting the point spread function to the first image; B. Performing matrix conversion between the pixel values ​​of each point in each target area of ​​the first image and the corresponding value of the point spread function to obtain a second image.

6. The method according to claim 5, characterized in that The point spread function is selected from at least one of a Gaussian elliptic function and a rectangular function.

7. The method according to claim 6, characterized in that The Gaussian elliptic function formula is as follows: ； in, ； In the formula, psf represents the spatial response characteristics of the optical system to a point light source; psf1 indicates the degree of blur; x and y represent the pixel coordinates; x_rot and y_rot represent the coordinates obtained by rotating the original coordinates (x, y) counterclockwise around the center of the first image (center[0], center[1]); sigma_x and sigma_y represent the standard deviation of the Gaussian distribution of the psf in the x and y directions; angle indicates the rotation angle.

8. The method according to claim 6, characterized in that The rectangular function formula is as follows: psf (x, y) = 1 / L,if (x * cos(θ) + y * sin(θ)) / L <= 1; psf (x, y) = 0, if else 0; in, x and y represent the pixel coordinates; θ represents the direction and angle of movement; L represents the length of the blur.

9. The method according to claim 5, characterized in that The matrix conversion is achieved by the following formula: in, lr represents the second image pixel matrix; f(m,n) represents the first image pixel matrix; g(m,n) represents the psf matrix.

10. The method according to claim 2, characterized in that The first image and the second image are both obtained by capturing images of the field of view.

11. The method according to claim 10, characterized in that The second image is obtained by capturing the image of the field of view by adjusting at least one of a scanning angle, a focusing mode, a scanning speed, an exposure time, and a light source intensity.

12. The method according to any one of claims 1 to 11, characterized in that: Inputting a plurality of groups of the image pairs into a machine learning model, extracting features of each second image, and training the machine learning model with the first image as a label to obtain the image resolution enhancement model, including: According to the same image segmentation method, the first image and the second image in the image pair are segmented to form a plurality of first sub-regions and a plurality of second sub-regions, wherein the image segmentation methods of different image pairs are the same or different; For each group of the image pairs, multiple first sub-regions and multiple second sub-regions are input into the machine learning model, the features of the second sub-regions are extracted, and the machine learning model is trained with the first sub-regions as labels to obtain an image resolution enhancement model.

13. The method according to claim 12, characterized in that Inputting the original image into the image resolution enhancement model to obtain a target image comprises: According to the image segmentation method, the original image is segmented to obtain a plurality of original sub-regions; Inputting the plurality of original sub-regions into the image resolution enhancement model, and outputting a plurality of target sub-regions; Based on the mapping relationship before and after the image is cut in the image segmentation method, the plurality of target sub-regions are reorganized into the target image.

14. The method according to claim 1, characterized in that The optically detectable biomolecule comprises an optically detectable label.

15. The method according to claim 14, characterized in that The optically detectable label is at least one of a fluorescent label, a chemiluminescent label, and a light scattering label.

16. The method according to claim 14, characterized in that The characteristics of the optically detectable mark include at least one of bright spot hue information, bright spot saturation information, bright spot pixel information, bright spot intensity information and bright spot position information.

17. The method according to claim 14, characterized in that The machine learning model adopts a spatial attention mechanism architecture.

18. The method according to claim 17, characterized in that The spatial attention mechanism architecture includes convolutional layers, attention layers, and upsampling layers. The training is carried out in the following manner: a. extracting a plurality of features of the second image using the convolutional layer; b. performing convolution processing on the multiple features of the second image using the attention layer to determine the weights of the multiple features of the second image; c. reconstructing the second image using the upsampling layer based on the weights of the respective features; d. Training the machine learning model based on the similarity between the reconstructed image and the first image to obtain an image resolution enhancement model.

19. The method according to any one of claims 14 to 18, characterized in that: The machine learning model is selected from at least one of SRCNN, SRGAN and EDSR.

20. The method according to claim 19, characterized in that Training the machine learning model includes: using at least one of mean square error loss, L1 loss, and structural similarity loss to determine when to terminate the training.

21. The method according to claim 20, characterized in that The determination of the termination timing includes: obtaining the loss function of N consecutive model trainings, and comparing the obtained loss functions. If, among the N model trainings, the loss function obtained in the latter model training does not decrease or even increases compared with the loss function obtained in the previous model training, then the training of the machine learning model is terminated, where N is a natural number greater than or equal to 5.

22. The method according to claim 1, characterized in that The optically detectable biomolecule is a nucleic acid molecule with an optically detectable label.

23. A system for improving image resolution, characterized in that: include: An original image acquisition module, used to acquire original images obtained based on biomolecule microscopic detection; A target image acquisition module, used for inputting the original image into an image resolution enhancement model to obtain a target image; The image resolution enhancement model is obtained by the following method: Acquire a plurality of sets of image pairs, wherein the plurality of sets of image pairs are image pairs obtained from images of at least one field of view acquired by a microscopic imaging system at at least two time points, wherein the field of view includes a plurality of optically detectable biomolecules, and a set of the image pairs includes a first image and a second image of the same field of view acquired at the same time point, and a peak signal-to-noise ratio of the first image is higher than a peak signal-to-noise ratio of the second image; Input multiple groups of image pairs into the machine learning model, extract the features of each second image, and train the machine learning model with the first image as a label to obtain the image resolution enhancement model.

24. An electronic device, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 22.

Images