Super-resolution image generation method and device, equipment and medium

Abstract

The invention discloses a super-resolution image generation method and device, equipment and a medium, and relates to the field of optical microscopic imaging, and the method comprises the steps: obtaining an original image sequence of a sample; preprocessing the original image sequence to obtain multiple frames of preprocessed images; carrying out adaptive threshold processing and region screening processing on each frame of preprocessed image, and determining a plurality of light spot regions; traversing all light spot areas of each frame of preprocessed image, extracting each pixel data of each light spot area, performing Gaussian fitting processing on each pixel data, and determining a corresponding light spot center position; generating a Gaussian pinhole on the blank image according to the central position of the light spot, and performing dot multiplication processing on the Gaussian pinhole and the original image sequence to obtain a dot-multiplied image; carrying out superposition fusion and deconvolution processing on all the point-multiplied images to generate a super-resolution image; the resolution of the super-resolution image is higher than that of the image after dot multiplication. According to the method, the signal-to-noise ratio of the reconstructed image is remarkably reduced, and the reconstruction speed and the imaging quality are improved.

Description

Technical Field

[0001] This application relates to the field of optical microscopy imaging technology, and in particular to a method, apparatus, device and medium for generating super-resolution images. Background Technology

[0003] In the life sciences, optical super-resolution microscopy is a core tool for analyzing microscopic structures. Structured Illumination Microscopy (SIM), as a key branch of this technology, plays a crucial role in improving imaging resolution. Among them, Multifocal Structured Illumination Microscopy (MSIM), with its larger imaging field of view and deeper imaging depth, has become a highly promising direction in SIM technology. MSIM uses multiple diffraction-limited Gaussian light spots as structured light, which can be regarded as a parallel form of image scanning microscopy (ISM), thereby acquiring the original image sequence of the sample. The original image sequence is then processed by reconstruction algorithms to obtain multifocal super-resolution images. To provide higher-quality imaging support for analyzing subcellular structures and observing microscopic dynamic processes in life sciences and other fields, research on how to generate multifocal super-resolution images is particularly important.

[0004] Currently, related technologies utilize the lattice light field acquired through imaging of thick samples to extract lattice vectors, offset vectors, and movement vectors, thereby reconstructing a digital pinhole. This digital pinhole is then applied to a sequence of sample images to filter out noise signals, resulting in a pixel relocation image sequence, which is subsequently processed into a super-resolution image. However, this approach requires additional high-quality calibration light field imaging to aid in localization before reconstruction, which is cumbersome. Furthermore, when the light field noise is high, it cannot distinguish between noise spots and sample spots, processing both together. This ultimately leads to a significant reduction in the signal-to-noise ratio of the reconstructed image, affecting imaging quality. Summary of the Invention

[0006] The purpose of this application is to provide a super-resolution image generation method, apparatus, device, and medium to solve the technical problem of significantly reduced signal-to-noise ratio of reconstructed images, which affects imaging quality.

[0007] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for generating super-resolution images, including: Obtain the original image sequence of the sample; The original image sequence is preprocessed to obtain multiple preprocessed images; Adaptive threshold processing and region screening processing are performed on each of the preprocessed images to determine a plurality of light spot regions; the light spot region refers to a region in which sample Gaussian light spot pixels are located; Each of the light spot regions of each frame of the preprocessed images is traversed to extract respective pixel data of each of the light spot regions, and Gaussian fitting processing is performed on the respective pixel data to determine a corresponding light spot center position; A Gaussian pinhole corresponding to the light spot center position is generated on a blank image according to the light spot center position, and point multiplication processing is performed on the blank image and the original image sequence to obtain a point-multiplied image; the resolution of the point-multiplied image is higher than that of the images in the original image sequence; the size of the Gaussian pinhole is half of the size of the light spot region; Super-resolution image generation module, for superposition fusion and deconvolution processing are performed on all point-multiplied images to generate a super-resolution image; the resolution of the super-resolution image is higher than that of the point-multiplied image.

[0008] In a second aspect, the present application provides a super-resolution image generation device, which comprises: An acquisition module is configured to acquire an original image sequence of a sample; A preprocessing module is configured to preprocess the original image sequence to obtain a plurality of preprocessed images; A light spot region determination module is configured to perform adaptive threshold processing and region screening processing on each of the preprocessed images to determine a plurality of light spot regions; the light spot region refers to a region in which sample Gaussian light spot pixels are located; A center position determination module is configured to traverse all light spot regions of each frame of the preprocessed images to extract respective pixel data of each of the light spot regions, and perform Gaussian fitting processing on the respective pixel data to determine a corresponding light spot center position; A point-multiplied image generation module is configured to generate a Gaussian pinhole corresponding to the light spot center position on a blank image according to the light spot center position, and perform point multiplication processing on the blank image and the original image sequence to obtain a point-multiplied image; the resolution of the point-multiplied image is higher than that of the images in the original image sequence; the size of the Gaussian pinhole is half of the size of the light spot region; A super-resolution image generation module is configured to perform superposition fusion and deconvolution processing on all point-multiplied images to generate a super-resolution image; the resolution of the super-resolution image is higher than that of the point-multiplied image.

[0009] In a third aspect, the present application provides a computer device, which comprises a memory, a processor, a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the steps of the super-resolution image generation method of any one of the above aspects.

[0010] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the super-resolution image generation method in any of the above.

[0011] According to the specific embodiments provided in the present application, the following technical effects are disclosed: The present application provides a super-resolution image generation method, device, equipment and medium, the method comprising: acquiring an original image sequence of a sample; preprocessing the original image sequence to obtain a plurality of preprocessed images; performing adaptive threshold processing and region screening processing on each preprocessed image to determine a plurality of light spot regions; the light spot region refers to a region where a sample Gaussian light spot pixel is located; traversing all light spot regions of each preprocessed image, extracting each pixel data of each light spot region, performing Gaussian fitting processing on each pixel data to determine a corresponding light spot center position; generating a Gaussian pinhole corresponding to the light spot position on a blank image according to the light spot center position, and performing point multiplication processing on the Gaussian pinhole and the original image sequence to obtain a point multiplied image; the resolution of the point multiplied image is higher than the resolution of the image in the original image sequence; the size of the Gaussian pinhole is half of the size of the light spot region; superimposing and fusing all the point multiplied images and performing deconvolution processing to generate a super-resolution image; the resolution of the super-resolution image is higher than the resolution of the point multiplied image.

[0012] Compared with the prior art, in the present application, the original image sequence of the sample is acquired, without the need for additional shooting of the high-quality calibration light field required by the traditional scheme, and the processing is directly based on the sample data, which greatly simplifies the operation process and reduces the experimental time; the original image sequence is preprocessed to eliminate the direct current component interference, providing high-contrast data support for subsequent light spot identification, and avoiding the background noise from covering the light spot details; adaptive threshold processing and region screening are performed on the preprocessed image, which can accurately distinguish the sample light spot regions under uneven illumination in the multi-focus Gaussian light field, and avoid misjudgment problems by using traditional global threshold; traversing the light spot regions and extracting the pixel data for Gaussian fitting to determine the light spot center can filter out noise light spot interference, accurately locate the effective sample light spot, avoid positioning deviation caused by noise affecting vector estimation, and improve the subsequent processing accuracy; according to the light spot center, a Gaussian pinhole with a size of half of the light spot is generated on a blank image and multiplied with the original sequence, which can focus on the high signal-to-noise ratio core region of the light spot and suppress redundant noise, directly preliminarily improve the image resolution, and without the need for the complex steps of traditional pixel relocation, simplifying the process and reducing the calculation amount; superimposing and fusing all the point multiplied images and performing deconvolution processing can further integrate multiple frames of effective information and break through the diffraction limit, finally generating a super-resolution image with a resolution much higher than that of the original image and the point multiplied image, significantly reducing the signal-to-noise ratio of the reconstructed image and improving the imaging quality. BRIEF DESCRIPTION OF DRAWINGS

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the application environment of a super-resolution image generation method according to an embodiment of this application; Figure 2 A flowchart illustrating a super-resolution image generation method provided in an embodiment of this application; Figure 3 A flowchart illustrating a method for preprocessing an original image sequence to obtain multiple preprocessed images, provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the process of generating a super-resolution image using a reconstruction algorithm, as provided in an embodiment of this application; Figure 5 (a) A schematic diagram of the PSF process of a PSF simulation system based on an embodiment of this application; Figure 5 (b) A dot plot used to simulate a computer device provided in an embodiment of this application; Figure 5 (c) A schematic diagram of one of the original image sequences simulated by a computer device according to an embodiment of this application; Figure 5 (d) is a schematic diagram of a real sample image simulated by a computer device according to an embodiment of this application; Figure 5 (e) A wide-field image in a computer device simulation provided in an embodiment of this application; Figure 5 (f) is a schematic diagram of a super-resolution image provided in an embodiment of this application; Figure 6 (a) A schematic diagram of an open-source wide-field image provided in an embodiment of this application; Figure 6 (b) A reconstructed graph obtained by processing open-source data using the conventional MSIM algorithm provided in an embodiment of this application; Figure 6 (c) is a schematic diagram of a super-resolution image obtained by processing open-source data using the GSRMRL algorithm provided in an embodiment of this application; Figure 6 (d) A method provided for an embodiment of this application Figure 6 (a) to Figure 6 (c) The Gaussian function fitting curve of the underlined part; Figure 7 A functional module schematic diagram of an ultra-resolution image generation device provided by an embodiment of the present application is provided. Figure 8 A structural schematic diagram of a computer device provided by an embodiment of the present application is provided. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0017] The above purposes, features and advantages of the present application can be more obvious and easy to understand. The present application will be described in further detail below with reference to the drawings and specific embodiments.

[0018] In the related art, a lattice light field obtained by shooting a thick sample is used to extract a lattice vector, an offset vector and a movement vector, and then a digital pinhole is reconstructed. The digital pinhole is applied to a sample image sequence to filter out noise signals, obtain a pixel relocation image sequence, and finally generate an ultra-resolution image through fusion and deconvolution processing. However, on the one hand, the acquisition of the lattice vector depends on the regular distribution of the Fourier spectrum of the periodic signal, and the noise signal will interfere with the harmonic component after Fourier transform, resulting in a decrease in the signal-to-noise ratio of the light field, and then affecting the accurate estimation of the lattice vector, causing the center of the light spot to be deviated. On the other hand, the algorithm needs to additionally shoot a high-quality calibration light field to assist positioning before reconstruction, which is complicated to operate, and when the light field noise is large, the traditional algorithm cannot distinguish between noise spots and sample spots, and will process them together, which will eventually lead to a significant decrease in the signal-to-noise ratio of the reconstructed image, affecting the imaging quality.

[0019] Based on the above defects, the application provides an ultrahigh-resolution image generation method. Compared with the prior art, in the present scheme, the original image sequence of the sample is obtained, without the need for additional shooting of the high-quality calibration light field required in the traditional scheme, and the processing is directly based on the sample data, greatly simplifying the operation process and reducing the experimental time consumption; the original image sequence is preprocessed to eliminate the interference of the direct current component and provide high-contrast data support for subsequent spot identification, avoiding the masking of spot details by background noise; the preprocessed image is subjected to adaptive threshold processing and region screening, which can accurately distinguish the sample spot region under uneven illumination in the multi-focus Gaussian light field, avoiding misjudgment problems by traditional global threshold; the spot region is traversed and the pixel data is extracted for Gaussian fitting to determine the spot center, which can filter out noise spot interference, accurately locate the effective sample spot, avoid positioning deviation caused by noise influence on vector estimation in the traditional scheme, and improve the subsequent processing accuracy; the original image sequence is subjected to Gaussian pinhole with half of the spot and point multiplication with the original sequence, which can focus the core region of the spot with high signal-to-noise ratio and suppress redundant noise, directly preliminarily improve the image resolution, and does not need the complex steps of traditional pixel relocation, simplifying the process and reducing the calculation amount; all the point-multiplication images are superimposed and fused and subjected to deconvolution processing, further integrating multiple-frame effective information and breaking through the diffraction limit, finally generating an ultrahigh-resolution image with a resolution much higher than that of the original image and the image after point multiplication, significantly reducing the signal-to-noise ratio of the reconstructed image and improving the imaging quality.

[0020] The ultrahigh-resolution image generation method provided by the embodiments of the application can be applied to the application environment of the ultrahigh-resolution image generation method as shown in the figure. Figure 1 The application environment includes a terminal 102, a server 104 and a data storage system. The terminal 102 communicates with the server 104 through a network. The data storage system can store the original image sequence of the sample obtained by the server 104. The data storage system can be separately arranged, integrated on the server 104, placed on the cloud or other servers. The terminal 102 can send the obtained original image sequence of the sample to the server 104, and the server 104 generates an ultrahigh-resolution image after determining the spot region, the spot center position and other processes after obtaining the original image sequence of the sample. In addition, in some embodiments, the ultrahigh-resolution image generation method can also be realized by the server 104 or the terminal 102 alone, such as the terminal 102 directly determining the spot region, the spot center position and other processes to generate an ultrahigh-resolution image.

[0021] The terminal 102 can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented by a standalone server or a server cluster composed of multiple servers, and can also be a cloud server.

[0022] In an exemplary embodiment, as shown in Figure 2 An ultrahigh-resolution image generation method is provided, which is executed by a computer device, specifically, can be executed by a terminal or a server, or both. In the embodiments of the present application, the method is applied to the server 104 in the system 100, and includes the following steps S201 to S206. Figure 1 Step S201: Obtain an original image sequence of a sample.

[0023] It should be noted that the original image sequence of the sample refers to a set of multiple two-dimensional images containing micro information of the sample and noise, which are captured by a camera synchronously in an MSIM system by sequentially projecting multiple frames of Gaussian light spot array images by a digital micro-mirror device.

[0024] Specifically, in the process of obtaining the original image sequence of the sample, the multi-focus structured light illumination microscope (MSIM) system is relied on. First, the imaging parameters are set according to the characteristics of the sample to be observed, and then the digital micro-mirror device (DMD) of the system generates and sequentially projects multiple frames of Gaussian light spot array images. Each time a spot array image is projected, the camera works synchronously with the digital micro-mirror device to capture an image of the sample under the current spot array illumination. After all the spot array images are projected and collected, the original image sequence composed of multiple two-dimensional sample images is obtained. The adaptive imaging parameters can include excitation light wavelength, detection light wavelength, objective numerical aperture, etc. For example, the excitation wavelength can be set to 488 nm, the detection wavelength can be set to 520 nm, and the objective numerical aperture can be set to 1.45.

[0025] ​The original image sequence contains not only microscopic structure information of the sample, but also inevitably mixes in system inherent noise, environmental interference signals and uniform background light from the data attribute. Due to the Gaussian distribution characteristics of the multi-focus point array illumination, there are differences in the light field brightness of different frames and different regions of the same frame in the original image sequence. The microscopic structure information may be, for example, the fluorescence intensity distribution of different regions of the sample, corresponding to the morphology, position characteristics of the sample. The system inherent noise may be, for example, camera dark current noise, light source fluctuation noise. The environmental interference signal may be stray light.

[0026] In this embodiment, by obtaining the original image sequence of the sample, data guidance information can be provided for subsequent preprocessing, spot recognition, center positioning and other steps. Compared with the traditional algorithm which needs to additionally shoot a calibration light field, this scheme can directly take the original image sequence of the sample as the core data, without the need for additional acquisition of calibration data, thus simplifying the experimental operation and reducing data redundancy from the beginning of the process, while avoiding error transmission caused by differences between the calibration light field and the sample imaging environment, and laying a data foundation for subsequent high signal-to-noise ratio reconstruction without calibration light field.

[0027] In step S202, the original image sequence is preprocessed to obtain a plurality of preprocessed images.

[0028] It can be understood that, since the original image sequence includes part of the noise interference, in order to improve the subsequent spot positioning accuracy, it is necessary to filter out the noise interference in the original image sequence, that is, to eliminate the interference of the direct current component.

[0029] In the preprocessing of the original image sequence, the original image sequence can be subjected to three-dimensional Fourier transform processing to obtain a corresponding three-dimensional frequency domain sequence. The first frame of the direct current component common to all images in the three-dimensional frequency domain sequence is set to zero to obtain a processed three-dimensional frequency domain sequence. The processed three-dimensional frequency domain sequence is subjected to inverse Fourier transform processing to obtain a plurality of preprocessed images.

[0030] In this embodiment, after obtaining the original image sequence, the original image sequence is first subjected to three-dimensional Fourier transform processing to convert the original image sequence from the spatial domain to the frequency domain. The original image sequence is "space-time" dimensional data, that is, three-dimensional data composed of a plurality of two-dimensional images. After three-dimensional Fourier transform, it is converted into a "frequency-time" dimensional three-dimensional frequency domain sequence. The first frame of the frequency domain sequence corresponds to the direct current component common to all images, that is, the background signal with constant brightness in the image, such as system inherent noise, uniform background light, etc. Such signals will mask the spot details and affect the subsequent spot region recognition and positioning.

[0031] The three-dimensional frequency domain sequence is directly processed by setting the first frame signal value corresponding to the direct current component in the frequency domain sequence to zero, thereby obtaining the processed three-dimensional frequency domain sequence. The uniform background and constant noise in the original image that do not need to be reserved can be accurately removed, and interference of the uniform background and constant noise on subsequent adaptive threshold processing for screening the light spot region is avoided. If the direct current component exists, the light field brightness reference will be offset, and the threshold value will be incorrectly determined to identify the background or noise as the light spot. After obtaining the processed three-dimensional frequency domain sequence, inverse three-dimensional Fourier transform processing is performed on the processed three-dimensional frequency domain sequence, so as to return to the spatial domain and obtain the preprocessed image. The inverse transform is performed on the three-dimensional frequency domain sequence after the direct current component is set to zero, and the processed frequency domain signal is converted into an image sequence in the "space-time" dimension, that is, the preprocessed image.

[0032] In this step, the original image sequence is preprocessed. On the one hand, the three-dimensional Fourier transform can convert the original image sequence in the "space-time" dimension into frequency domain data in the "frequency-time" dimension, accurately separate the direct current component (that is, the uniform background light, system constant noise and other redundant signals) shared by all images, and avoid the superposition of the direct current component and the sample light spot information in the spatial domain. On the other hand, by setting the direct current component to zero and inversely transforming it back to the spatial domain, the background interference can be effectively eliminated, the contrast between the sample light spot and the surrounding region in the preprocessed image can be greatly improved, and high-purity data basis is provided for the subsequent adaptive threshold processing to accurately identify the light spot region and accurately locate the light spot center through Gaussian fitting. At the same time, without introducing complex filtering algorithms, the processing flow is simplified and the loss of effective sample signal is reduced, thereby ensuring the accuracy and efficiency of the subsequent super-resolution image generation from the data source.

[0033] In step S203, adaptive threshold processing and region screening processing are performed on each preprocessed image to determine a plurality of light spot regions. The light spot region refers to a region where the sample Gaussian light spot pixels are located.

[0034] After obtaining each preprocessed image, the preprocessed image may have a brightness unevenness problem caused by the Gaussian distribution of the multi-focal point array illumination light field. In order to solve the problem of uneven illumination of the light field, it is necessary to filter out the noise and non-sample region in the image, and retain the connected region of the Gaussian light spot, so as to determine a plurality of light spot regions.

[0035] In one embodiment, a specific implementation manner of determining a plurality of light spot regions by performing adaptive threshold processing and region screening processing on each preprocessed image is also provided. Please refer to FIG. 8. Figure 3 As shown in FIG. 8, the specific implementation manner includes the following steps: In step S301, adaptive threshold processing is performed on each preprocessed image to obtain an intermediate image. The adaptive threshold processing is to divide the preprocessed image into a plurality of local regions, and dynamically adjust the corresponding threshold value of each local region according to the brightness value and the binary coefficient of each local region.

[0036] In step S302, the intermediate image is subjected to region screening processing by an open operation to remove noise regions and non-sample light spot regions, thereby obtaining a plurality of light spot regions.

[0037] Specifically, after obtaining the preprocessed image, adaptive threshold processing needs to be performed on the preprocessed image. Due to the characteristics of the microscopic system, the light spot and the background are in a Gaussian distribution as a whole due to the multi-focal point array illumination light field. In order to solve the problem of uneven light field, if global threshold processing is adopted, that is, a unified brightness threshold is set for the entire image to distinguish the foreground / background, the noise in the bright area may be misjudged as a light spot, and the light spot in the dark area may be misjudged as a background due to the uneven brightness of the light field (such as bright center and dark edge). The adaptive threshold processing in the present application divides the image into a plurality of local regions, and the threshold of each region is no longer fixed, but is dynamically adjusted according to the overall brightness of the region combined with the 0-1 range binary coefficient, for example, the threshold of the bright area is slightly higher, and the threshold of the dark area is slightly lower, so as to accurately distinguish the light spot and the background in each local region, and fundamentally solve the problem of uneven illumination of the light field. The suspected light spot region is preliminarily reserved, and an intermediate image is obtained.

[0038] After obtaining the intermediate image, an open operation is performed on the intermediate image to filter out noise and non-sample regions. After the adaptive threshold processing of the light spot connected region, small noise points (such as isolated bright pixels) and non-sample light spots (such as small light spots formed by system stray light) may still be left in the image, and part of the light spot profile may have a narrow neck or burr, which affects the subsequent pixel data extraction accuracy. By using the image processing operation of first erosion and then expansion in the open operation, the object profile can be smoothed, the narrow neck can be disconnected, and the small points can be eliminated. The above-mentioned redundant interference is removed, not only to remove noise points and non-sample small light spots, but also to optimize the integrity of the reserved light spot profile, so that only the continuous and clear Gaussian light spot connected region composed of sample signals is left, which prepares for the next step of extracting light spot pixel data. Each preprocessed image can include a plurality of light spot regions, and the size of each light spot region can be the same or different.

[0039] In this embodiment, the adaptive threshold processing and region screening processing are performed on each pre-processed image to determine a plurality of light spot regions, which can solve the problem of uneven brightness caused by the Gaussian distribution of the multi-focal point lattice illumination light field: the adaptive threshold processing divides the image into a plurality of local regions, dynamically adjusts the threshold according to the brightness of each region, avoids the situation that the light spots in the bright area are misjudged as noise and the light spots in the dark area are missed under the global threshold, and can accurately distinguish the light spots and the background in different brightness regions; the region screening processing is performed through the OR operation, which can further eliminate noise points, non-sample light spots and regions with incomplete contours, and only the continuous and clear sample Gaussian light spot connected regions are reserved. The combination of the two can improve the accuracy of light spot recognition, provide high-quality target regions for subsequent extraction of pixel data and Gaussian fitting positioning of the center, effectively reduce the interference of invalid data on subsequent processing, ensure the accuracy of light spot center positioning, and lay a key foundation for finally generating a high signal-to-noise ratio super-resolution image.

[0040] Step S204: All light spot regions of each pre-processed image are traversed, each pixel data of each light spot region is extracted, Gaussian fitting processing is performed on each pixel data, and the corresponding light spot center position is determined.

[0041] It should be noted that although the light spot regions have been obtained through threshold processing and screening in the pre-processed image, only the approximate range of the light spot can be determined, and the sub-pixel level accurate center position cannot be obtained. Super-resolution imaging has very high requirements for light spot positioning accuracy. In order to realize effective light spot positioning, the light spot center position needs to be determined in a more fine-grained manner.

[0042] Specifically, after obtaining each pre-processed image, for each screened light spot region, all pixel data contained therein are extracted, the pixel data can be the brightness value data of the pixel, that is, the gray value corresponding to each (x, y) coordinate, and the pixel data is substituted into the two-dimensional Gaussian formula for two-dimensional Gaussian fitting operation. The two-dimensional Gaussian formula can be expressed in the following form: ; Wherein, G(x, y) represents the function value of the pixel data at , that is, the intensity of the two-dimensional Gaussian distribution, A represents the maximum intensity of the Gaussian function at the light spot center position, that is, the peak value of the Gaussian function, represents the light spot center position of the Gaussian distribution, represents the standard deviation of the horizontal extension width of the Gaussian distribution in the x direction, represents the standard deviation of the vertical extension width of the Gaussian distribution in the y direction, and exp( ) represents the exponential function.

[0043] It can be understood that the above fitting process can adjust the parameters in the two-dimensional Gaussian formula by an optimization algorithm such as least squares method to minimize the error between the theoretical Gaussian curve and the actual pixel brightness distribution, and finally obtain the Gaussian model that best matches the brightness characteristics of the light spot area. Among them, the parameters include amplitude A, light spot center position , standard deviation and .

[0044] Since the sample light spot naturally conforms to the Gaussian distribution characteristics under the multi-focus structured light illumination, the fitted Gaussian model can accurately reflect the intensity decay law of the light spot, wherein the peak value of the Gaussian function (i.e. the position corresponding to A) is the point with the highest brightness of the light spot, and the pixel coordinates corresponding to the light spot center position are .

[0045] Further, all light spot center positions are obtained by sequentially traversing all light spot areas in each pre-processed image. By traversing all light spot areas in each image, it can be ensured that no effective sample signal is missed; extracting the pixel data of each area and performing Gaussian fitting is to utilize the characteristic that the sample light spot conforms to two-dimensional Gaussian distribution, i.e. the inherent characteristic of the multi-focus lattice illumination of the microscopic system, to accurately solve the light spot center coordinates , which improves the positioning accuracy from the pixel level to the sub-pixel level; at the same time, this process can further distinguish between real sample light spots (small fitting error) and residual noise (large fitting error), avoiding noise interference in subsequent processing.

[0046] The super-resolution imaging algorithm is used for a multi-focus structured light algorithm, which optimizes both the process efficiency and the imaging quality: on the one hand, it does not need to additionally shoot a calibration light field to assist in exciting lattice positioning as in the traditional algorithm, but directly uses the original sample image sequence as the basis for processing, saving the cumbersome steps of calibration light field acquisition, storage and preprocessing, greatly simplifying the experimental operation process and reducing data redundancy and experimental time consumption; on the other hand, in the noise suppression link, the algorithm first removes the direct current component in the original image through preprocessing to preliminarily filter out part of the noise, and then accurately judges and discards the residual noise signal through adaptive threshold, Gaussian fitting and other steps, and only applies a digital pinhole to the position of the effective sample signal that has been positioned, avoiding the loss of signal-to-noise ratio caused by the traditional algorithm which cannot distinguish between noise and sample light spot. Under the double optimization, the signal-to-noise ratio of the reconstructed image of the algorithm is doubled compared with the traditional algorithm, while the experimental convenience and imaging quality are taken into account.

[0047] Compared with the traditional algorithm, in this step, the Gaussian spot center can be located in the spatial domain using Gaussian fitting for reconstruction, improving the reconstruction efficiency. For example, using 80 frames for reconstruction can achieve the same effect as 224 frames, improving the time resolution by three times. Specifically, the mathematical model (i.e. two-dimensional Gaussian formula) fitting can eliminate the interference of single noise pixels on the center judgment. Even if there is brightness fluctuation at the edge of the spot, it can still output accurate coordinates at the sub-pixel level. And the solution of and The solution also reflects the ellipticity and other morphological characteristics of the spot, providing a basis for the dynamic adjustment of the size of the Gaussian pinhole. The final determined spot center position is not only a filter for distinguishing sample spots from residual noise, but also a coordinate reference for subsequent application of Gaussian pinholes and pixel relocation, achieving spot excitation positioning in the spatial domain for multi-focus original images. This provides a key coordinate reference for subsequent application of Gaussian pinholes and pixel-level relocation, in order to improve image resolution.

[0048] Step S205, generating a Gaussian pinhole corresponding to the spot center position on the blank image according to the spot center position, and performing point multiplication processing with the original image sequence to obtain a point multiplied image; the resolution of the point multiplied image is higher than that of the images in the original image sequence; the size of the Gaussian pinhole is half of the size of the spot region.

[0049] It should be noted that the Gaussian pinhole refers to a Gaussian distribution mode filter element generated on a blank image with the same size as the original image according to the spot center position and the two-dimensional Gaussian formula, and the size is set to half of the corresponding spot region. The point multiplied image is obtained by multiplying the Gaussian pinhole image with the original image sequence, and the resolution is higher than that of the intermediate processed image. The Gaussian pinhole image refers to a Gaussian distribution mode filter image generated on a blank image with the same size as the original image according to the determined spot center position of the sample, and the size of the Gaussian pinhole is set to half of the corresponding spot region.

[0050] In the process of generating the point multiplied image, a blank image with the same size as the images in the original image sequence is first constructed, and a Gaussian pinhole is generated at the position corresponding to the spot center position in the blank image to obtain a Gaussian pinhole image; the Gaussian pinhole image and the original image sequence are multiplied to obtain a point multiplied image.

[0051] Specifically, after the center of each light spot is located by the light spot area and the two-dimensional Gaussian formula, the accurate coordinates of the sample Gaussian light spot in each original image are determined. At this time, a blank image with the same size as the original image is first constructed to ensure subsequent matching with the original sequence, and then a Gaussian pinhole is generated at the position of the corresponding light spot center in the blank image according to the two-dimensional Gaussian formula, thereby generating a Gaussian pinhole image. The size of the Gaussian pinhole is set to be half of the size of the original image light spot, which takes into account both signal screening and resolution algorithm. This design not only avoids the mixing of noise signals caused by a too large pinhole, that is, effectively screens the sample signal, but also prevents the loss of effective sample signals caused by a too small pinhole, that is, avoids affecting the imaging integrity. It can accurately focus on the high signal-to-noise ratio region of the light spot center, and lay a foundation for subsequent resolution improvement.

[0052] After the Gaussian pinhole image is generated, the Gaussian pinhole image and the original image sequence can be point-multiplied, that is, the signal values of the corresponding pixel positions in the two images are multiplied. At this time, only the high signal-to-noise ratio sample signal in the original image that coincides with the pinhole position is retained, and the redundant background and noise signals are greatly suppressed, thereby obtaining a plurality of images after point multiplication. After this step, the clarity of the sample details in the image is significantly improved, and an image with a resolution about 1.4 times that of the original image is directly obtained, which provides high-quality data support for subsequent 2-fold resolution improvement through superposition fusion and RL deconvolution. Under certain conditions, a typical form of the light spot can be an Airy disk.

[0053] By setting the size of the Gaussian pinhole to be half of the size of the Airy disk in this step, the complex process of first extracting the light spot and then moving the position in the traditional algorithm can be avoided, greatly improving the speed of the reconstruction algorithm, for example, the algorithm speed is doubled.

[0054] In this embodiment, by constructing a blank image matching the size of the original image, accurate pinhole positioning and subsequent point multiplication without misalignment can be ensured. By focusing the Gaussian pinhole on the core region of the sample light spot with a size of half of the light spot, accurate screening of effective signals is facilitated. The point multiplication of the two can greatly suppress background noise and redundant information, not only directly improving the image resolution, but also avoiding the problem of inaccurate signal screening in traditional algorithms, while achieving efficient signal purification without complex preprocessing. This embodiment takes into account both processing accuracy and efficiency, and lays a high-quality data foundation for subsequent superposition fusion and deconvolution to generate higher resolution super-resolution images.

[0055] In step S206, all the point-multiplied images are superimposed and deconvoluted to generate a super-resolution image. The resolution of the super-resolution image is higher than that of the point-multiplied image.

[0056] After obtaining all the point-multiplied images, superimposed fusion processing is performed on all the point-multiplied images to obtain a fused image; and deconvolution processing is performed on the fused image to obtain a super-resolution image.

[0057] Specifically, since each frame of image after point multiplication is the product of the original image sequence combined with the corresponding Gaussian pinhole, although different frames of images are based on the same batch of samples, due to the angle and position difference of the multi-focus point array illumination, the sample details (such as edges and textures) captured by each image after point multiplication are slightly complementary, and a small amount of local noise may still remain in a single frame of image. By performing superimposed fusion processing on all the point-multiplied images, that is, adding and averaging all the point-multiplied images according to the pixel position, a fused image is obtained. On the one hand, the consistent sample effective signals (such as the signal in the center of the light spot) in multiple frames of images can be enhanced to make the sample microstructure clearer; on the other hand, the residual noise randomly distributed in each frame of image can be offset to further reduce noise interference. Optionally, the above superimposed fusion processing can obtain the fused image by using an "average superposition" or "weighted superposition" algorithm. The resolution of the fused image can be improved to 1.4 times compared with the resolution of the original image.

[0058] It can be understood that the essence of the above deconvolution processing is to eliminate the "point spread function (PSF)" influence in the imaging process of the microscopic system by mathematical algorithm to restore the real microstructure of the sample. In a multi-focus structured light illumination microscopic system, even after superimposed fusion, the fused image will still be blurred due to the diffraction effect of the optical system (such as objective aperture limitation and light wave nature), for example, a point light source on the sample will spread into a light spot with a certain intensity distribution after imaging, which is the manifestation of the point spread function, causing adjacent details to be confused and the resolution to be unable to break through the diffraction limit. The deconvolution processing needs to obtain a point spread function model of the microscopic system, which is, for example, a theoretical PSF calculated based on system parameters, or an actual PSF calibrated by experiment, and then uses the point spread function model to perform inverse operation on the fused image. The algorithm can be a Richardson-Lucy (RL) algorithm, which can strip the blurring effect of the point spread function on the image by algorithm, restore the diffused light spot signal to a sharp signal closer to the real morphology of the sample, and thus distinguish the fine structure in the fused image that cannot be distinguished. Finally, the image after deconvolution processing can break through the limitation of the original fused image and achieve a super-resolution level much higher than the diffraction limit, for example, a resolution of about 100 nm can be achieved, which is more than 2 times higher than the original image. At the same time, since the fused image already has a high signal-to-noise ratio, the deconvolution process will not significantly amplify the noise, and the finally generated super-resolution image.

[0059] In this step, the superposition fusion processing is performed on the multiplied images, so that the determined fusion image not only has a significantly higher signal-to-noise ratio than the single-frame point multiplication image, but also integrates the detailed information of multiple frames of data, thereby providing a high-quality data basis of "high signal-to-noise ratio + high detail density" for subsequent deconvolution processing, and avoiding the resolution limitation caused by single-frame data due to single information. And using deconvolution processing, a super-resolution image of "high resolution + high signal-to-noise ratio" can be generated, which meets the needs of fine observation of microstructure in the field of life science and other fields.

[0060] Please refer to Figure 4 As shown in the figure, after obtaining the original image sequence of the sample, the original image sequence is preprocessed, and the corresponding three-dimensional frequency domain sequence is obtained by three-dimensional Fourier transform processing. The direct current component in the frequency domain sequence is removed to obtain a plurality of preprocessed images, and then each preprocessed image is subjected to adaptive threshold processing and region screening processing to extract a plurality of light spot regions. Each pixel data of each light spot region is extracted by traversing all light spot regions of each preprocessed image, and the Gaussian fitting processing is performed on each pixel data to determine the corresponding light spot center position. A Gaussian pinhole is applied on the blank image according to the light spot center position, a 0.5 Airy spot Gaussian pinhole is applied at the light spot center position, and the point multiplication processing is performed with the image sequence to obtain a multiplied image. Further, the multiplied image is superimposed and fused to obtain a fusion image, and then the fusion image is subjected to deconvolution processing to obtain a super-resolution image of the multi-focus structured light.

[0061] Exemplarily, in the process of obtaining the original image sequence of the sample, the multi-focus structured light illumination microscope (MSIM) system can be used to realize the process, and the MSIM can be represented by the following imaging formula: ; Wherein, is the original image collected by the microscope, represents the physical position in the image space, is the excitation mode, which is , is the point array illumination mode, that is, the excitation point spread function of the system is convolved with the illumination point array, is the detection point spread function of the system.

[0062] The original image is the result of the real sample and the excitation mode first acting, and then convolving with the "probe point spread function (probe PSF)". Among them, the excitation mode is the convolution product of the system excitation point spread function (excitation PSF) and the point array illumination mode, representing the actual illumination distribution of the multi-focus point array illumination after the modulation of the optical characteristics of the system excitation end; the probe PSF is used to reflect the transmission and diffusion characteristics of the system detection end (such as the objective lens, the camera) to the light signal, and the two together constitute the optical transmission model of the MSIM system, which determines the presentation form of the sample information in the original image.

[0063] In order to accurately reproduce the optical effect of the MSIM system, the system PSF needs to be simulated and generated first. The simulation and generation of the PSF can be calculated by the following formula: Among them, is the PSF of the microscopic system, is the spatial coordinate, is the first-order Bessel function of the first kind, is the numerical aperture of the objective lens, is the wavelength of the emitted light.

[0064] According to the characteristics of the sample to be observed, the imaging parameters are set, which can include, for example: the numerical aperture (NA) of the objective lens is 1.45 (high NA ensures high resolution basis), the excitation wavelength is 480 nm, and the detection wavelength is 520 nm (matching the excitation-detection wavelength band of common fluorescence imaging); first, calculate the excitation PSF and the radiation PSF respectively, the excitation PSF is used to reflect the diffusion characteristics of the illumination light, and the radiation PSF is the detection PSF, which reflects the diffusion characteristics of the detection light, then the excitation PSF and the detection PSF are fused into the system PSF through point multiplication operation, the comprehensive optical effect of the excitation and detection links is simulated, and finally the simulation PSF as shown in Figure 5 (a) is obtained, which provides optical model support for subsequent illumination mode and original image generation.

[0065] And generate 224 inverted triangular point arrays in the computer device by running the computer device program, the specific process is to generate the point array with 1 pixel as the stepping unit, in which the adjacent point spacing in the x direction is 16 pixels and the adjacent spacing in the y direction is 14 pixels, and the overall distribution is inverted triangular, which can be seen from Figure 5 (b) shows; when generating the point array, first let all the points move 1 pixel in the x direction each time, move 1 pixel in the y direction after completing 16 x steps, and cycle to generate 16x14=224 point array images, and each point array point is 1 pixel in size, corresponding to a physical size of 41 nm, establishing the association between the pixel and the actual spatial scale.

[0066] After generating the point array, the simulation process of the multi-focus original sequence image is performed: first, each point array is convolved with the excitation PSF to obtain an illumination mode sequence image reflecting the illumination characteristics of the system; then each illumination mode image is sequentially multiplied with the real sample image to simulate the absorption and reflection of the sample to light, and finally convolved with the detection PSF to simulate the optical diffusion at the detection end, thereby generating 224 multi-focus original image sequences. One of the original image sequences can be seen from the simulated star-shaped image shown in Figure 5 (c); the real sample image can be seen from the simulated star-shaped image shown in Figure 5 (d). Superimposing all the original image sequences directly, a wide-field image can be generated, which can be seen from the simulated star-shaped image shown in Figure 5 (e), which is used for comparison with the subsequent super-resolution image to intuitively show the resolution improvement effect of the MSIM system.

[0067] In this application, the reconstruction algorithm can be represented by GSRMRL, and the wide-field image obtained by directly superimposing the original sequence image is represented by WF without any processing. The original sequence image obtained is processed by the reconstruction algorithm of the application to obtain a reconstructed image, i.e., a super-resolution image. The system PSF is used for RL deconvolution, and finally the Figure 5 (f) is obtained. Figure 5 (d) Figure 5 The white dashed line in (f) represents the resolvable limit area, and the resolution of the reconstructed image generated by the GSRMRL algorithm is about twice that of the wide-field reconstructed image. The reliability of the algorithm in recovering the resolution is verified by simulation.

[0068] Further, real open source data can also be used for testing. First, an open source data set is obtained, which can include a microtubule sample with a size of 128x128, and a 488nm wavelength excitation light is used, and the detection wavelength is 520nm, and the size of each pixel is 108nm. The wide-field image of the open source data can be seen from Figure 6 (a).

[0069] The algorithm of the application is represented by GSRMRL, and the traditional algorithm is represented by MSIM, and the wide-field image obtained by directly superimposing the original sequence image is represented by WF without any processing.

[0070] On this data, the algorithm proposed in the application is compared with the traditional multi-focus reconstruction algorithm, and the MSIM and GSRMRL reconstructed images (i.e., super-resolution images) are obtained by RL deconvolution. The reconstructed image obtained by the MSIM algorithm on the open source data can be seen from Figure 6 (b), and the super-resolution image obtained by the GSRMRL algorithm on the open source data can be seen from Figure 6 (c). Please refer toFigure 6 As shown in (d), from Figure 6 As can be seen from (d), Figure 6 (d) is Figure 6 (a) to (c) show the Gaussian function fitting curves of the underlined portions of the image. The GSRMRL algorithm can easily separate details 180nm apart, while MSIM struggles to do so. This demonstrates that GSRMRL exhibits higher reconstruction quality and stronger reconstruction capability in practical applications. This is due to the better noise resistance of the reconstruction algorithm in this application, as real data often contains a lot of noise signals, making the reconstruction algorithm in this application superior in real data. This further proves that when processing real data, the algorithm has better resolution recovery capability and more stable reconstruction results compared to traditional multifocal reconstruction methods.

[0071] This application provides a super-resolution image generation method. Compared with existing technologies, this method acquires the original image sequence of the sample without the need for additional high-quality calibration light field imaging required by traditional methods. It directly processes the sample data, significantly simplifying the operation process and reducing experimental time. The original image sequence is preprocessed to eliminate DC component interference, providing high-contrast data support for subsequent spot recognition and preventing background noise from obscuring spot details. Adaptive thresholding and region filtering are applied to the preprocessed image to accurately distinguish sample spot regions under uneven illumination in a multi-focal Gaussian light field, avoiding misjudgment problems caused by traditional global thresholding. Finally, the spot region is traversed, and pixel data is extracted and Gaussian fitting is performed to determine the spot center. It can filter out noise and spot interference, accurately locate effective sample spots, avoid positioning deviations caused by noise affecting vector estimation, and improve the accuracy of subsequent processing. Based on the center of the spot, it generates a Gaussian pinhole in the blank image with a size half that of the spot area and multiplies it with the original sequence. This can focus on the high signal-to-noise ratio core area of ​​the spot and suppress redundant noise, directly improving the initial image resolution. It also eliminates the need for the complex steps of traditional pixel relocation, simplifying the process and reducing the amount of computation. By superimposing, fusing, and deconvolving all multiplied images, it further integrates effective information from multiple frames, breaks through the diffraction limit, and finally generates a super-resolution image with a resolution much higher than that of the original image and the multiplied image. This significantly reduces the signal-to-noise ratio of the reconstructed image and improves the imaging quality.

[0072] Based on the same inventive concept, this application also provides an apparatus for implementing the super-resolution image generation described above. The solution provided by this apparatus is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the super-resolution image generation apparatus provided below can be found in the limitations of the super-resolution image generation method described above, and will not be repeated here.

[0073] In one exemplary embodiment, such as Figure 7As shown, an ultraprecision image generation device is provided, which comprises: The acquisition module 510 is configured to acquire a sequence of original images of a sample. The preprocessing module 520 is configured to preprocess the sequence of original images to obtain a plurality of preprocessed images. The light spot region determination module 530 is configured to perform adaptive threshold processing and region screening processing on each preprocessed image to determine a plurality of light spot regions; the light spot region refers to a region where a Gaussian light spot pixel of the sample is located. The center position determination module 540 is configured to traverse all light spot regions of each preprocessed image, extract respective pixel data of each light spot region, perform Gaussian fitting processing on the respective pixel data, and determine a corresponding light spot center position. The point multiplication image generation module 550 is configured to generate a Gaussian pinhole corresponding to the light spot center position on a blank image according to the light spot center position, and perform point multiplication processing on the blank image and the sequence of original images to obtain a point multiplied image; the resolution of the point multiplied image is higher than that of the images in the sequence of original images; the size of the Gaussian pinhole is half of the size of the light spot region. The ultraprecision image generation module 560 is configured to perform superposition fusion and deconvolution processing on all point multiplied images to generate an ultraprecision image; the resolution of the ultraprecision image is higher than that of the point multiplied image.

[0074] As an optional implementation, the light spot region determination module 530 is specifically configured to: perform adaptive threshold processing on the preprocessed image to obtain an intermediate image; the adaptive threshold processing is to divide the preprocessed image into a plurality of local regions, and dynamically adjust the corresponding threshold of each local region according to the brightness value and the binarization coefficient of each local region; perform region screening processing on the intermediate image by an OR operation to remove noise regions and non-sample light spot regions, and obtain a plurality of light spot regions.

[0075] As an optional implementation, the center position determination module 540 is specifically configured to: perform two-dimensional Gaussian fitting processing on the respective pixel data by substituting the respective pixel data into a two-dimensional Gaussian formula; take the pixel coordinates corresponding to the Gaussian peak value obtained after the Gaussian fitting as the light spot center position.

[0076] As an optional implementation, the two-dimensional Gaussian formula is: ; wherein, is a function value at the pixel data (x, y), A is a peak value of the Gaussian function, (x0, y0) is the light spot center position of the Gaussian distribution, and σ xis a standard deviation determining a horizontal extension width of the Gaussian distribution in the x direction y is a standard deviation determining a vertical extension width of the Gaussian distribution in the y direction, and exp() represents an exponential function.

[0077] As an optional implementation, the point multiplication image generation module 550 is specifically configured to: construct a blank image with the same size as the images in the original image sequence; generate a Gaussian pinhole at a position corresponding to a center position of the light spot in the blank image to obtain a Gaussian pinhole image; perform point multiplication processing on the Gaussian pinhole image and the original image sequence to obtain a point-multiplied image.

[0078] As an optional implementation, the super-resolution image generation module 560 is specifically configured to: perform superposition fusion processing on all the point-multiplied images to obtain a fused image; perform deconvolution processing on the fused image to obtain a super-resolution image.

[0079] As an optional implementation, the preprocessing module 520 is specifically configured to: perform three-dimensional Fourier transform processing on the original image sequence to obtain a corresponding three-dimensional frequency domain sequence; perform zero processing on a first frame of a direct current component common to all the images in the three-dimensional frequency domain sequence to obtain a processed three-dimensional frequency domain sequence; perform inverse Fourier transform processing on the processed three-dimensional frequency domain sequence to obtain a plurality of preprocessed images.

[0080] The super-resolution image generation device provided by the embodiment of the present application is characterized in that: the device directly processes the sample data on the basis of the obtained sample original image sequence, without the need for additional shooting of a high-quality calibration light field required by a traditional scheme, thereby greatly simplifying the operation process and reducing the experimental time consumption; the original image sequence is preprocessed to eliminate the direct current component interference, thereby providing high-contrast data support for subsequent light spot identification and avoiding the background noise from covering the light spot details; the preprocessed image is subjected to adaptive threshold processing and region screening, thereby accurately distinguishing the sample light spot region under uneven illumination in the multi-focus Gaussian light field and avoiding the misjudgment problem caused by the traditional global threshold; the light spot region is traversed, and the pixel data is extracted for Gaussian fitting to determine the light spot center, thereby filtering out the noise light spot interference, accurately positioning the effective sample light spot, avoiding the positioning deviation caused by the noise influence on the vector estimation, and improving the subsequent processing precision; the Gaussian pinhole with a size of half of the size of the light spot region is generated according to the light spot center in the blank image, and the point multiplication is performed on the original sequence, thereby focusing the core region of the light spot with a high signal-to-noise ratio, suppressing the redundant noise, directly preliminarily improving the image resolution, and simplifying the process and reducing the calculation amount without the complex steps of traditional pixel relocation; the super-resolution image with a resolution much higher than that of the original image and the image after the point multiplication is finally generated by superimposing and fusing all the point multiplication images and performing the deconvolution processing, thereby further integrating the effective information of multiple frames, breaking through the diffraction limit, significantly reducing the signal-to-noise ratio of the reconstructed image, and improving the imaging quality.

[0081] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram of the computer device can be as shown in Figure 8 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store video tag processing data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a super-resolution image generation method.

[0082] Those skilled in the art can understand that, Figure 8The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0083] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0084] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0085] In an exemplary embodiment, a computer program product is provided, including a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0087] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to a memory, a database or other medium used in the embodiments provided in the present application can include at least one of a non-volatile and a volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc.

[0088] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0089] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0090] The principles and implementation modes of the present application are described by using specific examples in the present application. The above embodiments are only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A super-resolution image generation method characterized by, The super-resolution image generation method comprises: obtaining an original image sequence of a sample; preprocessing the original image sequence to obtain a plurality of preprocessed images; performing adaptive threshold processing and region screening processing on each of the preprocessed images to determine a plurality of light spot regions; the light spot region refers to a region in which a Gaussian light spot pixel of the sample is located; traversing all light spot regions of each preprocessed image, extracting respective pixel data of each of the light spot regions, and performing Gaussian fitting processing on the respective pixel data to determine a corresponding light spot center position; generating a Gaussian pinhole corresponding to the light spot center position on a blank image according to the light spot center position, and performing point multiplication processing on the blank image and the original image sequence to obtain a point-multiplied image; the resolution of the point-multiplied image is higher than that of the images in the original image sequence; the size of the Gaussian pinhole is half of the size of the light spot region; superimposing and fusing all the point-multiplied images and performing deconvolution processing to generate a super-resolution image; the resolution of the super-resolution image is higher than that of the point-multiplied image.

2. The super-resolution image generation method according to claim 1, characterized in that, The adaptive threshold processing and region screening processing on each of the preprocessed images to determine a plurality of light spot regions comprises: performing adaptive threshold processing on each of the preprocessed images to obtain an intermediate image; the adaptive threshold processing comprises dividing the preprocessed image into a plurality of local regions, and dynamically adjusting the corresponding threshold of each local region according to the brightness value and the binarization coefficient of each local region; performing region screening processing on the intermediate image by an OR operation to remove noise regions and non-sample light spot regions, and obtaining the plurality of light spot regions.

3. The super-resolution image generation method of claim 1, wherein, The Gaussian fitting processing on the respective pixel data to determine a corresponding light spot center position comprises: substituting the respective pixel data into a two-dimensional Gaussian formula to perform two-dimensional Gaussian fitting processing; taking the pixel coordinates corresponding to the Gaussian peak value obtained after Gaussian fitting as the light spot center position.

4. The super-resolution image generation method according to claim 3, characterized in that, The two-dimensional Gaussian formula is: ； wherein, is a function value at pixel data (x, y), A is a peak value of the Gaussian function, (x0, y0) is a center position of a light spot of the Gaussian distribution, σ x is a standard deviation that determines a horizontal extension width of the Gaussian distribution in the x direction, y is a standard deviation that determines a vertical extension width of the Gaussian distribution in the y direction, and exp( ) indicates an exponential function.

5. The super-resolution image generation method of claim 1, wherein, The generation of the Gaussian pinhole corresponding to the light spot center position on the blank image according to the light spot center position, and the point multiplication processing of the blank image and the original image sequence to obtain a point-multiplied image comprises: constructing a blank image with the same size as the images in the original image sequence; generating a Gaussian pinhole at the position corresponding to the light spot center position in the blank image to obtain a Gaussian pinhole image; performing point multiplication processing on the Gaussian pinhole image and the original image sequence to obtain the point-multiplied image.

6. The super resolution image generation method of claim 1, wherein, The superimposition and fusion of all the point-multiplied images and the deconvolution processing to generate a super-resolution image comprises: superimposing and fusing all the point-multiplied images to obtain a fused image; performing deconvolution processing on the fused image to obtain the super-resolution image.

7. The super resolution image generation method of claim 1, wherein, The preprocessing of the original image sequence to obtain a plurality of preprocessed images comprises: performing three-dimensional Fourier transform processing on the original image sequence to obtain a corresponding three-dimensional frequency domain sequence; zero processing on a first frame of a direct current component common to all images in the three-dimensional frequency domain sequence to obtain a processed three-dimensional frequency domain sequence; Perform inverse Fourier transform processing on the processed three-dimensional frequency domain sequence to obtain the multiple frames of preprocessed images.

8. An apparatus for super-resolution image generation, characterized by comprising: The super-resolution image generation device comprises: An acquisition module configured to acquire an original image sequence of a sample; A preprocessing module configured to preprocess the original image sequence to obtain multiple frames of preprocessed images; A light spot region determination module configured to perform adaptive threshold processing and region screening processing on each of the preprocessed images to determine multiple light spot regions; the light spot region refers to a region in which a Gaussian light spot pixel of a sample is located; A center position determination module configured to traverse all light spot regions of each preprocessed image, extract pixel data of each of the light spot regions, perform Gaussian fitting processing on the pixel data, and determine a corresponding light spot center position; A point multiplication image generation module configured to generate a Gaussian pinhole corresponding to the light spot center position on a blank image according to the light spot center position, and perform point multiplication processing on the original image sequence to obtain a point-multiplied image; the resolution of the point-multiplied image is higher than that of an image in the original image sequence; the size of the Gaussian pinhole is half of the size of the light spot region; A super-resolution image generation module configured to perform superposition fusion and deconvolution processing on all point-multiplied images to generate a super-resolution image; the resolution of the super-resolution image is higher than that of the point-multiplied image.

9. A computer device comprising: A memory and a processor to store a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the super-resolution image generation method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the super-resolution image generation method of any one of claims 1-7.

Images