Image reconstruction method for fluorescence image, and device and computer-readable storage medium

Abstract

Provided in the present disclosure are an image reconstruction method for a fluorescence image, and a device and a computer-readable storage medium. The method comprises: performing image sharpening and noise reduction processing on a plurality of frames of original images, so as to obtain a plurality of frames of sharpened images, wherein the plurality of frames of original images are generated after basic groups to be tested, in a plurality of sequencing cycles, of at least one gene sequence to be tested are subjected to fluorescence imaging; performing image registration processing on the plurality of frames of sharpened images, so as to obtain a plurality of frames of registered images; and performing image reconstruction processing on the plurality of frames of registered images, so as to obtain a plurality of frames of reconstructed images. In this way, noise signals can be effectively removed from images by means of image sharpening and noise reduction processing, pixel offsets caused by the physical property of an image collection apparatus can be effectively reduced by means of image registration, and image reconstruction processing is then performed on the images which have been subjected to the image sharpening and noise reduction processing and the image registration processing, such that high-resolution reconstructed images can be obtained.

Description

Image reconstruction method, device and computer-readable storage medium for fluorescence images

[0001] This disclosure claims priority to a Chinese patent application filed with the Patent Office of China on January 31, 2024, with application number 202410140343.1 and application name “Image reconstruction method, device and storage medium for fluorescence images for nucleic acid sequencing”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of image processing technology, and in particular to an image reconstruction method, device, and computer-readable storage medium for a fluorescence image. Background Art

[0003] With the rapid development of biotechnology, the cost of gene sequencing technology has gradually decreased, making technologies such as disease gene screening gradually benefit ordinary people.

[0004] In related technologies, when performing gene sequencing, it is necessary to use industrial-grade cameras to capture the fluorescent signals generated by biochemical reagents equipped with high-fidelity reactions, and to perform base identification of the gene sequence by image analysis of the images captured during the capture process. Therefore, the quality of the collected fluorescent signal images will affect the accuracy of the base identification results. Due to the technical bottlenecks inherent in microscopic imaging technology, the collected images are prone to poor quality. Therefore, how to process the fluorescent images generated during the gene sequencing process to improve the accuracy of the gene sequencing results has become a technical problem that needs to be solved urgently in this field.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure at least provide a method, device, and computer-readable storage medium for reconstructing a fluorescence image.

[0007] In a first aspect, an embodiment of the present disclosure provides an image reconstruction method for a fluorescence image, comprising: performing image sharpening and noise reduction processing on multiple frames of original images to obtain multiple frames of sharpened images, wherein the multiple frames of original images are generated by fluorescence imaging of the test bases of at least one test gene sequence in multiple sequencing cycles; performing image registration processing on the multiple frames of sharpened images to obtain multiple frames of registered images; and performing image reconstruction processing on the multiple frames of registered images to obtain multiple frames of reconstructed images.

[0008] In an optional implementation, the performing image sharpening and noise reduction processing on the multiple frames of original images includes: performing image sharpening and noise reduction processing on the multiple frames of original images using a fractional-order differential image enhancement algorithm.

[0009] In an optional embodiment, the sharpening and noise reduction processing of multiple frames of original images using a fractional-order differential image enhancement algorithm includes: for each of the original images, determining a fractional-order differential filter kernel corresponding to the original image based on the fractional-order differential image enhancement algorithm; and performing convolution processing on the original image based on the fractional-order differential filter kernel corresponding to the original image to generate a sharpened image corresponding to the original image.

[0010] In an optional embodiment, the image registration processing of the multiple frames of sharpened images includes: for each of the sharpened images, transforming the sharpened image and the template image used for registration from time domain signals to frequency domain signals; calculating the cross-power spectrum between the sharpened image and the template image in the frequency domain, and transforming the calculated cross-power spectrum from the frequency domain signal to the time domain signal to obtain an offset function used to characterize the degree of offset between the sharpened image and the template image; sampling the offset function according to a preset sampling value, and performing weighted averaging on the sampling results to obtain the offset amount during the sharpened image registration.

[0011] In an optional embodiment, it also includes: before converting the sharpened image and the template image used for registration from time domain signals to frequency domain signals, performing image windowing processing on the sharpened image and the template image used for registration; the said converting the sharpened image and the template image used for registration from time domain signals to frequency domain signals includes: converting the sharpened image after image windowing processing and the template image used for registration from time domain signals to frequency domain signals.

[0012] In an optional embodiment, the image reconstruction processing of the multi-frame registered images includes: determining an energy function used to characterize the difference between before and after image reconstruction processing based on the offset corresponding to each frame of the sharpened image; determining different pixel attribute combinations, and based on the function values ​​corresponding to the energy function under different pixel attribute combinations, determining the target pixel attribute combination in the different pixel attribute combinations; and determining the reconstructed image based on the target pixel attribute combination.

[0013] In an optional embodiment, the offsets corresponding to the sharpened images are sub-pixel offsets; and the energy function used to characterize the difference between the image before and after reconstruction processing is determined based on the offsets corresponding to the sharpened images of each frame, including: constructing an image degradation model that includes each processing algorithm in the image reconstruction process based on the sub-pixel offsets corresponding to the sharpened images of each frame; and updating the graph cut energy function model pre-constructed in the Markov random field based on the image degradation model to obtain a target energy function constructed based on the image degradation model.

[0014] In an optional embodiment, determining different pixel attribute combinations includes: randomly assigning an image label to each pixel in the sharpened image; wherein the image label is used to characterize the pixel as an image foreground or image background; and for any two adjacent pixels in the sharpened image, generating different pixel attribute combinations by exchanging the image labels corresponding to the pixels.

[0015] In an optional embodiment, the function values ​​corresponding to the energy functions under different pixel attribute combinations are used to determine the target pixel attribute combinations in the different pixel attribute combinations, including: taking the pixel attribute combination with the smallest function value of the corresponding energy function as the target pixel attribute combination in the different pixel attribute combinations; and the reconstructed image is determined based on the target pixel attribute combination, including: taking the image generated based on the image labels corresponding to each pixel point in the target pixel attribute combination as the reconstructed image.

[0016] In a second aspect, an embodiment of the present disclosure further provides a computer device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are performed.

[0017] In a third aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0018] The fluorescent image reconstruction method, device, and computer-readable storage medium provided by the embodiments of the present disclosure perform sharpening and noise reduction processing on multiple frames of original images generated by fluorescence imaging of the test bases of at least one test gene sequence in multiple sequencing cycles, and sequentially perform image registration processing and image reconstruction processing on the multiple frames of sharpened images, thereby obtaining multiple frames of reconstructed images. In this way, image registration can effectively reduce pixel offsets caused by the physical properties of the image acquisition device, and image sharpening and noise reduction processing can effectively remove noise signals in the image. After image reconstruction processing is performed on the image after the image sharpening, noise reduction, and image registration processing, a high-resolution reconstructed image can be obtained.

[0019] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0021] FIG1 shows a flow chart of a method for reconstructing a fluorescence image provided by an embodiment of the present disclosure;

[0022] FIG2 shows a schematic diagram of a filter kernel in the image reconstruction method provided by an embodiment of the present disclosure;

[0023] FIG3 shows a flowchart of another image reconstruction method provided by an embodiment of the present disclosure;

[0024] FIG4-a shows a schematic diagram of an original low-resolution image in the image reconstruction method provided by an embodiment of the present disclosure;

[0025] FIG4-b is a schematic diagram showing a high-resolution image obtained after image reconstruction in the image reconstruction method provided by an embodiment of the present disclosure;

[0026] FIG5 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0028] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0029] Research has found that with the rapid development of biotechnology, the cost of gene sequencing technology has gradually decreased, making technologies such as disease gene screening increasingly accessible to the general public. In related technologies, gene sequencing requires industrial-grade cameras to capture the fluorescent signals generated by biochemical reagents with high-fidelity reactions. Base identification is performed by analyzing the images captured during the capture process. However, due to technical bottlenecks inherent in microscopic imaging technology, the quality of the captured images can be poor, which in turn affects the accuracy of nucleic acid base identification results.

[0030] Based on the above research, the present disclosure provides a method, device, and storage medium for reconstructing fluorescence images. The method performs sharpening and noise reduction on multiple frames of original images generated by fluorescence imaging of the test bases of at least one test gene sequence in multiple sequencing cycles, and sequentially performs image registration and image reconstruction on the multiple frames of sharpened images, thereby obtaining multiple frames of reconstructed images. Thus, image sharpening and noise reduction can effectively remove noise signals from the image, and image registration can effectively reduce pixel offsets caused by the physical properties of the image acquisition device. Subsequently, image reconstruction is performed on the image after image sharpening, noise reduction, and image registration, resulting in a high-resolution reconstructed image.

[0031] To facilitate understanding of this embodiment, we first provide a detailed introduction to an image reconstruction method disclosed in this embodiment. The image reconstruction method provided in this embodiment is generally executed by a computer device with sufficient computing power. This computer device may, for example, be a gene sequence detection device or a cloud-based server device that wirelessly communicates with the gene sequence detection device. In some possible implementations, this image reconstruction method can be implemented by a processor of the computer device invoking computer-readable instructions stored in a memory.

[0032] 1 , which is a flow chart of a method for reconstructing a fluorescence image according to an embodiment of the present disclosure, includes steps S101 to S103 , wherein:

[0033] S101: performing image sharpening and noise reduction processing on multiple frames of original images to obtain multiple frames of sharpened images, wherein the multiple frames of original images are generated by fluorescence imaging of bases to be tested in multiple sequencing cycles of at least one gene sequence to be tested.

[0034] S102: performing image registration processing on the multiple frames of sharpened images to obtain multiple frames of registered images.

[0035] S103: performing image reconstruction processing on the multiple frames of registered images to obtain multiple frames of reconstructed images.

[0036] The following is a detailed description of the above steps.

[0037] The process of image sharpening and noise reduction for multiple frames of original images in S101:

[0038] Here, in one sequencing cycle, one base to be tested in each of at least one gene sequence to be tested on the chip can be detected simultaneously. Through multiple sequencing cycles, the detection of multiple or all bases to be tested in each gene sequence to be tested on the chip can be completed, thereby realizing the sequencing of the gene sequence.

[0039] In one implementation, the chip may be a biochip, which is used to perform gene sequencing on a sample to be sequenced. During the sequencing process, the base recognition result of the gene sequence of the sample to be sequenced may be determined based on the fluorescence signal generated by the biochemical reagent during the sequencing process; the multiple frames of original images may be obtained after continuous acquisition by an image acquisition device within a preset time period, and the image acquisition device may be a camera deployed in a manner capable of capturing the biochip, and is used to acquire the fluorescence signal generated during the sequencing process in a dark light environment, thereby generating the original image through fluorescence imaging.

[0040] In a possible implementation, when performing image sharpening and noise reduction processing on multiple frames of original images, image sharpening and noise reduction processing may be performed on the multiple frames of original images using a fractional-order differential image enhancement algorithm.

[0041] Here, the fractional-order differential image enhancement algorithm can be one of a plurality of preset image enhancement algorithms. Different preset image enhancement algorithms use different fractional-order differential operators. The fractional-order differential operators can include a Grumwald-Letnikov operator, a Riesz operator, etc. An optional image enhancement algorithm will be specifically introduced below to perform image sharpening and noise reduction processing through the corresponding image enhancement algorithm.

[0042] In one possible implementation, when performing sharpening and noise reduction processing on multiple frames of the original image using a fractional-order differential image enhancement algorithm, for example, the following steps A1 to A2 may be performed:

[0043] A1: For each original image, determine a fractional-order differential filter kernel corresponding to the original image based on a fractional-order differential image enhancement algorithm.

[0044] A2: Perform convolution processing on the original image based on the fractional-order differential filter kernel corresponding to the original image to generate a sharpened image corresponding to the original image.

[0045] Here, the fractional-order differential image enhancement algorithm may be, for example, a Grunwald-Letnikov fractional-order differential algorithm.

[0046] Specifically, when determining the fractional-order differential filter kernel corresponding to the original image, it can be derived according to the following formula:

[0047] The above formula is used to express the definition of the fractional v-order derivative; where v represents the order of the fractional differential, a is the lower boundary of the signal duration, t is the upper boundary of the signal duration, h is the division unit of the signal duration, and r is the index variable; according to the above formula, if the duration of the unary signal s(t) is t∈[a, t], the signal duration [a, t] is divided equally by the unit h=1, and we can get:

[0048] Thus, the v-order fractional differential difference expression of the unary signal s(t) can be derived as:

[0049] According to the v-order fractional differential difference expression of the above-mentioned univariate signal s(t), the differential coefficient of the fractional differential can be obtained as follows:

[0050] Furthermore, the coefficients in the above differential coefficients are embedded into the fractional-order differential filter kernel template used in the image processing process.

[0051] Taking a 5×5 fractional-order differential filter kernel template as an example, the first three coefficients of the above differential coefficients are embedded into the filter kernel template, as shown below.

[0052] Among them, the filter kernel includes multiple weights to be determined, that is, Aa0, a1, and a2 are weights to be determined. Then according to formula (5), we can set a0=1, a1=-v, By embedding the three coefficients a0, a1, and a2 into the above-mentioned 5×5 fractional-order differential filter kernel template, the specific content of the 5×5 fractional-order differential filter kernel can be obtained, as shown in Figure 2, where v∈(0,1), for example, can be 0.1, 0.2, 0.3, 0.9, etc. In this way, through the above steps, a filter kernel for sharpening and denoising an image can be obtained, and by using the filter kernel to sharpen and denoise the original image, a sharpened image after the sharpening and denoising processing can be obtained.

[0053] Regarding the process of performing image registration processing on the multiple frames of sharpened images in S102:

[0054] Here, since the image acquisition device may have physical offsets during multiple rounds of gene sequencing, these offsets may cause certain differences in the spatial positions of the fluorescent images collected at the same position during different rounds of gene sequencing, and ultimately affect the accuracy of the base recognition results of the subsequent gene sequences. Therefore, it is necessary to spatially align the fluorescent images collected through multiple rounds of gene sequencing. This process of spatial alignment of image positions is also called image position correction or image registration. The offset of the image used for registration can be determined by image registration processing and other methods; in order to improve the accuracy of the final recognition result, the offset corresponding to the determined image can be a sub-pixel offset. Compared with the pixel-level offset, the registration accuracy of the sub-pixel offset is higher.

[0055] In a possible implementation, when performing image registration processing on the multiple frames of sharpened images, for example, the following steps B1 to B2 may be performed:

[0056] B1: For each of the sharpened images, transform the sharpened image and the template image used for registration from time domain signals to frequency domain signals.

[0057] Here, the template image may be an image selected in advance from a plurality of sharpened images, and the process of selecting the template image may be random selection or selection according to a received selection instruction.

[0058] In one possible implementation, before transforming the sharpened image and the template image used for registration from time domain signals to frequency domain signals, the sharpened image and the template image used for registration may be subjected to image windowing processing to reduce signal boundary effects.

[0059] Specifically, a preset window function (eg, a rectangular window, a triangular window, or a Hanning window) may be used to perform convolution processing on the sharpened image to obtain a windowed sharpened image.

[0060] Accordingly, when transforming the sharpened image and the template image for registration from time domain signals to frequency domain signals, the windowed sharpened image and the template image for registration can be transformed from time domain signals to frequency domain signals.

[0061] When transforming the time domain signal into the frequency domain signal, the time domain signal may be transformed into the frequency domain signal by processing in a manner such as Fourier transform.

[0062] Exemplarily, the sharpened image may be first subjected to a translation transformation, and then subjected to a Fourier transformation, so as to transform the sharpened image from a time domain signal to a frequency domain signal.

[0063] Specifically, the frequency domain signal corresponding to the sharpened image can be expressed as:

[0064] F sharpen1 (w x , w y )=F sharpen (w x , w y )exp{-j2π(w x e x +w y e y )} (6)

[0065] Among them, F sharpen1 (w x ,w y ) is the sharpened image F sharpen (w x ,w y ) is obtained by translation transformation, sharpen1 After Fourier transform, we get F sharpen1 (w x ,w y ) ; exp{-j2π(w x e x +w y e y )} is an exponential function, j represents an imaginary unit, and by adjusting e x and e y The value of can control the degree and direction of sharpening.

[0066] B2: Calculate the cross power spectrum between the sharpened image and the template image in the frequency domain, and transform the calculated cross power spectrum from the frequency domain signal to the time domain signal to obtain an offset function used to characterize the offset degree between the sharpened image and the template image.

[0067] Here, the template image can be selected from the sharpened image, and the first sharpened image is used as the template image. Then, the above formula (6) is processed according to the cross power spectrum calculation method to obtain:

[0068] in, It's F sharpen The conjugate of , A represents the calculated cross power spectrum.

[0069] Specifically, the cross power spectrum is a method for measuring the similarity between two signals and is often used to analyze the correlation between two signals. It can be obtained by calculating the product of the amplitude spectra of the Fourier transform (FFT) of the two signals. Therefore, the above formula (7) can be obtained by using the cross power spectrum calculation method and the above formula (6).

[0070] Furthermore, after calculating the cross power spectrum of the sharpened image and the template image according to the above formula (7), in order to further obtain the offset (i.e., the e in the formula x and e y ), the above formula (7) can also be subjected to inverse Fourier transform to obtain an offset function for characterizing the degree of offset between the sharpened image and the template image.

[0071] Specifically, since the above formula (7) represents the cross-power spectrum between the sharpened image and the template image in the frequency domain, in order to be able to continue the subsequent processing process in the time domain, the expression on the right side of the formula in the above formula (7) can be converted into the time domain (for example, an inverse Fourier transform) to obtain an expression containing the offset in the time domain, and the expression containing the offset constitutes the offset function.

[0072] B3: Sampling the offset function according to a preset sampling value, and performing weighted averaging on the sampling results to obtain the offset value for registration of the sharpened image.

[0073] Here, since the offset function is discrete, a preset sampling value may be used to sample the offset function. The sampling value may be, for example, 3, 5, 7, or the like.

[0074] Specifically, after sampling, the sampling results can be weighted averaged. Due to the sampling and weighted averaging processing, the final offset can be a non-integer. The non-integer offset can constitute a sub-pixel level offset in the image pixel level offset, that is, it is not the offset of the entire pixel, but the offset at the sub-pixel level accurate to the decimal point.

[0075] In this way, the above method can be used to obtain the sub-pixel offset of the sharpened image for use in registration. Compared with the pixel-level offset, the sub-pixel offset has higher registration accuracy.

[0076] Regarding the process of performing image reconstruction processing on the multi-frame registered images in S103:

[0077] In a possible implementation, when performing image reconstruction processing on the multi-frame registered images, for example, the following steps C1 to C3 may be performed:

[0078] C1: Based on the offsets corresponding to the sharpened images of each frame, an energy function is determined to characterize the difference between the image before and after the reconstruction process.

[0079] In one possible implementation, when determining the energy function for characterizing the difference between the image before and after the reconstruction process based on the offsets corresponding to the sharpened images of the frames, the following steps C11 to C12 may be performed:

[0080] C11: Based on the sub-pixel level offsets corresponding to the sharpened images of each frame, an image degradation model including each processing algorithm in the image reconstruction process is constructed.

[0081] Here, the constructed image degradation model can be:

[0082] f k =DT k H k f+η k ,k=1,2,…,n (8)

[0083] Among them, f k represents the kth degraded image, T k is the geometric transformation, H k is the point spread function, D is the downsampling operator, η k is the noise signal, f is the sharpened image, which can be expressed as f sharpen ; Among them, the value of D can be 4, 6, 8, etc.

[0084] Furthermore, according to the correspondence between high-resolution and low-resolution coordinate systems, the exact degradation process can be described as:

[0085] f k (x', y') = T k H k f(x+e x , y+e y )+η k (9)

[0086] And (x+e x , y+e y ) is the position relationship of low-resolution image pixels in the high-resolution coordinate system, (e x , e y ) is the error in the x and y directions. The pixel at (x', y') is given by (x+ex ， y+e y ) is determined by the pixel at the position and its surrounding pixels, and the determination form depends on the diffusion function H k , all original images are collected in the same environment, then the geometric transformation T k is an overall transformation. The point spread function has translation invariance, so the relationship between the high-resolution image and the low-resolution image can be:

[0087] f x (x', y') = H k f(x+e x , y+e y )+ηk (10)

[0088] Assuming that the point spread function remains unchanged during the image degradation process, the filter h can be used to approximately replace H k ,get:

[0089] f k (x', y') = h*f(x+e x , y+e y )+η k (11)

[0090] The point spread function can be constructed by using functions such as Bessel function and Gaussian function. Below, the point spread function is constructed using Gaussian function as an example:

[0091] Specifically, the two-dimensional filter kernel is constructed by the Kronecker product of the one-dimensional filter kernel, and the Gaussian function p(x) and the derivative function d(x) can be expressed as:

[0092] Then, the point spread function can be constructed as:

[0093] Among them, μ∈(0, 1) and σ∈(0, 1) in the above formula; the Gaussian point spread function adopts filter kernel templates such as 3×3, 5×5, and 7×7.

[0094] Furthermore, the degradation model (Formula 11 above) is subjected to first-order Taylor approximation to retain the sub-pixel information corresponding to the sub-pixel offset, and the following can be obtained:

[0095] According to the nature of convolution, let Then the degradation model can be:

[0096] f k (x', y') = (h + e x .h x +e y .h y )*f(x,y)+η k (15)

[0097] Where h'=h+e x .h x +e y .h y .

[0098] C12: Based on the image degradation model, the graph cut energy function model pre-constructed in the Markov random field is updated to obtain a target energy function constructed based on the image degradation model.

[0099] Here, the graph cut energy function model pre-constructed in the Markov random field can be:

[0100] Where p=(x+e x , y+e y ), p'=(x', y'), S represents the coordinate system of the sequencing image (i.e., the unified coordinate system corresponding to the image after registration), and N is the neighborhood system in the sequencing image coordinate system. The first term is the likelihood term, and the second term is the smoothing prior term. Assume that V p，q (f(p), f(q)) = min(T, |f(p) - f(q)|), where T is the threshold and λ∈(0, 1). The ratio of the likelihood term to the prior term is adjusted, and the new energy function is:

[0101] Specifically, when updating the graph cut energy function model pre-constructed in the Markov random field based on the image degradation model, h'=h+e in the above formula (15) can be used. x .h x +e y .h y By updating the above formula (17), the target energy function constructed based on the image degradation model can be obtained.

[0102] In this way, by constructing a graph cut energy function model under a Markov random field and using an image degradation model constructed based on sub-pixel offsets to update the graph cut energy function model, the sub-pixel offsets can be introduced into the final generated target energy function, thereby improving the accuracy of the target energy function and further improving the image quality of the final generated target image.

[0103] C2: Determine different pixel attribute combinations, and determine a target pixel attribute combination in the different pixel attribute combinations based on function values ​​corresponding to the energy function under the different pixel attribute combinations.

[0104] Here, after obtaining the target energy function, the target image corresponding to the minimum value of the target energy function can be obtained by adjusting pixel attributes, etc.; wherein, the image attributes may include the brightness of the pixel points, and the image quality can be improved by increasing the brightness of the foreground part of the image and reducing the brightness of the background part of the image.

[0105] In a possible implementation, when determining different pixel attribute combinations, the following steps C21 to C22 may be performed:

[0106] C21: Randomly assign an image label to each pixel in the sharpened image; wherein the image label is used to characterize whether the pixel is an image foreground or an image background.

[0107] C22: For any two adjacent pixels in the sharpened image, different pixel attribute combinations are generated by exchanging image labels corresponding to the pixels.

[0108] In this way, a variety of pixel attribute combinations can be generated through the above method for subsequent screening to find the minimum value of the target energy function.

[0109] C3: Determine a reconstructed image based on the target pixel attribute combination.

[0110] Furthermore, when determining the reconstructed image based on the target pixel attribute combination, the following steps C31 to C32 may be performed:

[0111] C31: taking the pixel attribute combination with the minimum function value of the corresponding energy function as the target pixel attribute combination among the different pixel attribute combinations.

[0112] C32: Using an image generated based on the image labels corresponding to each pixel point in the target pixel attribute combination as the reconstructed image.

[0113] In this way, by using the pixel attribute combination with the smallest function value of the corresponding energy function as the target pixel attribute combination in the different pixel attribute combinations, and using the image generated based on the image labels corresponding to each pixel point in the target pixel attribute combination as the reconstructed image, a reconstructed image with higher resolution composed of the pixel attribute combination corresponding to the calculated minimum value of the energy function can be obtained.

[0114] In addition, when determining the minimum value of the target energy function, different optimization algorithms can be used to determine the minimum value corresponding to the target energy function by adjusting the pixel attributes corresponding to the pixels in the image. The following uses the alpha-beta swap algorithm as an example to introduce the use of the optimization algorithm. For example, the following steps may be included:

[0115] Step 1: Randomly assign image labels to each pixel.

[0116] Here, the image label is used to characterize whether a pixel is a foreground part or a background part in an image, and different image labels may correspond to different initial pixel attributes.

[0117] Step 2: Set the flag bit flag=0.

[0118] Step 3: For each pair of labels Determine the energy minimum like but And set flag=1.

[0119] Among them, α and β represent any two adjacent pixel points in the image, and it is determined that the energy function value corresponding to the current two adjacent pixel points can be minimized. After the energy minimum value corresponding to the current two pixel points is found, the subsequent steps can be continued based on the current minimum value; the flag bit is used to indicate whether the current pixel point has completed the pixel attribute selection, that is, whether the function value of the corresponding energy function is a minimum value.

[0120] Specifically, when determining the minimum value corresponding to the energy function, different pixel attributes can be set for the above two adjacent pixel points, and by interacting pixel attributes, replacing pixel attributes, etc., a pixel attribute combination that can make the function value of the energy function corresponding to the current two adjacent pixel points the minimum value is determined, thereby completing the pixel attribute selection of the current two adjacent pixel points.

[0121] Step 4. When flag = 1, continue to search for any other two adjacent pixel points in the image, and process them according to the processing method of steps 2 to 3 above, determine the corresponding minimum value of the energy function, and repeat the above steps until the preset stopping condition is met. The preset stopping condition may include that the function value of the current corresponding energy function is a preset value (for example, 0), the number of repeated executions reaches a preset number, etc.

[0122] In this way, by decomposing the process of finding the global minimum of the energy function into the process of finding the local minimum (i.e., the minimum) of the energy function that corresponds to two pixel points of the vector, the task of finding the global optimum can be decomposed into sub-tasks of finding each local optimum, and the final local optimal minimum of the energy function is used as the final result. This can speed up the calculation speed of the entire calculation process while determining a certain image quality, thereby improving the image processing efficiency.

[0123] The following describes the image reconstruction method for a fluorescence image provided by an embodiment of the present disclosure in conjunction with specific embodiments. As shown in FIG3 , the processing of the improved image reconstruction method for a fluorescence image provided by an embodiment of the present disclosure may include the following steps:

[0124] Step 1: Acquire multiple frames of low-resolution images.

[0125] Step 2: sharpen the multiple low-resolution images based on a preprocessing algorithm to obtain multiple noise-reduced images.

[0126] Step 3: performing image alignment processing on the multiple frames of denoised images based on an image alignment algorithm.

[0127] Step 4: Determine the offset corresponding to each frame of the denoised image.

[0128] Step 5: Construct an energy function based on the offset.

[0129] Step 6: Adjust the function value of the energy function based on the preset optimization algorithm.

[0130] Step 7: Determine whether the function value of the energy function is minimum.

[0131] Here, if the judgment result is yes, the image corresponding to the function value of the energy function at this time can be used as a high-resolution image; if the judgment result is no, return to execute step 6, and continue to adjust the function value of the energy function based on the preset optimization algorithm until the preset stop condition is met. The preset stop condition may include that the function value of the current corresponding energy function is a preset value (for example, 0), the number of repeated executions reaches a preset number of times, etc.

[0132] Specifically, please refer to the above content for the detailed description of the above steps, which will not be repeated here.

[0133] For example, schematic diagrams of the original low-resolution image and the high-resolution image obtained after image reconstruction can be shown in Figures 4-a and 4-b. In Figure 4-a, in the original low-resolution image, the fluorescence signal is relatively weak; in Figure 4-b, in the high-resolution image obtained after image reconstruction, the fluorescence signal is relatively obvious, and subsequent base recognition and other processing can be performed.

[0134] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0135] Based on the same inventive concept, an image processing device corresponding to the image reconstruction method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image reconstruction method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0136] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Referring to Figure 5, there is a structural diagram of the computer device 500 provided in the embodiment of the present disclosure, which includes a processor 501, a memory 502, and a bus 503. Among them, the memory 502 is used to store execution instructions, including a memory 5021 and an external memory 5022; the memory 5021 here is also called an internal memory, which is used to temporarily store the calculation data in the processor 501, as well as the data exchanged with the external memory 5022 such as a hard disk. The processor 501 exchanges data with the external memory 5022 through the memory 5021. When the computer device 500 is running, the processor 501 communicates with the memory 502 through the bus 503, so that the processor 501 executes the following instructions:

[0137] Performing image sharpening and noise reduction processing on multiple frames of original images to obtain multiple frames of sharpened images, wherein the multiple frames of original images are generated by fluorescence imaging of bases to be tested in multiple sequencing cycles of at least one gene sequence to be tested;

[0138] Performing image registration processing on the multiple frames of sharpened images to obtain multiple frames of registered images;

[0139] Perform image reconstruction processing on the multiple frames of registered images to obtain multiple frames of reconstructed images.

[0140] It should be noted that the computer device 500 can be located on the gene sequencer itself, or on a back-end server that communicates wirelessly with the gene sequencer. If it is located on the server, the gene sequencer can collect fluorescent images and transmit the collected fluorescent images to the back-end server. The back-end server then reconstructs the fluorescent images and performs other necessary image processing on the fluorescent images through the image reconstruction method of the present invention, and performs base identification based on the final processed image. Finally, the identified gene sequencing results are returned to the gene sequencer. The user of the gene sequencer can view the gene sequencing results through the display device on the gene sequencer, and can also copy the gene sequencing results from the gene sequencer.

[0141] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the image reconstruction method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0142] The embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the image reconstruction method described in the above method embodiment. For details, please refer to the above method embodiment and will not be repeated here.

[0143] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0145] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0147] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0148] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A computer device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the following image reconstruction method is performed: Performing image sharpening and noise reduction processing on multiple frames of original images to obtain multiple frames of sharpened images, wherein the multiple frames of original images are generated by fluorescence imaging of bases to be tested in multiple sequencing cycles of at least one gene sequence to be tested; Performing image registration processing on the multiple frames of sharpened images to obtain multiple frames of registered images; Perform image reconstruction processing on the multiple frames of registered images to obtain multiple frames of reconstructed images.

2. The computer device according to claim 1, wherein: When the machine-readable instructions are executed by the processor, performing image sharpening and noise reduction processing on multiple frames of original images includes: The image sharpening and noise reduction are performed on multiple frames of original images using the fractional-order differential image enhancement algorithm.

3. The computer device according to claim 2, wherein: When the machine-readable instructions are executed by the processor, the image enhancement algorithm using fractional differentials is used to perform sharpening and noise reduction on multiple frames of original images, including: For the original image, determining a fractional-order differential filter kernel corresponding to the original image based on a fractional-order differential image enhancement algorithm; The original image is convolved based on a fractional-order differential filter kernel corresponding to the original image to generate a sharpened image corresponding to the original image.

4. The computer device according to claim 1, wherein: When the machine-readable instructions are executed by the processor, performing image registration processing on the multiple frames of sharpened images includes: For the sharpened image, transforming the sharpened image and the template image used for registration from time domain signals to frequency domain signals; Calculating a cross power spectrum in the frequency domain between the sharpened image and the template image, and transforming the calculated cross power spectrum from a frequency domain signal to a time domain signal to obtain an offset function for characterizing the degree of offset between the sharpened image and the template image; The offset function is sampled according to the preset sampling value, and the sampling results are weighted and averaged to obtain The offset for registering the sharpened image.

5. The computer device according to claim 4, wherein: When the machine-readable instructions are executed by the processor, before converting the sharpened image and the template image for registration from time domain signals to frequency domain signals, the method further includes: performing image windowing processing on the sharpened image and the template image for registration; The step of transforming the sharpened image and the template image for registration from time domain signals to frequency domain signals comprises: The sharpened image after image windowing processing and the template image used for registration are transformed from time domain signals to frequency domain signals.

6. The computer device according to claim 1, wherein: When the machine-readable instructions are executed by the processor, the image reconstruction process of the multiple registered images includes: Determining an energy function for characterizing the difference between the image before and after the image reconstruction process based on the offsets corresponding to the sharpened images of the frames; Determining different pixel attribute combinations, and determining a target pixel attribute combination in the different pixel attribute combinations based on function values corresponding to the energy function under the different pixel attribute combinations; A reconstructed image is determined based on the target pixel attribute combination.

7. The computer device according to claim 6, wherein: The offsets corresponding to the sharpened images are sub-pixel level offsets; When the machine-readable instructions are executed by the processor, determining an energy function for characterizing the difference between the image before and after the image reconstruction process based on the offsets corresponding to the sharpened images of each frame includes: Based on the sub-pixel level offsets corresponding to the sharpened images of each frame, an image degradation model including each processing algorithm in the image reconstruction process is constructed; Based on the image degradation model, a graph cut energy function model pre-constructed in a Markov random field is updated to obtain a target energy function constructed based on the image degradation model.

8. The computer device according to claim 6, wherein: When the machine-readable instructions are executed by the processor, determining different pixel attribute combinations includes: Randomly assigning an image label to each pixel in the sharpened image; wherein the image label is used to characterize whether the pixel is an image foreground or an image background; For any two adjacent pixels in the sharpened image, different pixel attribute combinations are generated by exchanging image labels corresponding to the pixels.

9. The computer device according to claim 6, wherein: When the machine-readable instructions are executed by the processor, determining a target pixel attribute combination among the different pixel attribute combinations based on function values corresponding to the energy functions under the different pixel attribute combinations includes: The pixel attribute combination with the minimum function value of the corresponding energy function is used as the target pixel attribute combination among the different pixel attribute combinations; Determining a reconstructed image based on the target pixel attribute combination includes: An image generated based on the image labels corresponding to each pixel point in the target pixel attribute combination is used as the reconstructed image.

10. A method for reconstructing a fluorescence image, characterized in that: include: Performing image sharpening and noise reduction processing on multiple frames of original images to obtain multiple frames of sharpened images, wherein the multiple frames of original images are generated by fluorescence imaging of bases to be tested in multiple sequencing cycles of at least one gene sequence to be tested; Performing image registration processing on the multiple frames of sharpened images to obtain multiple frames of registered images; Perform image reconstruction processing on the multiple frames of registered images to obtain multiple frames of reconstructed images.

11. The method according to claim 10, characterized in that The image sharpening and noise reduction processing on the multiple frames of original images includes: The image sharpening and noise reduction are performed on multiple frames of original images using the fractional-order differential image enhancement algorithm.

12. The method according to claim 11, characterized in that The image enhancement algorithm using fractional differentials is used to sharpen and reduce noise on multiple frames of original images, including: For the original image, determining a fractional-order differential filter kernel corresponding to the original image based on a fractional-order differential image enhancement algorithm; The original image is convolved based on a fractional-order differential filter kernel corresponding to the original image to generate a sharpened image corresponding to the original image.

13. The method according to claim 1, wherein The performing image registration processing on the multiple frames of sharpened images includes: For the sharpened image, transforming the sharpened image and the template image used for registration from time domain signals to frequency domain signals; Calculate the cross power spectrum between the sharpened image and the template image in the frequency domain, and convert the calculated cross power spectrum from The frequency domain signal is transformed into a time domain signal to obtain an offset function for characterizing the offset degree between the sharpened image and the template image; The offset function is sampled according to a preset sampling value, and the sampling results are weighted and averaged to obtain the offset value of the sharpened image during registration.

14. The method according to claim 13, characterized in that Also includes: Before converting the sharpened image and the template image for registration from time domain signals to frequency domain signals, performing image windowing processing on the sharpened image and the template image for registration; The step of transforming the sharpened image and the template image for registration from time domain signals to frequency domain signals comprises: The sharpened image after image windowing processing and the template image used for registration are transformed from time domain signals to frequency domain signals.

15. The method according to claim 1, wherein The image reconstruction processing of the multi-frame registered images includes: Determining an energy function for characterizing the difference between the image before and after the image reconstruction process based on the offsets corresponding to the sharpened images of the frames; Determining different pixel attribute combinations, and determining a target pixel attribute combination in the different pixel attribute combinations based on function values corresponding to the energy function under the different pixel attribute combinations; A reconstructed image is determined based on the target pixel attribute combination.

16. The method according to claim 15, characterized in that The offsets corresponding to the sharpened images are sub-pixel level offsets; The step of determining an energy function for characterizing the difference between the image before and after the image reconstruction process based on the offsets corresponding to the sharpened images of the frames includes: Based on the sub-pixel level offsets corresponding to the sharpened images of each frame, an image degradation model including each processing algorithm in the image reconstruction process is constructed; Based on the image degradation model, a graph cut energy function model pre-constructed in a Markov random field is updated to obtain a target energy function constructed based on the image degradation model.

17. The method according to claim 15, characterized in that Determining different pixel attribute combinations includes: Randomly assign an image label to each pixel in the sharpened image; wherein the image label is used to characterize the image The pixel is the image foreground or image background; For any two adjacent pixels in the sharpened image, different pixel attribute combinations are generated by exchanging image labels corresponding to the pixels.

18. The method according to claim 15, characterized in that The determining, based on the function values corresponding to the energy functions under the different pixel attribute combinations, a target pixel attribute combination in the different pixel attribute combinations includes: The pixel attribute combination with the minimum function value of the corresponding energy function is used as the target pixel attribute combination among the different pixel attribute combinations; Determining a reconstructed image based on the target pixel attribute combination includes: An image generated based on the image labels corresponding to each pixel point in the target pixel attribute combination is used as the reconstructed image.

19. A gene sequencing device, characterized in that: The gene sequencing device is used to reconstruct the fluorescence image using the image reconstruction method described in any one of claims 10 to 18, and perform base recognition based on the reconstructed image to obtain the gene sequencing result.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the image reconstruction method according to any one of claims 10 to 18.

Images