Image reconstruction and alignment method, computer equipment and computer readable storage medium

Abstract

The invention provides an image reconstruction and alignment method, computer equipment and a storage medium, and the method comprises the steps: obtaining a plurality of frames of to-be-reconstructed fluorescence images from a plurality of sequencing cycles; the multiple frames of fluorescence images to be reconstructed are input into a trained image reconstruction model, multiple frames of reconstructed fluorescence images output by the image reconstruction model are obtained, and the reconstructed fluorescence images comprise light spot centroid information in the fluorescence images to be reconstructed; and determining an alignment template according to the multiple frames of reconstructed fluorescence images, and realizing image alignment based on the alignment template.

Description

Technical Field

[0001] The present disclosure relates to the technical field of gene sequencing, and in particular to an image reconstruction and alignment method, a computer device, and a computer-readable storage medium. Background Art

[0002] During high-throughput sequencing, after each nucleic acid fragment (such as a DNA fragment) is clustered through an amplification reaction, any cluster will be presented in the sequencing image in the form of a fluorescent spot in each sequencing cycle during the sequencing reaction. Therefore, a series of sequencing images will be generated during the sequencing process. Since the sequencing image includes the fluorescent spot, the sequencing image is also called a fluorescent image. Due to the mechanical errors of the sequencing equipment itself, the same photographed area has positional deviations in the sequencing images obtained in different sequencing cycles, which makes the fluorescent spots of the same cluster have positional deviations in the fluorescent images taken in different sequencing cycles, resulting in a decrease in the accuracy of the positions of each cluster located during the sequencing process, which in turn causes a decrease in the sequencing quality of the corresponding nucleic acid fragments.

[0003] In order to reduce the problems caused by mechanical errors, during the sequencing process, it is necessary to use a template image (also called an alignment template) to correct and align the position of the fluorescent spots in the sequencing images corresponding to different sequencing cycles of the same cluster. However, the current method of constructing a template image for image alignment is usually to sharpen and segment the fluorescent image used to construct the template image, which has a relatively low accuracy and results in low sequencing quality. Summary of the invention

[0004] The embodiments of the present disclosure at least provide an image reconstruction and alignment method, a computer device, and a computer-readable storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides an image reconstruction and alignment method, the method comprising:

[0006] Acquire multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles;

[0007] Inputting the multiple frames of fluorescence images to be reconstructed into the trained image reconstruction model to obtain multiple frames of reconstructed fluorescence images output by the image reconstruction model, wherein the reconstructed fluorescence images contain the centroid information of the light spots in the fluorescence images to be reconstructed, and the centroid information of the light spots is used as a reference feature for image alignment;

[0008] An alignment template is determined according to the reconstructed fluorescence images of multiple frames, and image alignment is achieved based on the alignment template.

[0009] Optionally, the image reconstruction model is trained in the following manner;

[0010] According to the intensity values ​​corresponding to the respective pixel positions in the multiple frames of sample fluorescence images, the multiple frames of sample fluorescence images are respectively subjected to image segmentation processing, and the light spot labels of the multiple frames of sample fluorescence images are generated according to the results of the segmentation processing of the multiple frames of sample fluorescence images; the multiple frames of sample fluorescence images are used for model training and are derived from multiple sequencing cycles;

[0011] The image reconstruction model is trained based on multiple frames of sample fluorescence images and their corresponding spot labels.

[0012] Optionally, the image segmentation processing is performed on the multiple frames of sample fluorescence images according to the intensity values ​​corresponding to each pixel position in the multiple frames of sample fluorescence images, and the spot labels of the multiple frames of sample fluorescence images are generated according to the results of the segmentation processing of the multiple frames of sample fluorescence images, including:

[0013] Performing image alignment on multiple frames of sample fluorescence images to obtain multiple frames of aligned sample fluorescence images;

[0014] Determine, according to the grayscale value of each pixel position in the sample fluorescence image after the multi-frame alignment, the base sequence corresponding to the candidate pixel position of the light spot in the sample fluorescence image after the multi-frame alignment; the candidate pixel position of the light spot includes at least part of the pixel position of the light spot in the sample fluorescence image after the alignment;

[0015] Based on the consistency information between the base sequence of the candidate light spot pixel position and the base sequence of the candidate light spot pixel position in the corresponding target neighborhood, and / or based on the fault tolerance information between the base sequence of the candidate light spot pixel position and the base sequence in the reference genome, determine the pixel position of the centroid of the fluorescent light spot from the candidate light spot pixel positions;

[0016] The spot label is generated according to the pixel position of the centroid of the fluorescent spot.

[0017] Optionally, determining the base sequence corresponding to the candidate pixel position of the light spot according to the gray value of each pixel position in the sample fluorescent image after the multi-frame alignment includes:

[0018] According to the gray value of each pixel point in the sample fluorescence image after multi-frame alignment, the fluorescence spot is taken as the foreground, and the image segmentation of the foreground and the background is performed on the sample fluorescence image after multi-frame alignment to obtain the candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment; and the candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment is fused to obtain the fused candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment;

[0019] and performing image sharpening on the multiple aligned frames of sample fluorescence images to obtain multiple sharpened frames of sample fluorescence images;

[0020] Each pixel position in the fused candidate fluorescent spot area is used as a spot candidate pixel position, and a base sequence corresponding to the spot candidate pixel position is determined according to the gray value of the spot candidate pixel position in the multi-frame sharpened sample fluorescent image.

[0021] Optionally, before the step of taking the fluorescent spot as the foreground and performing image segmentation of the foreground and background on the sample fluorescent images after the multi-frame alignment according to the grayscale value of each pixel in the sample fluorescent images after the multi-frame alignment, the step further includes:

[0022] Performing image noise reduction processing on the multiple aligned frames of sample fluorescence images respectively to obtain multiple frames of sample fluorescence images after noise reduction processing;

[0023] The method of taking the fluorescent spot as the foreground and performing image segmentation of the foreground and background on the sample fluorescent images after the multi-frame alignment according to the gray value of each pixel in the sample fluorescent images after the multi-frame alignment comprises:

[0024] According to the grayscale value of each pixel in the sample fluorescence image after the multi-frame noise reduction processing, the fluorescence spot is used as the foreground, and the image segmentation of the foreground and background is performed on the sample fluorescence image after the multi-frame alignment.

[0025] Optionally, image segmentation processing is performed on the multiple frames of sample fluorescence images according to the intensity values ​​corresponding to the respective pixel positions in the multiple frames of sample fluorescence images, and spot labels of the multiple frames of sample fluorescence images are generated according to the results of the segmentation processing on the multiple frames of sample fluorescence images:

[0026] According to the gray value of each pixel position in the multi-frame sample fluorescence image, the fluorescence spot is taken as the foreground, and the multi-frame sample fluorescence image is segmented into the foreground and the background respectively, so as to obtain the candidate fluorescence spot area in the multi-frame sample fluorescence image;

[0027] Parabolic interpolation processing is performed on the candidate fluorescent spot areas in the multiple frames of sample fluorescent images to obtain the fluorescent spot centroid of the candidate fluorescent spot area and generate a spot label.

[0028] Optionally, the step of taking the fluorescent spot as the foreground and performing image segmentation of the foreground and the background on the multiple frames of sample fluorescent images respectively according to the grayscale value of each pixel position in the multiple frames of sample fluorescent images includes:

[0029] The multiple frames of sample fluorescence images are subjected to image denoising respectively, and according to the grayscale value of each pixel position in the multiple frames of sample fluorescence images after denoising, the fluorescent light spot is used as the foreground, and the multiple frames of sample fluorescence images are subjected to image segmentation of foreground and background respectively.

[0030] Optionally, the multiple frames of sample fluorescence images are acquired in the following manner:

[0031] Acquire multiple frames of original fluorescence images from multiple sequencing cycles for model training, and select the multiple frames of sample fluorescence images from the multiple frames of original fluorescence images; or,

[0032] The multiple frames of original fluorescence images derived from multiple sequencing cycles are obtained, and the multiple frames of original fluorescence images are divided into multiple sub-images respectively; and the multiple frames of sample fluorescence images are determined from the multiple sub-images in the multiple frames of original fluorescence images according to the signal-to-noise ratios corresponding to the multiple sub-images respectively.

[0033] Optionally, the training of the image reconstruction model based on multiple frames of sample fluorescence images and their corresponding spot labels includes:

[0034] Performing fusion processing on multiple frames of sample fluorescence images to obtain a fused fluorescence image, inputting the fused fluorescence image into the image reconstruction model to be trained to obtain a first reconstructed image; wherein the pixel value of each pixel position in the first reconstructed image represents the probability that the pixel position belongs to the centroid of the light spot; determining the first model loss according to the first reconstructed image and the light spot label;

[0035] and / or,

[0036] Input multiple frames of sample fluorescence images into the image reconstruction model to be trained to obtain a second reconstructed image; wherein the pixel value of each pixel position in the second reconstructed image represents the probability that the pixel position belongs to the centroid of the light spot; determine the second model loss based on the second reconstructed image and the light spot label; and use the first model loss and / or the second model loss to adjust the parameters of the image reconstruction model to be trained to obtain the target image reconstruction model.

[0037] Optionally, the step of obtaining multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles includes:

[0038] determining a reference fluorescence image from a plurality of frames of raw fluorescence images derived from a plurality of sequencing cycles for image alignment;

[0039] Dividing the reference fluorescent image into a plurality of sub-images, and determining a target sub-image from the plurality of sub-images according to signal-to-noise ratios respectively corresponding to the plurality of sub-images;

[0040] According to the position of the target sub-image in the reference fluorescent image, sub-images having the same position are respectively intercepted from the multiple frames of original fluorescent images as multiple frames of fluorescent images to be reconstructed.

[0041] Optionally, determining a reference fluorescence image from a plurality of frames of original fluorescence images from a plurality of sequencing cycles for image alignment comprises:

[0042] Determine the signal-to-noise ratios corresponding to at least a portion of the original fluorescence images in a plurality of frames of original fluorescence images derived from a plurality of sequencing cycles for image alignment;

[0043] The reference fluorescence image is determined from the at least part of the original fluorescence image according to the signal-to-noise ratios respectively corresponding to the at least part of the original fluorescence image.

[0044] In a second aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions perform the steps in the above-mentioned first aspect, or any possible implementation of the first aspect.

[0045] In a third aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are executed.

[0046] In the image reconstruction and alignment method provided by the embodiment of the present disclosure, after obtaining multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles, the multiple frames of fluorescence images to be reconstructed are input into a trained image reconstruction model, and the image reconstruction model performs image reconstruction processing on the multiple frames of fluorescence images to be reconstructed to obtain multiple frames of reconstructed fluorescence images, and determines an alignment template based on the multiple frames of reconstructed fluorescence images, and performs image alignment based on the alignment template, so that the image reconstruction model is used to reconstruct the fluorescence images to be reconstructed with higher accuracy. When performing image alignment based on the alignment template determined based on the reconstructed fluorescence images, the influence of mechanical errors can be eliminated to a greater extent, thereby improving sequencing quality.

[0047] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0049] Figure 1 A schematic diagram of a sequencing system provided by some embodiments of the present disclosure is shown;

[0050] Figure 2 A schematic diagram of a sequencing chip provided in some embodiments of the present disclosure is shown;

[0051] Figure 3 A flowchart of an image reconstruction and alignment method provided by some embodiments of the present disclosure is shown;

[0052] Figure 4 A flowchart of a method for training an image reconstruction model provided by some embodiments of the present disclosure is shown;

[0053] Figure 5 A flowchart showing a specific method of generating a light spot label provided by some embodiments of the present disclosure is shown;

[0054] Figure 6 One example of base sequences corresponding to each pixel position provided by some embodiments of the present disclosure is shown;

[0055] Figure 7 The second example of the relative position relationship between the pixel position N and its corresponding target neighborhood pixel position N' provided in some embodiments of the present disclosure is shown;

[0056] Figure 8 The third example of the original fluorescent image, the reconstructed fluorescent image, and the alignment template provided in some embodiments of the present disclosure is shown;

[0057] Fig. 9 A schematic diagram of a computer device provided by some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0059] To facilitate the understanding of the technical solution of the present disclosure, the technical terms in the embodiments of the present disclosure are first explained:

[0060] Library construction

[0061] The genomic DNA or RNA molecules to be sequenced are broken by physical or chemical means, for example, by ultrasound, to form DNA or RNA fragments. The two ends of the DNA or RNA fragments are first filled with enzymes, and then the two ends of the fragments are connected to a specific DNA or RNA sequence (usually, this specific DNA sequence or RNA is also called a linker) with a specific enzyme to form a mixture of DNA or RNA. This mixture of DNA or RNA is also called a library in the industry.

[0062] In order to save sequencing costs, generally, multiple samples will be sequenced in the sequencer at the same time. In order to distinguish the sequencing results of different samples, when preparing libraries of different samples, the connectors will contain a DNA or RNA sequence (usually containing 6-8 bases) that can identify the source of the sample. This DNA or RNA sequence that can identify the source of the sample can also be called a sample tag (or Index, Barcode). It can be understood that each sample library connector contains its own sample tag.

[0063] Typically, library construction is done outside of the sequencer, for example, by experimental manipulation in the laboratory to obtain the library.

[0064] Amplification reaction

[0065] Taking the amplification of a DNA library by bridge PCR (polymerase chain reaction) as an example, after the library is constructed, the library can be inoculated onto a sequencing chip and amplified on the sequencing chip. The adapters at both ends of the library are complementary to the first amplification primer on the sequencing chip, so the library can be inoculated onto the sequencing chip through complementary hybridization.

[0066] After the library is inoculated onto the sequencing chip, an amplification reaction can be performed using the library as a template chain. For example, the amplification reaction process can be to first add dNP and polymerase to the sequencing chip. The polymerase will start from the first amplification primer and synthesize a new DNA chain along the template chain. The new DNA chain is completely complementary to the template chain, so it is also called the complementary chain of the template chain. The complementary chain is covalently linked to the sequencing chip. Next, a NaOH alkaline solution is added to the sequencing chip for washing. The template chain and the complementary chain are mutually untied in the presence of the NaOH alkaline solution, and the template chain is washed away with the alkaline solution, while the complementary chain covalently linked to the sequencing chip is retained. Then add neutral liquid to the sequencing chip to neutralize the NaOH alkaline solution. The entire environment inside the sequencing chip becomes neutral, and the other end of the complementary chain will continue to hybridize with the second amplification primer on the sequencing chip. Add dNP and polymerase, and the polymerase will start from the second amplification primer and synthesize a new DNA chain along the complementary chain. At this time, the new DNA chain is completely complementary to the complementary chain and is exactly the same as the template chain. Then add NaOH alkaline solution to untie the two chains from each other, and then you can get two chains that are covalently connected to the sequencing chip and complementary to each other. Repeat this process, and the number of DNA chains will grow exponentially.

[0067] After amplification, the sequencing chip will retain the DNA double strands identical to the template strand and the complementary strand, respectively. Then, a specific reaction reagent is added to the sequencing chip to cut off the DNA strand synthesized from one of the amplification primers, for example, the DNA strand identical to the complementary strand is cut off, and the DNA strand identical to the template strand is retained. Then, a NaOH alkaline solution is added to the sequencing chip for washing. The alkaline solution disentangles the DNA double strands from each other, and the cut DNA strands are also washed away with the alkaline solution, ultimately leaving only single DNA strands on the sequencing chip. At this time, the number of single DNA strands retained on the sequencing chip is exponentially times the number at the beginning of amplification, thereby forming a DNA cluster, and all single DNA strands in a DNA cluster are identical. Then, a neutral solution is added, and all single DNA strands in the DNA cluster can be sequenced in a neutral solution environment.

[0068] It should be noted that the above-mentioned amplification reaction can be completed outside the sequencer, such as amplifying the library through experimental operations in the laboratory, or it can be completed inside the sequencer. When the amplification reaction is completed inside the sequencer, the library involved in the amplification reaction and various reaction reagents (for example, dNP, polymerase, NaOH alkaline solution, neutral solution, etc.) can be added to the sequencing chip through the liquid path system or fluid system of the sequencer. In addition, the above amplification reaction is only exemplary, and the present application is not limited to the use of bridge PCR amplification mode, and other amplification modes can also be used for amplification, for example, loop-mediated isothermal amplification (LAMP), nucleic acid-dependent amplification (NASBA, Nuclear acid sequence-based amplification), rolling circle amplification (RCA, Rolling Circle Amplification), multiplex probe amplification (MPA, Multiplex Probe Amplification), etc.

[0069] Sequencing reaction

[0070] Taking the principle of sequencing by synthesis as an example, when sequencing is performed, four dNTPs with fluorescent groups are added to the sequencing chip through the liquid system, each dNTP can only be synthesized with one of the four bases of ATCG, and the 3' end of the dNTP has been blocked by a blocking group (the blocking group includes but is not limited to an azide group), and then polymerase is added to the sequencing chip through the liquid system. Through the action of the polymerase, one of the four dNTPs will be synthesized with the complementary base on the single strand being sequenced, and because the 3' end of the dNTP is blocked by a blocking group, only one dNTP can be extended on the single strand being sequenced each time. After synthesis, specific chemical reagents are added to the sequencing chip through the liquid system to flush out excess dNTPs and polymerase. Next, the optical detection system can be used to excite the fluorescent group of the dNTP that has been synthesized on the single chain, causing the fluorescent group to emit a fluorescent signal. Since the fluorescent group of each dNTP on a single chain in a cluster will emit the same fluorescent signal, the fluorescent signal is amplified. Therefore, the optical detection system can collect the fluorescent signal and generate a fluorescent image.

[0071] The computer system processes and analyzes the fluorescence image to determine which dNTP is synthesized on the sequenced single strand, and then based on the principle of complementarity, it can be inferred which base is synthesized with the dNTP on the sequenced single strand. At this point, a sequencing cycle is completed.

[0072] Next, specific chemical reagents are added to the sequencing chip through the liquid system to cut off the blocking group and the fluorescent group, thereby exposing the hydroxyl group at the 3' end of the dNTP.

[0073] Then enter the next sequencing cycle and repeat the above process.

[0074] It is understood that one sequencing cycle can detect one base, and after multiple sequencing cycles, multiple bases in the sequenced single strand can be detected. Specifically, the number of sequencing cycles can be determined according to the set sequencing read length, for example, 150 or 300 sequencing cycles.

[0075] Of course, it should be noted that the above sequencing reactions are only exemplary, and the present application is not limited to the sequencing-by-synthesis principle, and other sequencing principles may also be used.

[0076] Sequencing system

[0077] See also Figure 1 As shown, the sequencing system includes: a sequencing chip 10, a chip platform 20, a reagent storage container 30, a liquid path system (or a flow guide system) 40, an optical detection system 50, a computer system 60 and a waste liquid storage container 70. Among them:

[0078] The sequencing chip 10 is configured to provide a reaction area for amplification reaction and sequencing reaction;

[0079] A chip platform 20 configured to fix and support the sequencing chip 10;

[0080] A reagent storage container 30, configured to store one or more mixed sample libraries, one or more reagents;

[0081] The liquid circuit system 40 is configured to controllably transport one or more mixed sample libraries and one or more reagents from the reagent storage container 30 to the sequencing chip 10 so as to perform an amplification reaction and a sequencing reaction in the sequencing chip 10, and controllably transport waste liquid after the reaction from the sequencing chip 10 to the waste liquid storage container 70;

[0082] An optical detection system 50 is configured to excite and collect fluorescent signals during a sequencing reaction and generate a fluorescent image based on the fluorescent signals;

[0083] A computer system 60 is configured to obtain a fluorescent image from the optical detection system 50 and identify a base sequence of the sample library based on the fluorescent image;

[0084] The waste liquid storage container 70 is configured to store the waste liquid generated after the reaction.

[0085] Sequencing chip

[0086] As a carrier of amplification reaction and sequencing reaction, the sequencing chip can provide a reaction area for these reactions, and this area is a channel. Generally, a sequencing chip 10 can include one or more channels 11 (for example, 2, 4, 6, 8), and the channels are isolated from each other. Figure 2 As shown, taking four channels as an example, each channel 11 has a small hole 12 at both ends for fluid (e.g., biological samples, reaction reagents) to flow in and out. The upper and lower surfaces in each channel are chemically modified and inoculated with two amplification primers in a covalent bond manner, and the two amplification primers are complementary to the adapters at both ends of the library to achieve amplification of the library.

[0087] During high-throughput sequencing, after each nucleic acid fragment (such as a DNA fragment) is clustered through an amplification reaction, in the sequencing reaction, any cluster will be presented in the sequencing image in the form of a fluorescent spot in each sequencing cycle, so a series of sequencing images will be generated during the sequencing process. Since the fluorescent spot is included in the sequencing image, the sequencing image is also called a fluorescent image. Due to the mechanical error of the sequencing device itself, the same photographed area has a position deviation in the sequencing images obtained in different sequencing cycles, which makes the fluorescent spot of the same cluster have a position deviation in the fluorescent images taken in different sequencing cycles, resulting in a decrease in the accuracy of the position of each cluster located during the sequencing process, which will cause the sequencing quality of the corresponding nucleic acid fragments to decrease. In order to reduce the problems caused by mechanical errors, it is necessary to construct an alignment template of the sequencing image in the current sequencing process in real time during the sequencing process, and use the alignment template to perform position correction and alignment on the fluorescent spots corresponding to the same cluster in different sequencing cycles. In this process, the selection of the alignment template is crucial. The richer the feature information that can be used as a reference contained in the alignment template, the less noise, and the higher the sequencing quality of the nucleic acid fragment.

[0088] Image alignment requires the selection of an alignment template as a reference. The alignment template needs to have clear features and a high signal-to-noise ratio, which is reflected in the alignment template, that is, the grayscale value difference between the area where the light spot is located and the area where the background is located is large, that is, the grayscale value of the pixel corresponding to the light spot is high, and the boundary of the light spot is clear, and the foreground and background are clearly distinguished. Using image reconstruction technology to generate an alignment template can effectively reduce image noise in the alignment template and highlight the feature information of the light spot in the image. Selecting the reconstructed image as the alignment template is more conducive to improving the registration accuracy.

[0089] However, currently, when constructing an alignment template for image alignment, it is usually achieved by sharpening and segmenting the fluorescent image used to construct the alignment template, which has a relatively low accuracy and results in low sequencing quality.

[0090] In addition, current reconstruction methods usually perform image reconstruction based on a single-frame original fluorescence image, which results in the reconstructed alignment template usually only containing feature information in a single-frame original fluorescence image, but lacking the common features included in multiple frames of original fluorescence images. This makes the segmentation accuracy of the alignment template obtained by template reconstruction based on a single-frame original fluorescence image low, which makes the impact of mechanical errors eliminated by image registration based on the reconstructed alignment template limited, resulting in the problem of low sequencing quality.

[0091] In the image reconstruction and alignment method provided by the embodiment of the present disclosure, after obtaining multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles, the multiple frames of fluorescence images to be reconstructed are input into a trained image reconstruction model, and the image reconstruction model performs image reconstruction processing on the multiple frames of fluorescence images to be reconstructed to obtain multiple frames of reconstructed fluorescence images, and determines an alignment template based on the multiple frames of reconstructed fluorescence images, and performs image alignment based on the alignment template, so that the image reconstruction model is used to reconstruct the fluorescence images to be reconstructed with higher accuracy. When performing image alignment based on the alignment template determined based on the reconstructed fluorescence images, the influence of mechanical errors can be eliminated to a greater extent, thereby improving sequencing quality.

[0092] When training the image reconstruction model in the embodiment of the present disclosure, the sample fluorescence images used are derived from multiple frames of original fluorescence images. This allows the image reconstruction model to learn relevant features of sample fluorescence images of multiple sequencing cycles during the training process, so that the reconstructed alignment template can include features of fluorescence images corresponding to multiple sequencing cycles, thereby improving the reconstruction accuracy of the image reconstruction model.

[0093] At the same time, in some embodiments, the signal-to-noise ratio of the sample fluorescence image meets certain conditions, so that the spot label of the sample fluorescence image has higher accuracy, and the image reconstruction model also has a stronger alignment template reconstruction capability. Therefore, the alignment template generated when the image reconstruction model is used for image reconstruction contains less noise on the basis of the common features of multiple frames of fluorescence images. Therefore, when performing image registration based on the reconstructed alignment template, the influence of mechanical errors can be eliminated to a greater extent, thereby improving sequencing quality.

[0094] To facilitate understanding of this embodiment, a method for generating an image reconstruction model disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the method for generating an image reconstruction model provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, which may be a computer system 60 in a sequencing system, or other computers with computing capabilities located inside or outside the sequencing system. In some possible implementations, the method for generating an image reconstruction model may be implemented by a processor calling computer-readable instructions stored in a memory.

[0095] The image reconstruction and alignment method provided by the embodiment of the present disclosure is described below.

[0096] See also Figure 3 FIG. 1 is a flowchart of an image reconstruction and alignment method provided by an embodiment of the present disclosure, wherein the method comprises steps S301 to S303, wherein:

[0097] S301: Acquire multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles.

[0098] S302: inputting the multiple frames of fluorescence images to be reconstructed into the trained image reconstruction model to obtain multiple frames of reconstructed fluorescence images output by the image reconstruction model, wherein the reconstructed fluorescence images contain the centroid information of the light spots in the fluorescence images to be reconstructed, and the centroid information of the light spots is used as a reference feature for image alignment;

[0099] S303: Determine an alignment template according to the reconstructed fluorescence images of multiple frames, and implement image alignment based on the alignment template.

[0100] The above S301 to S303 are described in detail below.

[0101] Regarding the above S301:

[0102] In a specific implementation, multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles can be determined, for example, by using either of the following two methods a1 or a2:

[0103] a1: According to the number of sequencing cycles, multiple frames of original fluorescence images of each sequencing cycle used for image alignment are determined as multiple frames of fluorescence images to be reconstructed.

[0104] a2: First, a reference fluorescence image is determined from multiple frames of original fluorescence images from multiple sequencing cycles used for image alignment, and then the reference fluorescence image is divided into multiple sub-images, and a target sub-image is determined from the multiple sub-images according to the signal-to-noise ratios corresponding to the multiple sub-images.

[0105] In a specific implementation, the reference fluorescence image may be, for example, an original fluorescence image corresponding to the first sequencing cycle among multiple sequencing cycles, or may be a frame of original fluorescence image selected from original fluorescence images according to the signal-to-noise ratio corresponding to the original fluorescence image.

[0106] Specifically, the embodiment of the present disclosure provides a specific method for screening a reference fluorescent image from an original fluorescent image, including:

[0107] Determine the signal-to-noise ratios corresponding to at least a portion of the original fluorescent images in the plurality of frames;

[0108] The reference fluorescence image is determined from the at least part of the original fluorescence image according to the signal-to-noise ratios respectively corresponding to the at least part of the original fluorescence image.

[0109] Here, the signal-to-noise ratio of the original fluorescence images corresponding to at least some of the sequencing cycles in the plurality of sequencing cycles may be determined first; then, according to the signal-to-noise ratio of the original fluorescence images corresponding to at least some of the sequencing cycles, a reference fluorescence image may be selected from the original fluorescence images corresponding to at least some of the sequencing cycles. The reference fluorescence image may be, for example, the original fluorescence image with the highest signal-to-noise ratio.

[0110] Afterwards, the size of the sub-image can be determined according to the size of the original fluorescent image, and the reference fluorescent image can be divided into multiple sub-images according to the size of the sub-image. Usually, the size of the original fluorescent image is an integer multiple of the size of the sub-image. For example, assuming that the size of the original fluorescent image is 4000*4000, the size of the sub-image can be determined to be 40*40. The specific size can be determined according to actual needs, and the embodiments of the present disclosure are not limited thereto.

[0111] When dividing the reference fluorescent image into a plurality of sub-images, the signal-to-noise ratio corresponding to each sub-image may be determined in the following manner:

[0112] Performing filtering processing on the multiple sub-images respectively to obtain filtered sub-images;

[0113] The signal-to-noise ratio of each sub-image is determined according to the pixel value of each pixel point in each sub-image and the pixel value of each pixel point in the corresponding filtered sub-image.

[0114] Exemplarily, the signal-to-noise ratio of any sub-image satisfies the following formula (1):

[0115]

[0116] In the above formula (1), M and N represent the number of pixels in the length and width of the sub-image, respectively. f(i,j) and g(i,j) are the pixel values ​​of the sub-image and the filtered sub-image at the pixel position (i,j), respectively. The signal-to-noise ratio is a quality parameter used to compare the evaluated image with the original image. The larger the signal-to-noise ratio, the better the image quality.

[0117] In addition, there may be other signal-to-noise ratio determination methods, such as power spectrum density, peak signal-to-noise ratio, etc. The signal-to-noise ratio determination method to be used may be selected according to actual needs, and the embodiments of the present disclosure do not limit this.

[0118] After obtaining the signal-to-noise ratios corresponding to the multiple sub-images, the multiple sub-images can be sorted in descending order of the signal-to-noise ratios, and then N sub-images with higher signal-to-noise ratios are selected as target sub-images. Alternatively, the signal-to-noise ratios corresponding to the multiple sub-images can be compared with a preset signal-to-noise ratio threshold, and the sub-image with a signal-to-noise ratio higher than the signal-to-noise ratio threshold is determined as the target sub-image.

[0119] Afterwards, according to the position of the target sub-image in the reference fluorescence image, sub-images having the same position are respectively intercepted from multiple frames of original fluorescence images as multiple frames of fluorescence images to be reconstructed.

[0120] In a specific implementation, when dividing a reference fluorescent image into a plurality of image blocks, a mapping relationship between pixel positions in each image block and the reference fluorescent image is established. For example, the pixel position corresponding to the first pixel position in the upper left corner of each image block in the reference fluorescent image can be recorded. After determining a target image block, the pixel position corresponding to the target image block in the reference fluorescent image can be determined as a cut-off position based on the mapping relationship, and the size of the image block can be determined as a cut-off size. Based on the cut-off position and the cut-off size, multiple frames of fluorescent images to be reconstructed are cut off from multiple frames of original fluorescent images.

[0121] Regarding the above S302:

[0122] In the specific implementation, see Figure 4 As shown, the embodiment of the present disclosure provides a specific method for training an image reconstruction model, including:

[0123] S401: performing image segmentation processing on the multiple frames of sample fluorescence images according to the intensity values ​​corresponding to the respective pixel positions in the multiple frames of sample fluorescence images, and generating light spot labels of the multiple frames of sample fluorescence images according to the results of the segmentation processing on the multiple frames of sample fluorescence images; the multiple frames of sample fluorescence images are used for model training and are derived from multiple sequencing cycles;

[0124] S402: Training the image reconstruction model based on multiple frames of sample fluorescence images and their corresponding spot labels.

[0125] In the above S401: for example, the following method can be used to obtain multiple frames of sample fluorescence images:

[0126] A plurality of frames of original fluorescence images from a plurality of sequencing cycles for model training are obtained, and a plurality of frames of sample fluorescence images for model training are screened from the plurality of frames of original fluorescence images. For example, according to the number of sequencing cycles, the original fluorescence images of each sequencing cycle are determined as sample fluorescence images.

[0127] Alternatively, the multiple frames of original fluorescence images from multiple sequencing cycles can be obtained, a reference fluorescence image can be determined from the multiple frames of original fluorescence images, and the reference fluorescence image can be divided into multiple sub-images; based on the signal-to-noise ratios corresponding to the multiple sub-images, some sub-images can be determined from the multiple sub-images as the multiple frames of sample fluorescence images.

[0128] Here, the method of determining multiple frames of sample fluorescence images from multiple frames of original fluorescence images derived from multiple sequencing cycles used for model training is similar to the method of determining multiple frames of fluorescence images to be reconstructed from multiple frames of original fluorescence images derived from multiple sequencing cycles used for image alignment in the above S301, and will not be repeated here.

[0129] When image segmentation processing is performed on the sample fluorescence image according to the intensity values ​​corresponding to each pixel position in multiple frames of sample fluorescence images, for example, the centroid of the light spot in the sample fluorescence image is taken as the foreground content, and other areas except the centroid of the light spot are taken as the background content, and the sample fluorescence image is subjected to image segmentation processing, the obtained light spot label may, for example, include: the light spot centroid information and background information in the sample fluorescence image.

[0130] For example, at least one of the following methods b1 and b2 may be used to generate a spot label of the sample fluorescence image:

[0131] b1: See Figure 5 As shown, the embodiment of the present disclosure provides a method for generating a spot label of a sample fluorescence image, comprising:

[0132] S501: performing image alignment on the multiple frames of sample fluorescence images respectively to obtain multiple frames of aligned sample fluorescence images;

[0133] S502: determining a base sequence corresponding to a candidate pixel position of a light spot in the aligned sample fluorescence images according to a gray value of each pixel position in the aligned sample fluorescence images; the candidate pixel position of the light spot includes at least a part of the pixel positions of the light spot in the aligned sample fluorescence images;

[0134] S503: Determine the pixel position of the centroid of the fluorescent light spot from the candidate light spot pixel positions based on the consistency information between the base sequence of the candidate light spot pixel position and the base sequence of the candidate light spot pixel position in the corresponding target neighborhood, and / or based on the fault tolerance information between the base sequence of the candidate light spot pixel position and the base sequence in the reference genome;

[0135] S504: Generate the light spot label according to the pixel position of the centroid of the fluorescent light spot.

[0136] Regarding the above S501:

[0137] In a specific implementation, the original fluorescence image includes fluorescence images corresponding to multiple sequencing cycles; and the fluorescence image corresponding to each sequencing cycle usually has multiple color channels. The mainstream second-generation sequencing uses two channels or four channels for taking pictures (different channels are for different wavelengths of fluorescence emitted by different base types), and takes pictures separately under excitation light of different wavelengths to form different color channels in the fluorescence image.

[0138] In the process of high-throughput gene sequencing, for the same cluster, each sequencing cycle needs to be subjected to a chemical reaction and detected by a high-resolution optical detection system 50, and the fluorescence image is the original fluorescence image.

[0139] During the sequencing process, in the process of collecting fluorescence signals and generating fluorescence images, the optical detection system 50 and the chip platform 20 are constantly moving relative to each other. For example, the chip platform 20 moves relative to the optical detection system 50, and this movement can easily cause mechanical errors, resulting in deviations in the light spot positions of the same cluster in the fluorescence images of different sequencing cycles or even the same sequencing cycle, making it impossible to accurately align the base sequences of the same cluster, thereby affecting the accuracy of identifying the cluster base type. Therefore, in order to reduce the problems caused by position deviation, after obtaining multiple frames of sample fluorescence images of multiple sequencing cycles, it is necessary to first perform image alignment on the multiple frames of sample fluorescence images of multiple sequencing cycles.

[0140] Regarding the above S502:

[0141] The candidate pixel position of the light spot is, for example, a pixel position in the sample fluorescence image that has a high probability of belonging to the light spot area.

[0142] When determining the base sequence, for example, the following method can be used:

[0143] According to the gray value of each pixel point in the sample fluorescence image after multi-frame alignment, the fluorescence spot is taken as the foreground, and the image segmentation of the foreground and the background is performed on the sample fluorescence image after multi-frame alignment to obtain the candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment; and the candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment is fused to obtain the fused candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment;

[0144] and performing image sharpening on the multiple aligned frames of sample fluorescence images to obtain multiple sharpened frames of sample fluorescence images;

[0145] Each pixel position in the fused candidate fluorescent spot area is used as a spot candidate pixel position, and a base sequence corresponding to the spot candidate pixel position is determined according to the gray value of the spot candidate pixel position in the multi-frame sharpened sample fluorescent image.

[0146] In a specific implementation, when segmenting multiple frames of sample fluorescence images based on the pixel values ​​of each pixel in the sample fluorescence images after multi-frame alignment, for example, a pixel gray value threshold can be determined, and the gray value of each pixel can be compared with the gray value threshold; if the gray value of a certain pixel is greater than or equal to the gray value threshold, the pixel is used as a pixel in the candidate spot area; if the gray value of a certain pixel is less than the gray value threshold, the pixel is used as a pixel in the background area, thereby implementing the segmentation of the sample fluorescence image to obtain the fluorescent spot areas corresponding to the multiple frames of sample fluorescence images. The above gray value threshold can be pre-set and determined based on experience, or can be determined using the Otsu method, Kmeans classification algorithm, etc., which is not limited in the embodiments of the present disclosure.

[0147] Then, the union of the candidate fluorescent spot regions corresponding to the multiple frames of sample fluorescent images is determined to obtain the fused candidate fluorescent spot region. After obtaining the fused candidate fluorescent spot region, each pixel position in the fused candidate fluorescent spot region is used as a spot candidate pixel position to determine the base sequence corresponding to each spot candidate pixel position.

[0148] When determining the base sequence corresponding to each candidate light spot pixel position, the base types corresponding to each candidate light spot pixel position and multiple sequencing cycles can be determined based on the grayscale values ​​corresponding to each candidate light spot pixel position in multiple frames of sharpened sample fluorescence images; then the base types corresponding to each candidate light spot pixel position in multiple sequencing cycles are combined to form the base sequence of each candidate light spot pixel position.

[0149] like Figure 6 In the example shown, assuming that there are N sequencing cycles, the corresponding fluorescent head image to be marked has N frames; for pixel 1 located at the same candidate pixel position of the light spot in the N frames of sample fluorescence images, the base types of the pixel 1 in multiple sequencing cycles can be obtained; assuming that the corresponding base type in the first sequencing cycle is A, the corresponding base type in the second sequencing cycle is T, the corresponding base type in the third sequencing cycle is G, ..., and the corresponding base type in the Nth sequencing cycle is A, then the base sequence corresponding to the pixel 1 is: ATG ... A. The base sequences corresponding to the multiple candidate pixel positions of the light spot constitute a pixel sequence set.

[0150] Here, when determining the base type of each candidate pixel position of each spot in different sequencing cycles, for example, a real-time unsupervised classification model can be used, or a target classification model can be obtained through multiple iterations of training to identify the base type. The classification model can use K-nearest neighbor, decision tree, random forest, SVM, Xgboost and other algorithms, which are not limited in the embodiments of the present disclosure.

[0151] Regarding the above S503:

[0152] Regarding the case where the pixel position of the centroid of the fluorescent light spot is determined by using the consistency information between the base sequence of the light spot candidate pixel position and the base sequence of the light spot candidate pixel position in the corresponding target neighborhood:

[0153] After the DNA molecules are bound to the surface of the circulation pool, thousands of DNA copies will be generated around them through PCR amplification. In theory, these copies are considered to be exactly the same, but due to the influence of the reaction environment such as enzymes, errors will gradually accumulate during the amplification clustering process. Therefore, under normal circumstances, the base sequence corresponding to the pixel point at the center of the light spot is more consistent, and the consistency of the base sequence corresponding to the pixel point closer to the edge of the light spot is lower. Therefore, in the embodiment of the present disclosure, when determining the pixel position of the centroid of the fluorescent light spot from multiple candidate light spot pixel positions, the consistency of the base sequence of each candidate light spot pixel position in the sample fluorescent image and the base sequence of other surrounding candidate light spot pixel positions is determined, and then the pixel position belonging to the centroid of the fluorescent light spot is determined from the candidate light spot pixel positions based on the consistency of the base sequence of the candidate light spot pixel position.

[0154] Specifically, the embodiment of the present disclosure provides a specific method for determining the consistency information of the base sequence between each of the candidate light spot pixel positions and each of the candidate light spot pixel positions in the corresponding target neighborhood, including:

[0155] Determine a target neighborhood of each candidate pixel position of the light spot;

[0156] Determine the degree of difference between the base sequence corresponding to each of the light spot candidate pixel positions and the base sequences corresponding to each of the light spot candidate pixel positions in the corresponding target neighborhood; the degree of difference includes: the number of different bases between the base sequence of each of the light spot candidate pixel positions and the base sequence of the light spot candidate pixel positions in the corresponding target neighborhood, or the percentage of the number of different bases in the base sequence corresponding to the light spot candidate pixel position;

[0157] The consistency information is determined according to the degree of difference.

[0158] In a specific implementation, the target neighborhood corresponding to each candidate light spot pixel position may be, for example, other candidate light spot pixel positions adjacent to the candidate light spot pixel position. i ,y j ), the other candidate pixel positions of the light spot located in the neighborhood of the target include the following pixel positions: (x i-1 ,y j )、(x i ,yj-1 )、(x i+1 ,y j )、(x i ,y j+1 ).

[0159] Alternatively, the candidate pixel positions of the light spot can also be within an area with the candidate pixel position of the light spot as the center and k pixel positions as the radius. For example, when k is 1, the candidate pixel positions of the light spot located in the target neighborhood include the following pixel positions: (x i-1 ,y j )、(x i ,y j-1 )、(x i+1 ,y j )、(x i ,y j+1 )、(x i-1 ,y j-1 )、(x i+1 ,y j-1 )、(x i+1 ,y j+1 )、(x i-1 ,y j+1 ).

[0160] like Figure 7 As shown, the embodiment of the present disclosure also provides an example of the relative position relationship between a spot candidate pixel position N and other spot candidate pixel positions N' in the corresponding target neighborhood. In this example, k = 1, and all spot candidate pixel positions located within a distance of 1 near the spot candidate pixel position can be used as the domain pixel position corresponding to the spot candidate pixel position N. For example, a first domain pixel position is Figure 7 N' in.

[0161] After the target neighborhood pixel position corresponding to each pixel position is determined, the degree of difference between the base sequence corresponding to the pixel position and the base sequence corresponding to the target neighborhood pixel position is determined.

[0162] Here, for example, Euclidean distance, cosine similarity, Hamming distance, Sorensen-Dice index, Dynamic Time Warping (DTW), etc. can be used to determine the degree of difference between each pixel position and the corresponding target neighborhood pixel position.

[0163] The consistency information between each light spot candidate pixel position and other light spot candidate pixel positions in its target neighborhood is determined according to the degree of difference.

[0164] Here, for example, at least one of the sum, average, etc. of the degree of difference between each light spot candidate pixel position and the surrounding light spot candidate pixel positions in the target neighborhood can be calculated, and the sum or average value can be used as a value to measure the consistency information. The larger the sum, the lower the consistency between each light spot candidate pixel position and other light spot candidate pixel positions in the corresponding target neighborhood; the smaller the sum, the higher the consistency between each light spot candidate pixel position and other light spot candidate pixel positions in the corresponding target neighborhood.

[0165] When determining the candidate pixel position belonging to the center area of ​​the light spot from multiple candidate pixel positions of the light spot according to the consistency information, the candidate pixel position of the light spot with the highest consistency can be selected from the multiple candidate pixel positions of the light spot as the pixel position of the centroid of the fluorescent light spot. Alternatively, the candidate pixel position of the light spot with higher consistency can be selected as the candidate pixel position belonging to the center area of ​​the light spot.

[0166] Ideally, the base sequence of the centroid of the spot in the sample fluorescence image has the highest consistency. In the same spot, the farther from the spot centroid, the lower the base sequence consistency. The method of spot shape recognition based on base sequence consistency has high signal recognition accuracy for the spot centroid position. The farther from the spot centroid, the lower the recognition accuracy. Therefore, after determining the candidate pixel position belonging to the center area of ​​the fluorescent spot, the candidate pixel position belonging to the center area of ​​the fluorescent spot can be used as a seed point to perform regional growth processing to obtain a complete spot area, and then determine the pixel position belonging to the centroid of the fluorescent spot from the complete spot area.

[0167] When performing regional growing processing, it is necessary to select high-quality (i.e., high consistency) points as candidate seed points. The disclosed embodiment uses high-resolution fluorescence images of multiple sequencing cycles to perform base recognition for each candidate pixel position of the light spot in the high-resolution image aligned with the alignment template, and outputs the base sequence of each candidate pixel position of the light spot to obtain a base sequence set of each candidate pixel position of the light spot. The base sequence of the candidate pixel position N (i, j) of the light spot is compared with the base sequence of the surrounding candidate pixel positions of the light spot, and the boundary distance is calculated, and the base sequence with better consistency, that is, the base sequence with the number of mismatches with the surrounding base sequences is lower than the threshold, is selected as the set of candidate seed points, that is, the set consisting of the candidate pixel positions in the center area of ​​the light spot.

[0168] Specifically, the embodiment of the present disclosure also provides a specific method for performing region generation processing on seed points:

[0169] Perform multiple iterations and perform the following region growing process in each iteration:

[0170] The target seed point corresponding to the current iteration cycle is determined from the candidate pixel positions belonging to the center area of ​​the fluorescent light spot whose light spot affiliation has not been determined.

[0171] According to the position information of the target seed point, the candidate light spot pixel position located in the second neighborhood of the target seed point is determined.

[0172] Based on the similarity between the base sequence of the target seed point and the base sequence of the candidate light spot pixel position corresponding to the second neighborhood, the target seed point is subjected to region growing processing to obtain the light spot region of the light spot where the target seed point is located.

[0173] Specifically, if the current iteration cycle is the first iteration cycle, the first pixel position to which the light spot belongs is determined to be all the candidate pixel positions belonging to the center area of ​​the fluorescent light spot determined in the above steps. Any of the candidate pixel positions belonging to the center area of ​​the fluorescent light spot can be determined as the target seed point corresponding to the current iteration cycle. The candidate pixel positions belonging to the center area of ​​the fluorescent light spot can also be sorted from high to low in accordance with the consistency information corresponding to each candidate pixel position belonging to the center area of ​​the fluorescent light spot, and the candidate pixel position belonging to the center area of ​​the fluorescent light spot with the highest consistency can be determined as the target seed point of the first iteration cycle.

[0174] If the current iteration cycle is not the first iteration cycle, since in each iteration cycle before the current iteration cycle, some of the candidate pixel positions belonging to the center area of ​​the fluorescent spot have been used as seed points for regional growth processing, or in the process of regional growth processing of the seed points, the spot affiliation consistent with the seed point will be determined for some of the candidate pixel positions belonging to the center area of ​​the fluorescent spot, so the candidate pixel positions belonging to the center area of ​​the fluorescent spot whose affiliation has been determined can be deleted from the candidate center set; in the non-first iteration cycle, the candidate pixel positions belonging to the center area of ​​the fluorescent spot whose affiliation has not been determined include the remaining candidate pixel positions belonging to the center area of ​​the fluorescent spot in the candidate center set. At this time, any candidate pixel position belonging to the center area of ​​the fluorescent spot in the candidate center set can be used as the target seed point of the current iteration cycle, or the candidate pixel position with the highest consistency can be determined from the remaining candidate pixel positions belonging to the center area of ​​the fluorescent spot in the candidate center set in order of consistency from high to low as the target seed point of the current iteration cycle.

[0175] The second neighborhood may be a region in the image coordinate system whose distance from the target seed point is less than a preset distance range. It may be the same as the target neighborhood or different from the target neighborhood. When determining the candidate light spot pixel position in the second neighborhood according to the position information corresponding to the target seed point, for example, the candidate light spot pixel position whose light spot has not been determined to belong to in the current iteration cycle is determined from the second neighborhood as the second neighborhood pixel position corresponding to the target seed point.

[0176] Specifically, the second neighborhood may be, for example, a four-neighborhood or an eight-neighborhood of the target seed point.

[0177] In a specific implementation, based on whether the similarity between the base sequence of the target seed point and the base sequence between the candidate light spot pixel position in the corresponding second neighborhood meets the preset similarity condition, the second pixel position belonging to the same light spot area as the target seed point is determined from the second neighborhood pixel positions; and the following steps are executed repeatedly until no new second pixel position is generated: the second pixel position is used as a growable pixel position, and based on whether the similarity between the base sequence of the growable pixel position and the base sequence between the candidate light spot pixel position in the corresponding third neighborhood meets the preset similarity condition, the second pixel position belonging to the same light spot area as the target seed point is determined from the third neighborhood pixel positions.

[0178] In this way, the center of the light spot is obtained through the complete region growing process of the above process, and then the centroid of the light spot is determined from the region where the center of the light spot is located.

[0179] For the case where the pixel position of the centroid of the fluorescent light spot is determined from the light spot candidate pixel positions based on the error tolerance information between the base sequence of the light spot candidate pixel position and the base sequence in the reference genome:

[0180] The fault tolerance information between the base sequence of the candidate pixel position of the light spot and the base sequence in the reference genome, for example, refers to the number of differential bases between the base sequence of the candidate pixel position of the light spot and the base sequence in the reference genome, or the proportion of the number of differential bases to the total number of bases in the base sequence of the candidate pixel position of the light spot. For example, the candidate pixel positions of the light spot can be traversed, and the base sequences of the traversed candidate pixel positions of the light spot can be matched with the base sequences in the reference genome to determine the fault tolerance information between the two, and the candidate pixel position of the light spot with the best fault tolerance can be selected as the pixel position of the centroid of the fluorescent light spot. Alternatively, the candidate pixel position of the light spot with better fault tolerance can be selected as the candidate pixel position belonging to the center area of ​​the light spot. For example, the determined fault tolerance information is compared with a preset fault tolerance threshold; if the fault tolerance information is greater than the fault tolerance threshold, it is considered that the difference between the base sequence of the traversed candidate pixel position of the light spot and the base sequence in the reference genome is too large, the fault tolerance is poor, and it does not belong to the center area of ​​the light spot; if the fault tolerance information between the two is less than or equal to the fault tolerance threshold, it is considered that the difference between the base sequence of the traversed candidate pixel position of the light spot and the base sequence in the reference genome is small, the fault tolerance is good, and there is a high possibility that it belongs to the center area of ​​the light spot, so the traversed candidate pixel position of the light spot is used as the candidate pixel position of the light spot area.

[0181] Afterwards, the candidate pixel positions of the light spot belonging to the central area of ​​the light spot can be used as seed points to perform region growing processing to obtain the area where the light spot is located, and the pixel position of the centroid of the fluorescent light spot is determined from the area where the light spot is located.

[0182] Here, the method of performing region growing processing on the candidate pixel positions of the light spot belonging to the center area of ​​the light spot is similar to the above-mentioned region growing method, which will not be described in detail here.

[0183] It should be noted here that the reference genome can be the genome of any species whose base sequence is known. For example, the reference genome can be Genome.

[0184] Regarding the above S504:

[0185] After the target pixel position is determined from the sample fluorescence image, since the target pixel position belongs to the area where the centroid of the fluorescent spot is located, the pixel value of the target pixel position can be assigned to the first value, and the pixel value of the non-target pixel position can be assigned to the second value, thereby obtaining the result of image segmentation processing of the sample fluorescence image.

[0186] After obtaining the result of image segmentation processing on the sample fluorescence image, the result of the image segmentation processing can be directly used as a label to obtain a binary spot label, or specific pixel values ​​can be assigned to the pixel positions belonging to the centroid of the fluorescence spot, and other pixel values ​​can be assigned to the pixel positions that do not belong to the centroid of the fluorescence spot to obtain a spot label.

[0187] In addition, in another embodiment of the present disclosure, before performing image segmentation of foreground and background on the sample fluorescence images after the multi-frame alignment according to the gray value of each pixel point in the sample fluorescence images after the multi-frame alignment, taking the centroid of the fluorescence spot as the foreground, respectively, the sample fluorescence images after the multi-frame alignment can also be subjected to image denoising respectively to obtain the sample fluorescence images after the multi-frame denoising; thereafter, performing image segmentation of foreground and background on the sample fluorescence images after the multi-frame alignment according to the gray value of each pixel point in the sample fluorescence images after the multi-frame alignment according to the centroid of the fluorescence spot as the foreground, respectively, including:

[0188] According to the grayscale value of each pixel in the sample fluorescence image after the multi-frame noise reduction processing, the centroid of the fluorescence spot is used as the foreground, and the image segmentation of the foreground and background is performed on the sample fluorescence image after the multi-frame alignment.

[0189] b2: The present disclosure provides another specific method for generating a spot label of a sample fluorescence image, including:

[0190] According to the gray value of each pixel position in the multi-frame sample fluorescence image, the fluorescence spot is taken as the foreground, and the multi-frame sample fluorescence image is segmented into the foreground and the background respectively, so as to obtain the candidate fluorescence spot area in the multi-frame sample fluorescence image;

[0191] Parabolic interpolation processing is performed on the candidate fluorescent spot areas in the multiple frames of sample fluorescent images to obtain the fluorescent spot centroid of the candidate fluorescent spot area and generate a spot label.

[0192] When the multiple frames of original fluorescence images are segmented into foreground and background respectively based on the grayscale value of each pixel position in the multiple frames of sample fluorescence images, the fluorescence spot is taken as the foreground, and image segmentation of foreground and background is performed on the multiple frames of sample fluorescence images respectively. For example, image denoising can be performed on the multiple frames of sample fluorescence images respectively, and based on the grayscale value of each pixel position in the multiple frames of sample fluorescence images after denoising, the fluorescence spot is taken as the foreground, and image segmentation of foreground and background is performed on the multiple frames of sample fluorescence images respectively.

[0193] In a specific implementation, when performing image noise reduction on a sample fluorescence image, for example, a GL (Grünwald-Letnikov) fractional order differential method may be used, and the specific process may be as follows:

[0194] (1): The sample fluorescence image is grayed and normalized in turn, and the gray value of each pixel in the sample fluorescence image is normalized to the interval [0,1].

[0195] (2): Fractional differential mask construction.

[0196] (3): The constructed fractional differential mask is used as the convolution kernel to perform convolution processing on the normalized sample fluorescence image, and the convolution processing result is denormalized to obtain the denoised multi-frame sample fluorescence image.

[0197] After obtaining the denoised multi-frame sample fluorescence images, the segmentation threshold can be determined according to the intensity values ​​(i.e., grayscale values) corresponding to each pixel position in the sharpened image block, such as using the mean of the intensity values ​​corresponding to each pixel position as the segmentation threshold, or determining the segmentation threshold based on any one of the Otsu method, K-means, support vector machine algorithm, etc., and performing segmentation processing on the sharpened image block based on the segmentation threshold.

[0198] In the segmentation process, for example, each pixel position in the multi-frame sample fluorescence image after noise reduction can be traversed, and the intensity value of the traversed pixel position can be compared with the segmentation threshold; when the intensity value of the traversed pixel position is greater than or equal to the segmentation threshold, the traversed pixel position is determined as a pixel position belonging to the light spot; when the intensity value of the traversed pixel position is less than the segmentation threshold, the traversed pixel position is determined as the background to obtain a segmented image. The pixel value of each pixel position in the segmented image is used to indicate whether the pixel position belongs to the area where the light spot is located.

[0199] Afterwards, the segmented image is subjected to parabolic interpolation processing to obtain the centroid of the fluorescent spot in the candidate fluorescent spot area.

[0200] When performing parabolic interpolation processing on the segmented image, for example, the area where the light spot is located can be determined first according to the pixel values ​​of each pixel position in the segmented image, and the target point to be interpolated can be determined from the area where the light spot is located, and then the neighborhood of the target point can be determined, and the neighborhood can be, for example, a 3*3 neighborhood, a 5*5 neighborhood, etc., which can be determined specifically according to the size of the area where the light spot is located. Then, a parabolic equation is established, that is, the following formula (2):

[0201] f(x,y)=a0+a1x+a2y+a3x 2 +a4xy+a5y 2 (2)

[0202] In formula (2), f(x, y) represents the pixel value of the pixel position at position (x, y). The coordinates of each pixel position in the neighborhood and the corresponding pixel value are substituted into the above parabola equation to obtain an equation group, and the coefficients a0, a1, a2, a3, a4, and a5 in the equation group are solved to obtain the coefficient value of each coefficient.

[0203] Afterwards, the coordinate position of the target point is substituted into the parabola equation to obtain the interpolation result of the target point.

[0204] All pixel positions in the segmented image that need to be interpolated by parabola are traversed to finally obtain a parabola interpolation image of the segmented image; the parabola interpolation image is the result of image segmentation processing on the sample fluorescence image.

[0205] The light spot information includes, for example, position information of the centroid of the light spot in the image coordinate system.

[0206] Afterwards, a light spot label can be generated according to the light spot information.

[0207] Regarding the above S402: after the light spot label of the sample fluorescence image is generated based on the above method, the image reconstruction model can be trained using the sample fluorescence image and the light spot label.

[0208] In a specific implementation, the image reconstruction model includes, for example, any one of the following models: Unet, modified Unet, Res-UNet, Unet based on attention mechanism, Attention Res-Unet, etc.

[0209] In the image reconstruction model, a batch normalization (BN) layer is included; this can make the model training converge quickly, help solve the problem of internal covariate shift, make the feature distribution of each layer more stable, and enhance the generalization of the model. The BN layer can be added between two convolutional layers. In order to prevent the model from overfitting, a Dropout layer is added to randomly discard a part of the neurons so that the model does not rely on certain local features, thereby enhancing the generalization ability of the model; the dropout layer can be placed in the middle layer of the Unet layer, or in the top three layers or the bottom three layers, and the parameters can be set to 0.3 or 0.5. In order to prevent the inhomogeneity of the model, the diversity of the data can be increased, and a data enhancement layer can be added to add corresponding noise to the input data, or rotate it, which can increase the diversity of the training data and enhance the generalization of the model.

[0210] When training the image reconstruction model to be trained based on the sample fluorescence image and the spot label, for example, the following method can be used:

[0211] Performing fusion processing on multiple frames of sample fluorescence images to obtain a fused fluorescence image;

[0212] Inputting the fused fluorescence image into the image reconstruction model to be trained to obtain a first reconstructed image; wherein the pixel value of each pixel position in the first reconstructed image represents the probability that the pixel position belongs to the centroid of the light spot; determining a first model loss according to the first reconstructed image and the light spot label;

[0213] and / or,

[0214] Inputting multiple frames of sample fluorescence images into the image reconstruction model to be trained to obtain a second reconstructed image; wherein the pixel value of each pixel position in the second reconstructed image represents the probability that the pixel position belongs to the centroid of the light spot;

[0215] The first model loss and / or the second model loss are used to adjust the parameters of the image reconstruction model to be trained to obtain the target image reconstruction model.

[0216] In a specific implementation, the spot label itself constitutes a label image of the same size as the sample fluorescence image. The pixel value of each pixel position in the label image represents whether the corresponding pixel position in the sample fluorescence image belongs to the centroid of the spot. When the sample fluorescence images with the same interception position corresponding to multiple sequencing cycles are fused, for example, the pixel values ​​of multiple sequencing cycles corresponding to the same pixel position can be weighted and accumulated, and the sample fluorescence images of multiple sequencing cycles can be fused into one image by weighted averaging, that is, a fused fluorescence image.

[0217] The fused fluorescence image and the spot label are input into the image reconstruction model to be trained, and the fused fluorescence image is reconstructed using the image reconstruction model to be trained. The prediction result of the image reconstruction model is the first reconstructed image of the fused fluorescence image. According to the obtained first reconstructed image and the spot label, the first model loss can be determined.

[0218] In addition, in another embodiment of the present disclosure, sample fluorescence images corresponding to a plurality of sequencing cycles may be input into the image reconstruction model to be trained to obtain second reconstructed images corresponding to each sample fluorescence image, and then the second model loss may be determined using each second reconstructed image and the spot label;

[0219] Afterwards, the parameters of the image reconstruction model to be trained can be adjusted based on the first model loss and / or the second model loss to obtain the image reconstruction model.

[0220] After the image reconstruction model is obtained, for example, the method may further include: performing pruning processing and / or quantization processing on the image reconstruction model.

[0221] The purpose of pruning is to reduce the scale of the image reconstruction model, so as to reduce the amount of computation required by the image reconstruction model during the reasoning process, and make the target alignment template intermediate model after pruning more suitable for deployment in devices with lower computing power, such as nucleic acid sequencing systems, host computers in optical detection systems, etc. The purpose of quantization is to reduce the accuracy of parameters in the image reconstruction model, so as to reduce the computing power required during the reasoning process.

[0222] After the image reconstruction model is obtained by training in the above manner, the image reconstruction model can be used to process the fluorescence image to be reconstructed to obtain multiple frames of reconstructed fluorescence images.

[0223] Regarding the above S303:

[0224] In a specific implementation, after the fluorescence image to be reconstructed is reconstructed according to the image reconstruction model to obtain multiple frames of reconstructed fluorescence images, for example, an image with the most light spot feature information can be selected from the reconstructed fluorescence images as an alignment template. For example, the number of pixel points belonging to the light spot in each frame of the reconstructed fluorescence image, or the number of light spots, can be counted to select the seat alignment template with the most light spot feature information, and then the alignment template can be used to align the original fluorescence images.

[0225] like Figure 8 In the example shown, Figure 8 In the figure, a is the original fluorescence image; b is the reconstructed fluorescence image obtained by reconstructing the fluorescence image to be reconstructed by the target image reconstruction model; and c is the alignment template obtained by post-processing the reconstructed fluorescence image.

[0226] After obtaining the reconstructed fluorescence image or alignment template, the reconstructed fluorescence image or alignment template is used to perform alignment processing on each frame of the original fluorescence image.

[0227] After the alignment process, the original fluorescent image after the alignment process can be subjected to light spot recognition to obtain the base sequence corresponding to the gene fragment to be tested.

[0228] The alignment template obtained above that highlights the characteristic information is used as the alignment template for subsequent registration. The alignment template contains very little noise and highlights the characteristic information of the light spot in the image. It is then used as an alignment template to align other sequencing images, which can improve the accuracy of sequencing.

[0229] Table 1 below shows the quality comparison of phage libraries, and Table 2 shows the performance comparison (time calculation: in a single experiment, from the beginning to the end of alignment; and each method was tested three times in the same environment, and the average value was the final time).

[0230] Table 1

[0231]

[0232] Table 2

[0233] Method Type Run time Non-AI model approaches 14min56s AI model method (no post-processing added) 15min01s AI model method (add post-processing) 16min22s

[0234] From the above results, it can be found that: compared with the existing technical method, the error rate of the algorithm without post-processing is optimized by 0.027%, and the matching rate is improved by 0.089%; compared with the algorithm without post-processing, the error rate of the algorithm with post-processing is optimized by 0.088%, and the matching rate can be improved by 0.016%.

[0235] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0236] The present disclosure also provides a computer device, such as Fig. 9 FIG. 1 is a schematic diagram of a computer device structure provided in an embodiment of the present disclosure, including:

[0237] A processor 91 and a memory 92; the memory 92 stores machine-readable instructions executable by the processor 91, and the processor 91 is used to execute the machine-readable instructions stored in the memory 92. When the machine-readable instructions are executed by the processor 91, the processor 91 performs the following steps:

[0238] Acquire multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles;

[0239] Inputting the multiple frames of fluorescence images to be reconstructed into the trained image reconstruction model to obtain multiple frames of reconstructed fluorescence images output by the image reconstruction model, wherein the reconstructed fluorescence images contain the centroid information of the light spot in the fluorescence images to be reconstructed;

[0240] An alignment template is determined according to the reconstructed fluorescence images of multiple frames, and image alignment is achieved based on the alignment template.

[0241] The above-mentioned memory 92 includes internal memory 921 and external memory 922; the memory 921 here is also called internal memory, which is used to temporarily store the calculation data in the processor 91 and the data exchanged with the external memory 922 such as the hard disk. The processor 91 exchanges data with the external memory 922 through the internal memory 921.

[0242] The specific execution process of the above instructions can refer to the steps of the image reconstruction model generation method or the fluorescence image registration method described in the embodiments of the present disclosure, which will not be repeated here.

[0243] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for generating an image reconstruction model or the method for registering a fluorescence image described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0244] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the image reconstruction model generation method or the fluorescence image registration method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0245] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0246] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the aforementioned embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. An image reconstruction and alignment method, characterized in that: The method comprises: Acquire multiple frames of fluorescence images to be reconstructed from multiple sequencing cycles; Inputting the multiple frames of fluorescence images to be reconstructed into the trained image reconstruction model to obtain multiple frames of reconstructed fluorescence images output by the image reconstruction model, wherein the reconstructed fluorescence images contain the centroid information of the light spots in the fluorescence images to be reconstructed, and the centroid information of the light spots is used as a reference feature for image alignment; An alignment template is determined according to the reconstructed fluorescence images of multiple frames, and image alignment is achieved based on the alignment template.

2. The method according to claim 1, characterized in that The image reconstruction model is trained in the following manner; According to the intensity values ​​corresponding to the respective pixel positions in the multiple frames of sample fluorescence images, the multiple frames of sample fluorescence images are respectively subjected to image segmentation processing, and the light spot labels of the multiple frames of sample fluorescence images are generated according to the results of the segmentation processing of the multiple frames of sample fluorescence images; the multiple frames of sample fluorescence images are used for model training and are derived from multiple sequencing cycles; The image reconstruction model is trained based on multiple frames of sample fluorescence images and their corresponding spot labels.

3. The method according to claim 2, characterized in that The method of performing image segmentation processing on the multiple frames of sample fluorescence images according to the intensity values ​​corresponding to the respective pixel positions in the multiple frames of sample fluorescence images, and generating spot labels of the multiple frames of sample fluorescence images according to the results of the segmentation processing on the multiple frames of sample fluorescence images, comprises: Performing image alignment on multiple frames of sample fluorescence images to obtain multiple frames of aligned sample fluorescence images; Determine, according to the grayscale value of each pixel position in the sample fluorescence image after the multi-frame alignment, the base sequence corresponding to the candidate pixel position of the light spot in the sample fluorescence image after the multi-frame alignment; the candidate pixel position of the light spot includes at least part of the pixel position of the light spot in the sample fluorescence image after the alignment; Based on the consistency information between the base sequence of the candidate light spot pixel position and the base sequence of the candidate light spot pixel position in the corresponding target neighborhood, and / or based on the fault tolerance information between the base sequence of the candidate light spot pixel position and the base sequence in the reference genome, determine the pixel position of the centroid of the fluorescent light spot from the candidate light spot pixel positions; The spot label is generated according to the pixel position of the centroid of the fluorescent spot.

4. The method according to claim 3, characterized in that The step of determining the base sequence corresponding to the candidate pixel position of the light spot according to the gray value of each pixel position in the sample fluorescent image after the multi-frame alignment includes: According to the gray value of each pixel point in the sample fluorescence image after multi-frame alignment, the fluorescence spot is taken as the foreground, and the image segmentation of the foreground and the background is performed on the sample fluorescence image after multi-frame alignment to obtain the candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment; and the candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment is fused to obtain the fused candidate fluorescence spot area in the sample fluorescence image after multi-frame alignment; and performing image sharpening on the multiple aligned frames of sample fluorescence images to obtain multiple sharpened frames of sample fluorescence images; Each pixel position in the fused candidate fluorescent spot area is used as a spot candidate pixel position, and a base sequence corresponding to the spot candidate pixel position is determined according to the gray value of the spot candidate pixel position in the multi-frame sharpened sample fluorescent image.

5. The method according to claim 4, characterized in that Before the foreground and background image segmentation of the sample fluorescent images after the multi-frame alignment is performed based on the grayscale value of each pixel in the sample fluorescent images after the multi-frame alignment, the fluorescent spot is used as the foreground, and the image is segmented respectively into the foreground and the background. Performing image noise reduction processing on the multiple aligned frames of sample fluorescence images respectively to obtain multiple frames of sample fluorescence images after noise reduction processing; The method of taking the fluorescent spot as the foreground and performing image segmentation of the foreground and background on the sample fluorescent images after the multi-frame alignment according to the gray value of each pixel in the sample fluorescent images after the multi-frame alignment comprises: According to the grayscale value of each pixel in the sample fluorescence image after the multi-frame noise reduction processing, the fluorescence spot is taken as the foreground, and the image segmentation of the foreground and background is performed on the sample fluorescence image after the multi-frame alignment.

6. The method according to any one of claims 2 to 5, characterized in that: According to the intensity values ​​corresponding to the respective pixel positions in the multiple-frame sample fluorescence images, the multiple-frame sample fluorescence images are respectively subjected to image segmentation processing, and the spot labels of the multiple-frame sample fluorescence images are generated according to the results of the segmentation processing of the multiple-frame sample fluorescence images: According to the gray value of each pixel position in the multi-frame sample fluorescence image, the fluorescence spot is taken as the foreground, and the multi-frame sample fluorescence image is segmented into the foreground and the background respectively, so as to obtain the candidate fluorescence spot area in the multi-frame sample fluorescence image; Parabolic interpolation processing is performed on the candidate fluorescent spot areas in the multiple frames of sample fluorescent images to obtain the fluorescent spot centroid of the candidate fluorescent spot area and generate a spot label.

7. The method according to claim 6, characterized in that The method of segmenting the foreground and background of the multiple frames of sample fluorescence images respectively by taking the fluorescent spot as the foreground according to the gray value of each pixel position in the multiple frames of sample fluorescence images comprises: The multiple frames of sample fluorescence images are subjected to image denoising respectively, and according to the grayscale value of each pixel position in the multiple frames of sample fluorescence images after denoising, the fluorescence spot is used as the foreground, and the multiple frames of sample fluorescence images are subjected to image segmentation of foreground and background respectively.

8. The method according to any one of claims 2 to 7, characterized in that: The multiple frames of sample fluorescence images are obtained in the following manner: Acquire multiple frames of original fluorescence images from multiple sequencing cycles for model training, and screen the multiple frames of sample fluorescence images from the multiple frames of original fluorescence images; or, The multiple frames of original fluorescence images derived from multiple sequencing cycles are obtained, and the multiple frames of original fluorescence images are divided into multiple sub-images respectively; and the multiple frames of sample fluorescence images are determined from the multiple sub-images in the multiple frames of original fluorescence images according to the signal-to-noise ratios corresponding to the multiple sub-images respectively.

9. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor performs the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a computer device, the computer device performs the steps of the method according to any one of claims 1 to 8.

Images