Fluorescence microscopic imaging method, device, computer device and storage medium

Abstract

This application relates to a fluorescence microscopy imaging method, apparatus, computer device, storage medium, and computer program product. The method includes: determining a target imaging strategy among preset imaging strategies based on the number of preset dyes; acquiring a reference fluorescence image corresponding to a reference laser based on the target imaging strategy; using the reference laser to excite each preset dye to emit reference fluorescence, so that the reference fluorescence image includes the fluorescence spots to be classified corresponding to each preset dye; determining the target dye corresponding to each fluorescence spot to be classified among the preset dyes based on the target imaging strategy and the reference fluorescence image; and generating a target fluorescence microscopy image based on the reference fluorescence image and the target dye corresponding to each fluorescence spot to be classified. This method can improve the imaging quality of fluorescence microscopy.

Description

Technical Field

[0001] This application relates to the field of microscopic imaging technology, and in particular to a fluorescence microscopic imaging method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of microscopic imaging technology, fluorescence microscopy has emerged, which can acquire targeted fluorescence microscopic images of samples. These images include classification and morphological information of the substances within the sample. Therefore, this technology enables the classification of substances in a sample and the acquisition of their morphological and structural information. Fluorescence microscopy includes multi-channel single-molecule ultra-high-resolution microscopy.

[0003] In traditional fluorescence microscopy, for samples stained with multiple fluorescent dyes (referred to as target samples for easy differentiation), a different laser is applied to the target sample each time, causing the corresponding fluorescent dye in the target sample to fluoresce. A fluorescence microscopic image of the target sample is then acquired using a data acquisition device. This process is repeated a predetermined number of times with a new laser, resulting in a predetermined number of fluorescence microscopic images. This predetermined number equals the number of lasers and the number of fluorescent dyes. For any given fluorescence microscopic image, based on the laser corresponding to the image and the type of fluorescent dye excited by the laser, the target fluorescent dye is identified from among the fluorescent dyes used to stain the target sample. This target fluorescent dye is then used as the fluorescent dye corresponding to all fluorescent spots in that fluorescence microscopic image. Based on this predetermined number of fluorescence microscopic images, the target fluorescence microscopic image is determined.

[0004] However, current multi-channel single-molecule ultra-high-resolution microscopy techniques often experience crosstalk between channels when performing multi-channel imaging (i.e., using multiple fluorescent dyes to stain samples), which reduces the imaging quality of the target fluorescence microscopy image. Summary of the Invention

[0005] Therefore, it is necessary to provide a fluorescence microscopy imaging method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the imaging quality of fluorescence microscopy imaging in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a fluorescence microscopy imaging method. The method includes:

[0007] Based on the number of preset dyes, the target imaging strategy is determined among the preset imaging strategies.

[0008] According to the target imaging strategy, a reference fluorescence image corresponding to the reference laser is obtained; the reference laser is used to excite each of the preset dyes to emit reference fluorescence, so that the reference fluorescence image includes the fluorescence spot to be classified corresponding to each of the preset dyes;

[0009] Based on the target imaging strategy and the reference fluorescence image, determine the target dye corresponding to each of the fluorescent spots to be classified in the preset dye;

[0010] Based on the baseline fluorescence image and the target dye corresponding to each of the fluorescence spots to be classified, a target fluorescence microscopic image is generated.

[0011] In one embodiment, the preset imaging strategy includes a first imaging strategy and a second imaging strategy; determining the target dye corresponding to each of the fluorescent spots to be classified in the preset dye according to the target imaging strategy and the reference fluorescence image includes:

[0012] When the target imaging strategy includes the first imaging strategy, for any comparison laser, a comparison fluorescence image corresponding to the comparison laser is acquired, and based on each comparison fluorescence image and the reference fluorescence image, the target dye corresponding to each fluorescent spot to be classified is determined among each of the preset dyes; the comparison laser is used to excite each of the preset dyes to emit comparison fluorescence, so that the comparison fluorescence image includes the comparison fluorescence spot corresponding to each of the preset dyes; or...

[0013] When the target imaging strategy includes the second imaging strategy, the target dye corresponding to each fluorescent spot to be classified is determined from among the preset dyes based on the correspondence of the spectral parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image.

[0014] In one embodiment, the reference fluorescence image includes a first reference image and a second reference image; the step of determining the target dye corresponding to each of the preset dyes among the compared fluorescence images and the reference fluorescence images includes:

[0015] For any of the fluorescent spots to be classified, the first light intensity and the second light intensity of the fluorescent spot to be classified are compared to obtain a first comparison result, and a first candidate dye is determined from each of the preset dyes based on the first comparison result; wherein, the first light intensity is the light intensity of the fluorescent spot to be classified on the first reference image, and the second light intensity is the light intensity of the fluorescent spot to be classified on the second reference image.

[0016] If the number of the first candidate dyes is less than or equal to the first threshold, the first candidate dye is used as the target dye corresponding to the fluorescent spot to be classified.

[0017] In one embodiment, the method further includes:

[0018] If the number of dyes in the first candidate dyes is greater than the first threshold, a first image is determined in the reference fluorescence image, and a second image is determined in each of the comparison fluorescence images; the first image and the second image are acquired using the same acquisition device.

[0019] The light intensity of the fluorescent spot to be classified in the first image is compared with the light intensity in the second image to obtain a second comparison result, and a second candidate dye is determined from the first candidate dyes based on the second comparison result;

[0020] If the number of dyes in the second candidate dye is greater than the first threshold, the first image is used as the first reference image, the second image is used as the second reference image, the second candidate dye is used as the new first candidate dye, and the process jumps to the step of determining the first image in the reference fluorescence image and determining the second image in each of the comparison fluorescence images, until the number of dyes in the second candidate dye is less than or equal to the first threshold.

[0021] The second alternative dye is used as the target dye corresponding to the fluorescent spot to be classified.

[0022] In one embodiment, determining the first candidate dye from each of the preset dyes based on the first comparison result includes:

[0023] The first dye category is determined based on the first comparison result, the reference laser, and the preset correspondence; the preset correspondence is used to characterize the relationship between the light intensity comparison result, the laser, and the dye category.

[0024] Based on the first dye category, at least one first candidate dye is determined among the preset dyes;

[0025] The step of determining a second candidate dye from the first candidate dyes based on the comparison result includes:

[0026] The second dye category is determined based on the second comparison result, the first laser corresponding to the first image, the second laser corresponding to the second image, and the preset correspondence.

[0027] Based on the second dye category, a second candidate dye is determined from the first candidate dyes.

[0028] In one embodiment, determining the first image in the reference fluorescence image includes:

[0029] For any of the fluorescent spots to be classified, if the first light intensity of the fluorescent spot to be classified is greater than or equal to the second light intensity, then the first reference image is used as the first image corresponding to the fluorescent spot to be classified; or,

[0030] If the first light intensity of the fluorescent spot to be classified is less than the second light intensity, then the second reference image is used as the first image corresponding to the fluorescent spot to be classified.

[0031] In one embodiment, determining the target dye corresponding to each fluorescent spot to be classified among the preset dyes based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image includes:

[0032] Based on the correlation of the spectroscopic parameters of the spectrometer, the correlation of the fluorescence data corresponding to each preset dye, and the reference fluorescence image, the predicted light intensity distribution data corresponding to each preset dye is determined; the reference fluorescence image includes a first reference image and a second reference image.

[0033] Based on the first reference image and the second reference image, determine the image to be classified;

[0034] Based on the actual light intensity of each fluorescent spot to be classified on the image to be classified, and the predicted light intensity distribution data corresponding to each preset dye, the target dye corresponding to each fluorescent spot to be classified is determined among the preset dyes.

[0035] In one embodiment, determining the predicted light intensity distribution data corresponding to each preset dye based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image includes:

[0036] Based on the correspondence of the spectroscopic parameters of the spectrometer and the correspondence of the fluorescence data of each preset dye, the transmittance and reflectance of the reference fluorescence corresponding to each preset dye are determined.

[0037] For any of the aforementioned reference fluorescence images, image autocorrelation processing is performed on the reference fluorescence images to obtain autocorrelation data;

[0038] Image correlation processing is performed on the first reference image and the second reference image to obtain relevant data;

[0039] Based on the transmittance, reflectance, autocorrelation data, and correlation data, the predicted light intensity distribution data corresponding to each preset dye is determined.

[0040] In one embodiment, determining the target imaging strategy among preset imaging strategies based on the number of preset dyes includes:

[0041] When the number of preset dyes is equal to the second threshold, the second imaging strategy is used as the target imaging strategy.

[0042] Secondly, this application also provides a fluorescence microscopy imaging device. The device includes:

[0043] The strategy determination module is used to determine the target imaging strategy from among the preset imaging strategies based on the number of preset dyes.

[0044] The image acquisition module is used to acquire a reference fluorescence image corresponding to the reference laser according to the target imaging strategy; the reference laser is used to excite each of the preset dyes to emit reference fluorescence, so that the reference fluorescence image includes the fluorescence spot to be classified corresponding to each of the preset dyes;

[0045] The dye determination module is used to determine the target dye corresponding to each of the fluorescent spots to be classified in the preset dyes based on the target imaging strategy and the reference fluorescence image.

[0046] The image generation module is used to generate a target fluorescence microscopic image based on the reference fluorescence image and the target dye corresponding to each of the fluorescence spots to be classified.

[0047] In one embodiment, the preset imaging strategy includes a first imaging strategy and a second imaging strategy; the dye determination module is specifically used for:

[0048] When the target imaging strategy includes the first imaging strategy, for any comparison laser, a comparison fluorescence image corresponding to the comparison laser is acquired, and based on each comparison fluorescence image and the reference fluorescence image, the target dye corresponding to each fluorescent spot to be classified is determined among each of the preset dyes; the comparison laser is used to excite each of the preset dyes to emit comparison fluorescence, so that the comparison fluorescence image includes the comparison fluorescence spot corresponding to each of the preset dyes; or...

[0049] When the target imaging strategy includes the second imaging strategy, the target dye corresponding to each fluorescent spot to be classified is determined from among the preset dyes based on the correspondence of the spectral parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image.

[0050] In one embodiment, the reference fluorescence image includes a first reference image and a second reference image; the dye determination module is specifically used for:

[0051] For any of the fluorescent spots to be classified, the first light intensity and the second light intensity of the fluorescent spot to be classified are compared to obtain a first comparison result, and a first candidate dye is determined from each of the preset dyes based on the first comparison result; wherein, the first light intensity is the light intensity of the fluorescent spot to be classified on the first reference image, and the second light intensity is the light intensity of the fluorescent spot to be classified on the second reference image.

[0052] If the number of the first candidate dyes is less than or equal to the first threshold, the first candidate dye is used as the target dye corresponding to the fluorescent spot to be classified.

[0053] In one embodiment, the dye determination module is further configured to:

[0054] If the number of dyes in the first candidate dyes is greater than the first threshold, a first image is determined in the reference fluorescence image, and a second image is determined in each of the comparison fluorescence images; the first image and the second image are acquired using the same acquisition device.

[0055] The light intensity of the fluorescent spot to be classified in the first image is compared with the light intensity in the second image to obtain a second comparison result, and a second candidate dye is determined from the first candidate dyes based on the second comparison result;

[0056] If the number of dyes in the second candidate dye is greater than the first threshold, the first image is used as the first reference image, the second image is used as the second reference image, the second candidate dye is used as the new first candidate dye, and the process jumps to the step of determining the first image in the reference fluorescence image and determining the second image in each of the comparison fluorescence images, until the number of dyes in the second candidate dye is less than or equal to the first threshold.

[0057] The second alternative dye is used as the target dye corresponding to the fluorescent spot to be classified.

[0058] In one embodiment, the dye determination module is specifically used for:

[0059] The first dye category is determined based on the first comparison result, the reference laser, and the preset correspondence; the preset correspondence is used to characterize the relationship between the light intensity comparison result, the laser, and the dye category.

[0060] Based on the first dye category, at least one first candidate dye is determined among the preset dyes;

[0061] The step of determining a second candidate dye from the first candidate dyes based on the comparison result includes:

[0062] The second dye category is determined based on the second comparison result, the first laser corresponding to the first image, the second laser corresponding to the second image, and the preset correspondence.

[0063] Based on the second dye category, a second candidate dye is determined from the first candidate dyes.

[0064] In one embodiment, the dye determination module is specifically used for:

[0065] For any of the fluorescent spots to be classified, if the first light intensity of the fluorescent spot to be classified is greater than or equal to the second light intensity, then the first reference image is used as the first image corresponding to the fluorescent spot to be classified; or,

[0066] If the first light intensity of the fluorescent spot to be classified is less than the second light intensity, then the second reference image is used as the first image corresponding to the fluorescent spot to be classified.

[0067] In one embodiment, the dye determination module is specifically used for:

[0068] Based on the correlation of the spectroscopic parameters of the spectrometer, the correlation of the fluorescence data corresponding to each preset dye, and the reference fluorescence image, the predicted light intensity distribution data corresponding to each preset dye is determined; the reference fluorescence image includes a first reference image and a second reference image.

[0069] Based on the first reference image and the second reference image, determine the image to be classified;

[0070] Based on the actual light intensity of each fluorescent spot to be classified on the image to be classified, and the predicted light intensity distribution data corresponding to each preset dye, the target dye corresponding to each fluorescent spot to be classified is determined among the preset dyes.

[0071] In one embodiment, the dye determination module is specifically used for:

[0072] Based on the correspondence of the spectroscopic parameters of the spectrometer and the correspondence of the fluorescence data of each preset dye, the transmittance and reflectance of the reference fluorescence corresponding to each preset dye are determined.

[0073] For any of the aforementioned reference fluorescence images, image autocorrelation processing is performed on the reference fluorescence images to obtain autocorrelation data;

[0074] Image correlation processing is performed on the first reference image and the second reference image to obtain relevant data;

[0075] Based on the transmittance, reflectance, autocorrelation data, and correlation data, the predicted light intensity distribution data corresponding to each preset dye is determined.

[0076] In one embodiment, the strategy determination module is specifically used for:

[0077] When the number of preset dyes is equal to the second threshold, the second imaging strategy is used as the target imaging strategy.

[0078] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps described in the first aspect.

[0079] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps described in the first aspect.

[0080] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the steps described in the first aspect.

[0081] The aforementioned fluorescence microscopy imaging method, apparatus, computer equipment, storage medium, and computer program product determine a target imaging strategy among preset imaging strategies based on the number of preset dyes; acquire a reference fluorescence image corresponding to a reference laser based on the target imaging strategy; use the reference laser to excite each preset dye to emit reference fluorescence, so that the reference fluorescence image includes the fluorescence spots to be classified corresponding to each preset dye; determine the target dye corresponding to each fluorescence spot to be classified among the preset dyes based on the target imaging strategy and the reference fluorescence image; and generate a target fluorescence microscopy image based on the reference fluorescence image and the target dye corresponding to each fluorescence spot to be classified. In this method, the target dye corresponding to each fluorescence spot to be classified among the preset dyes can be determined based on the target imaging strategy and the reference fluorescence image. The reference fluorescence image includes the fluorescence spots to be classified corresponding to each preset dye. Therefore, this method can determine the target dye of each fluorescence spot to be classified in the same reference fluorescence image, thus solving the crosstalk problem between channels and improving the imaging quality of the target fluorescence microscopy image. Attached Figure Description

[0082] Figure 1 This is a diagram illustrating the application environment of a fluorescence microscopy imaging method in one embodiment;

[0083] Figure 2 This is a diagram illustrating the application environment of the fluorescence microscopy imaging method in another embodiment;

[0084] Figure 3 This is a schematic flowchart of a fluorescence microscopy imaging method in one embodiment;

[0085] Figure 4 This is a flowchart illustrating a method for determining a target dye in one embodiment;

[0086] Figure 5 This is a flowchart illustrating a method for determining the target dye in another embodiment;

[0087] Figure 6 This is a flowchart illustrating a method for determining the first image in one embodiment;

[0088] Figure 7 This is a flowchart illustrating a method for determining the target dye in another embodiment;

[0089] Figure 8 This is a flowchart illustrating the fluorescence microscopy imaging method in another embodiment;

[0090] Figure 9 This is a schematic diagram of a reference image in one embodiment;

[0091] Figure 10 This is a schematic diagram of the first and second images in one embodiment;

[0092] Figure 11 This is a schematic diagram of the first and second images in another embodiment;

[0093] Figure 12 This is a flowchart illustrating the fluorescence microscopy imaging method in another embodiment;

[0094] Figure 13 This is a structural block diagram of a fluorescence microscopy imaging device in one embodiment;

[0095] Figure 14 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0096] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0097] The fluorescence microscopy imaging method provided in this application can be applied to a terminal, which can be a terminal with multi-channel fluorescence microscopy imaging function, such as a multi-channel fluorescence microscopy imaging system. The multi-channel fluorescence microscopy imaging system includes a multi-channel single-molecule ultra-high-resolution microscopy imaging system (i.e., a single-molecule fluorescence localization microscopy imaging system). Figure 1 or Figure 2 The diagram shown illustrates the application environment of a fluorescence microscopy imaging system provided in this embodiment of the application. Figure 1 or Figure 2 The fluorescence microscopy imaging system shown includes a laser unit 102, a microscope unit 104, an imaging unit 106, and a control unit 108. The control unit 108 is electrically connected to the laser unit 102 and controls the laser unit 102 to emit lasers of different wavelengths. The control unit 108 is also electrically connected to the imaging unit 106, so that the fluorescence images acquired by the imaging unit 106 are sent to the control unit 108. The microscope unit 104 includes a microscope and a motorized displacement platform. The microscope may be a total internal reflection fluorescent microscope (TIRFM). The control unit 108 is electrically connected to the motorized displacement platform and controls its movement. The imaging unit 106 includes an aperture (mask), a beam splitter 114, a first acquisition device 116, and a second acquisition device 118. The beam splitter 114 includes a dichroic mirror. In one embodiment, the imaging unit 106 further includes a collimating lens 120, a first imaging lens 122, and a second imaging lens 124.

[0098] Control unit 108 controls laser unit 102 to emit a reference laser, which is incident on the sample to be imaged in microscope unit 104. The sample emits reference fluorescence under the excitation of the reference laser, and the reference fluorescence is incident on spectrometer 114. Spectrometer 114 transmits the reference fluorescence to first acquisition device 116 and reflects it to second acquisition device 118. First acquisition device 116 sends the acquired first initial fluorescence image to control unit 108. Second acquisition device 118 sends the acquired second initial fluorescence image to control unit 108. Control unit 108 performs image registration processing on the first and second initial fluorescence images to obtain a first reference image and a second reference image. Based on a target imaging strategy, control unit 108 processes the image data of all fluorescent spots to be classified in the reference fluorescence image and determines the target dye corresponding to each fluorescent spot in a preset dye set based on the data processing results. The control unit 108 generates a target fluorescence microscopic image based on the image data of each fluorescent spot to be classified in the first reference image, the image data of each fluorescent spot to be classified in the second reference image, and the target dye corresponding to each fluorescent spot to be classified.

[0099] In one embodiment, such as Figure 3 As shown, a fluorescence microscopy imaging method is provided, which can be applied to... Figure 1 Taking the fluorescence microscopy imaging system in the image as an example, the following steps are included:

[0100] Step 302: Determine the target imaging strategy among the preset imaging strategies based on the number of preset dyes.

[0101] In this embodiment, the control unit 108 matches a target imaging strategy corresponding to the number of preset dyes among various preset imaging strategies, based on the number of preset dyes. The preset dyes are used to stain the sample to be imaged, and the preset imaging strategies include a first imaging strategy and a second imaging strategy. Optionally, the number of preset dyes can correspond to one target imaging strategy or multiple target imaging strategies. For example, when the number of preset dyes is 3, the preset imaging strategies corresponding to the number of preset dyes include both the first and second imaging strategies; therefore, the target imaging strategy corresponding to the number of preset dyes can be either the first or the second imaging strategy. When the number of preset dyes is 5, the first imaging strategy can be used to determine two of the preset dyes, and the second imaging strategy can be used to determine the remaining three preset dyes. The preset dyes include, but are not limited to, AF633, AF647, AF680, AF700, and AF750 dyes.

[0102] Step 304: According to the target imaging strategy, acquire the reference fluorescence image corresponding to the reference laser.

[0103] The reference laser is used to excite each preset dye to emit reference fluorescence, so that the reference fluorescence image includes the fluorescent spot to be classified corresponding to each preset dye.

[0104] In this embodiment, the control unit 108 controls the laser unit 102 to emit a reference laser according to the target imaging strategy. The reference laser is incident on the sample to be imaged in the microscope unit 104. The sample to be imaged emits reference fluorescence under the excitation of the reference laser, and the reference fluorescence is incident on the spectrometer 114. The spectrometer 114 transmits the reference fluorescence to the first acquisition device 116 and reflects it to the second acquisition device 118. The first acquisition device 116 sends the acquired first initial fluorescence image to the control unit 108. The second acquisition device 118 sends the acquired second initial fluorescence image to the control unit 108. The control unit 108 performs image registration processing on the first and second initial fluorescence images to obtain a first reference image and a second reference image. The reference laser includes, but is not limited to, a 639nm laser, a 643nm laser, a 656nm laser, and a 671nm laser. The reference fluorescence image includes both the first and second reference images. It is understood that the number of fluorescence spots to be classified and the image position data contained in the first and second reference images after image registration processing are the same.

[0105] Step 306: Based on the target imaging strategy and the reference fluorescence image, determine the target dye corresponding to each fluorescent spot to be classified in the preset dye.

[0106] In this embodiment, the control unit 108 processes the image data of all fluorescent spots to be classified in the reference fluorescence image based on a target imaging strategy, and determines the target dye corresponding to each fluorescent spot in the preset dyes based on the data processing results. The image data includes image position data and light intensity. It is understood that fluorescent spots with the same image position data correspond to the same target dye. For example, if there is a fluorescent spot 1 to be classified with image position data (a, b) in the first reference image, and a fluorescent spot 2 to be classified with image position data (a, b) in the second reference image, then the target dye corresponding to fluorescent spot 1 is the same as the target dye corresponding to fluorescent spot 2. It is understood that all fluorescence images (including the reference fluorescence image) share a single image coordinate system, and the image position data of each fluorescent spot to be classified is determined based on this image coordinate system.

[0107] Step 308: Generate a target fluorescence microscopic image based on the baseline fluorescence image and the target dye corresponding to each fluorescence spot to be classified.

[0108] In this embodiment, the control unit 108 generates a target fluorescence microscopic image based on the image data of each fluorescent spot to be classified in the first reference image, the image data of each fluorescent spot to be classified in the second reference image, and the target dye corresponding to each fluorescent spot to be classified. For example, the control unit 108 performs image overlay processing on the first and second reference images to obtain an overlaid reference fluorescence image, and generates the target fluorescence microscopic image based on the overlaid reference fluorescence image and the target dye corresponding to each fluorescent spot to be classified. The method for generating the target fluorescence microscopic image based on the overlaid reference fluorescence image and the target dye corresponding to each fluorescent spot to be classified can be referred to the description in related technologies, and will not be repeated here.

[0109] In the aforementioned fluorescence microscopy imaging method, based on the target imaging strategy and the reference fluorescence image, the target dye corresponding to each fluorescence spot to be classified in the reference fluorescence image can be determined from the preset dyes. The reference fluorescence image includes the fluorescence spots to be classified corresponding to each preset dye. Therefore, this method can determine the target dye for each fluorescence spot to be classified in the same reference fluorescence image, thus solving the crosstalk problem between channels and improving the imaging quality of the target fluorescence microscopy image.

[0110] In one embodiment, such as Figure 4As shown, the preset imaging strategy includes a first imaging strategy and a second imaging strategy; based on the target imaging strategy and the reference fluorescence image, the target dye corresponding to each fluorescent spot to be classified in the preset dye is determined, including:

[0111] Step 402: When the target imaging strategy includes the first imaging strategy, for any comparison laser, acquire the comparison fluorescence image corresponding to the comparison laser, and based on each comparison fluorescence image and the reference fluorescence image, determine the target dye corresponding to each fluorescent spot to be classified among each preset dye; the comparison laser is used to excite each preset dye to emit comparison fluorescence, so that the comparison fluorescence image includes the comparison fluorescence spot corresponding to each preset dye. Alternatively,

[0112] Step 404: When the target imaging strategy includes the second imaging strategy, the target dye corresponding to each fluorescent spot to be classified is determined among the preset dyes based on the corresponding relationship of the spectroscopic parameters of the spectroscopic device, the corresponding relationship of the fluorescence data of each preset dye, and the reference fluorescence image.

[0113] In this embodiment, the control unit 108 first determines the imaging strategies included in the target imaging strategy. If the target imaging strategy includes a first imaging strategy, the control unit 108 executes step 402; if the target imaging strategy includes a second imaging strategy, the control unit 108 executes step 404. If the target imaging strategy includes both the first and second imaging strategies, the control unit 108 may execute either step 402 or step 404.

[0114] Regarding step 402, if the target imaging strategy includes the first imaging strategy, for any comparison laser, the control unit 108 acquires the comparison fluorescence image corresponding to the comparison laser. The acquisition process of the comparison fluorescence image is similar to the acquisition process of the reference fluorescence image, and will not be described again here. Optionally, the comparison fluorescence image corresponding to each comparison laser may include a first comparison fluorescence image and a second comparison fluorescence image, or it may be either the first comparison fluorescence image or the second comparison fluorescence image. The first comparison fluorescence image is the image after image registration processing of the first initial comparison image acquired by the first acquisition device 116, and the second comparison fluorescence image is the image after image registration processing of the second initial comparison image acquired by the second acquisition device 118.

[0115] It is understood that images acquired by the same acquisition device use the same image registration parameters when performing image registration processing. Therefore, after executing step 304, the control unit 108 can determine the image registration parameters corresponding to the image acquired by the first acquisition device 116 and the image registration parameters corresponding to the image acquired by the second acquisition device 118, and then perform registration processing on the images acquired by each acquisition device based on the image registration parameters.

[0116] Regarding step 404, if the target imaging strategy includes a second imaging strategy, the control unit 108 calculates the wavelength of the reference fluorescence emitted by each preset dye under the excitation of the reference laser, based on the wavelength of the reference laser and the corresponding fluorescence data of each preset dye. The fluorescence data correspondence characterizes the relationship between the wavelength of the reference laser, the type of preset dye, and the wavelength of the reference fluorescence. For the reference fluorescence emitted by any preset dye, the control unit 108 calculates the reflectance of the reference fluorescence reflected by the spectrometer 114 and the transmittance of the reference fluorescence transmitted through the spectrometer 114, based on the wavelength of the reference fluorescence and the corresponding relationship of the spectroscopic parameters of the spectrometer 114.

[0117] The correspondence of the beam splitting parameters includes the correspondence between incident wavelength and reflectivity, and the correspondence between incident wavelength and transmittance. The correspondence between incident wavelength and reflectivity is used to characterize the correspondence between the wavelength of the incident light incident on the beam splitting device 114 and the reflectivity reflected by the beam splitting device 114. The correspondence between incident wavelength and transmittance is used to characterize the correspondence between the wavelength of the incident light incident on the beam splitting device 114 and the transmittance transmitted through the beam splitting device 114.

[0118] The control unit 108 calculates the predicted light intensity distribution data corresponding to each preset dye based on the reflectance and transmittance of the reference fluorescence emitted by each preset dye and the reference fluorescence image. Based on the predicted light intensity distribution data corresponding to each preset dye and the reference fluorescence image, the control unit 108 determines the target dye corresponding to each fluorescent spot to be classified in the reference fluorescence image.

[0119] In this embodiment, the method for determining the target dye is determined by identifying the specific imaging strategies included in the target imaging strategy. Therefore, this method can employ multiple methods for determining the target dye to identify the target dye corresponding to the fluorescent spot to be classified.

[0120] In one embodiment, such as Figure 5 As shown, the reference fluorescence image includes a first reference image and a second reference image; based on each comparison fluorescence image and the reference fluorescence image, the target dye corresponding to each fluorescent spot to be classified is determined among each preset dye, including:

[0121] Step 502: For any fluorescent spot to be classified, compare the first light intensity and the second light intensity of the fluorescent spot to be classified to obtain a first comparison result, and determine the first candidate dye from each preset dye according to the first comparison result.

[0122] Wherein, the first light intensity is the light intensity of the fluorescent spot to be classified on the first reference image, and the second light intensity is the light intensity of the fluorescent spot to be classified on the second reference image.

[0123] In this embodiment, for any fluorescent spot to be classified, the control unit 108 compares the first light intensity and the second light intensity of the fluorescent spot to be classified to obtain a first comparison result. For any fluorescent spot to be classified, the control unit 108 matches a first candidate dye from each preset dye according to the first comparison result. It can be understood that, based on the correspondence between the fluorescence data corresponding to the preset dyes and the correspondence between the spectroscopic parameters of the spectrometer, the reflectance and transmittance of the reference fluorescence emitted by each preset dye can be calculated, and then the predicted comparison result corresponding to the preset dye can be calculated. Of course, the predicted comparison result corresponding to each preset dye can also be preset manually. If the predicted comparison result corresponding to the preset dye is the same as the first comparison result corresponding to the fluorescent spot to be classified, the control unit 108 uses the preset dye as the first candidate dye for the fluorescent spot to be classified.

[0124] Step 504: If the number of first candidate dyes is less than or equal to the first threshold, the first candidate dye is used as the target dye corresponding to the fluorescent spot to be classified.

[0125] In this embodiment of the application, when the number of first candidate dyes is less than or equal to a first threshold, the control unit 108 selects the first candidate dye as the target dye corresponding to the fluorescent spot to be classified. For example, the first threshold is 1.

[0126] In this embodiment, by comparing the first light intensity and the second light intensity of the fluorescent spot to be classified, the first candidate dye corresponding to the fluorescent spot to be classified is determined. If the number of dyes in the first candidate dye is less than or equal to a first threshold, the first candidate dye is used as the target dye corresponding to the fluorescent spot to be classified. Therefore, this method can determine the target dye of the fluorescent spot to be classified based on a reference fluorescence image including various preset dyes corresponding to the fluorescent spots to be classified, thus solving the crosstalk problem between multiple channels of fluorescent spots to be classified.

[0127] In one embodiment, such as Figure 5 As shown, the method also includes:

[0128] Step 506: If the number of dyes in the first candidate dyes is greater than the first threshold, determine the first image in the reference fluorescence image and determine the second image in each comparison fluorescence image.

[0129] The first image and the second image are acquired using the same acquisition device.

[0130] In this embodiment, when the number of dyes of the first candidate dyes corresponding to the fluorescent spot to be classified is greater than a first threshold, the control unit 108 selects one of the reference fluorescence images as the first image. The control unit 108 then matches a second image among the comparison fluorescence images based on the acquisition device corresponding to the first image and the acquisition device corresponding to each comparison fluorescence image.

[0131] Step 508: Compare the light intensity of the fluorescent spot to be classified on the first image with the light intensity on the second image to obtain a second comparison result, and determine the second candidate dye from the first candidate dye based on the second comparison result.

[0132] In this embodiment, the control unit 108 compares the light intensity of the fluorescent spot to be classified in the first image with the light intensity in the second image to obtain a second comparison result. The control unit 108 then determines a second candidate dye from the first candidate dyes based on the second comparison result. It is understood that the method for determining the second candidate dye based on the second comparison result is similar to the method for determining the first candidate dye based on the first comparison result, and will not be described in detail here.

[0133] Step 510: If the number of dyes of the second candidate dye is greater than the first threshold, the first image is used as the first reference image, the second image is used as the second reference image, the second candidate dye is used as the new first candidate dye, and the process jumps to the step of determining the first image in the reference fluorescence image and determining the second image in each comparison fluorescence image, until the number of dyes of the second candidate dye is less than or equal to the first threshold.

[0134] In this embodiment of the application, when the number of dyes of the second candidate dye is greater than the first threshold, the control unit 108 uses the first image as the new first reference image, the second image as the new second reference image, and the second candidate dye as the new first candidate dye, and jumps to step 506 until the number of dyes of the second candidate dye is less than or equal to the first threshold.

[0135] Step 512: Select the second alternative dye as the target dye corresponding to the fluorescent spot to be classified.

[0136] In this embodiment of the application, for any fluorescent spot to be classified, if the number of second candidate dyes corresponding to the fluorescent spot to be classified is less than or equal to the first threshold, the control unit 108 uses the second candidate dye as the target dye corresponding to the fluorescent spot to be classified.

[0137] In this embodiment, by comparing the light intensity of the fluorescent spot to be classified in the first image with the light intensity in the second image, a second candidate dye corresponding to the fluorescent spot to be classified is determined. If the number of dyes in the second candidate dye is less than or equal to a first threshold, the second candidate dye is selected as the target dye corresponding to the fluorescent spot to be classified. Therefore, this method can determine the target dye of the fluorescent spot to be classified based on a reference fluorescence image containing various preset dyes and a comparison fluorescence image, thus solving the crosstalk problem between multiple channels of fluorescent spots to be classified.

[0138] In one embodiment, determining a first candidate dye from a set of preset dyes based on a first comparison result includes:

[0139] Based on the first comparison result, the reference laser, and the preset correspondence, a first dye category is determined; based on the first dye category, at least one first candidate dye is determined among the preset dyes.

[0140] Among them, the preset correspondence is used to characterize the relationship between the light intensity comparison results, the laser, and the dye category.

[0141] In this embodiment of the application, the control unit 108 matches the dye category (i.e., the first dye category) of the dye corresponding to the fluorescent spot to be classified according to the first comparison result, the wavelength of the reference laser and the preset correspondence, and determines the first candidate dye corresponding to the first dye category among the preset dyes, that is, the first candidate dye corresponding to the fluorescent spot to be classified.

[0142] In this embodiment, the first candidate dye corresponding to the fluorescent spot to be classified is determined based on the first comparison result, the reference laser, and the preset correspondence. Therefore, this method can initially screen out the first candidate dye corresponding to the fluorescent spot to be classified from various preset dyes, thus providing a prerequisite for the subsequent method of determining the target dye of the fluorescent spot to be classified based on the first candidate dye.

[0143] In one embodiment, determining a second candidate dye from a first candidate dye based on a comparison result includes:

[0144] Based on the second comparison result, the first laser corresponding to the first image, the second laser corresponding to the second image, and the preset correspondence, the second dye category is determined; based on the second dye category, the second candidate dye is determined from the first candidate dyes.

[0145] In this embodiment of the application, the control unit 108 matches the dye category (i.e., the second dye category) of the dye corresponding to the fluorescent spot to be classified according to the second comparison result, the first laser corresponding to the first image, the second laser corresponding to the second image, and the preset correspondence. Based on the second dye category, the control unit 108 determines the second candidate dye corresponding to the second dye category among each first candidate dye, which is the second candidate dye corresponding to the fluorescent spot to be classified.

[0146] In this embodiment, a second candidate dye corresponding to the fluorescent spot to be classified is determined based on the second comparison result, the first laser corresponding to the first image, the second laser corresponding to the second image, and a preset correspondence. Therefore, this method can further screen out the second candidate dye corresponding to the fluorescent spot to be classified from each first candidate dye, thereby providing a prerequisite for the subsequent method of determining the target dye of the fluorescent spot to be classified based on the second candidate dye.

[0147] In one embodiment, such as Figure 6 As shown, in the reference fluorescence image, a first image is determined, including:

[0148] Step 602: For any fluorescent spot to be classified, if the first light intensity of the fluorescent spot to be classified is greater than or equal to the second light intensity, then the first reference image is used as the first image corresponding to the fluorescent spot to be classified. Alternatively,

[0149] Step 604: If the first light intensity of the fluorescent spot to be classified is less than the second light intensity, then the second reference image is used as the first image corresponding to the fluorescent spot to be classified.

[0150] In this embodiment, for any fluorescent spot to be classified, if the first light intensity of the fluorescent spot to be classified is greater than or equal to the second light intensity, the control unit 108 uses the first reference image as the first image corresponding to the fluorescent spot to be classified. For any fluorescent spot to be classified, if the first light intensity of the fluorescent spot to be classified is less than the second light intensity, the control unit 108 uses the second reference image as the first image corresponding to the fluorescent spot to be classified.

[0151] In this embodiment, the first image corresponding to the fluorescent spot to be classified is determined by the first light intensity and the second light intensity of the fluorescent spot to be classified, thereby providing a prerequisite for the subsequent method of determining the target dye of the fluorescent spot to be classified based on the first image.

[0152] In one embodiment, such as Figure 7 As shown, based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image, the target dye corresponding to each fluorescence spot to be classified is determined among the preset dyes, including:

[0153] Step 702: Determine the predicted light intensity distribution data corresponding to each preset dye based on the corresponding relationship of the spectroscopic parameters of the spectroscopic device, the corresponding relationship of the fluorescence data of each preset dye, and the reference fluorescence image.

[0154] The reference fluorescence image includes a first reference image and a second reference image.

[0155] In this embodiment, the control unit 108 acquires the wavelength of the reference fluorescence. This application does not limit the method of acquiring the wavelength of the reference fluorescence; please refer to related technologies, which will not be elaborated here. For example, the wavelength of the reference fluorescence can be acquired using a wavelength measurement device, or it can be directly acquired based on wavelengths obtained from historical experiments. For any preset dye emitting reference fluorescence, the control unit 108 calculates the reflectance of the reference fluorescence reflected by the spectrometer 114 and the transmittance of the reference fluorescence transmitted through the spectrometer 114, based on the wavelength of the reference fluorescence and the corresponding relationship of the spectroscopic parameters of the spectrometer 114. The control unit 108 calculates the predicted light intensity distribution data corresponding to each preset dye based on the reflectance and transmittance of the reference fluorescence emitted by each preset dye, the corresponding relationship of the fluorescence data for each preset dye, and the reference fluorescence image. The predicted light intensity distribution data includes the predicted light intensity corresponding to each image position data.

[0156] Step 704: Determine the image to be classified based on the first reference image and the second reference image.

[0157] In this embodiment, the control unit 108 performs image overlay processing on the first reference image and the second reference image to obtain the image to be classified. It can be understood that the light intensity distribution in the image to be classified is equal to the sum of the light intensity distributions of the first reference image and the second reference image.

[0158] Step 706: Based on the actual light intensity of each fluorescent spot to be classified on the image to be classified and the predicted light intensity distribution data corresponding to each preset dye, determine the target dye corresponding to each fluorescent spot to be classified among the preset dyes.

[0159] In this embodiment, for any fluorescent spot to be classified, the control unit 108 matches the target dye corresponding to the fluorescent spot to be classified among the preset dyes based on the actual light intensity of the fluorescent spot on the image to be classified, the image position data of the fluorescent spot on the image to be classified, and the predicted light intensity distribution data corresponding to each preset dye. Specifically, the control unit 108 determines the predicted light intensity (referred to as reference light intensity) corresponding to each preset dye on the target position data based on the image position data of the fluorescent spot to be classified (for ease of distinction, referred to as target position data). The control unit 108 takes the reference light intensity that is closest to the actual light intensity of the fluorescent spot to be classified as the target light intensity, and takes the preset dye corresponding to the target light intensity as the target dye corresponding to the fluorescent spot to be classified.

[0160] In this embodiment, the target dye corresponding to each fluorescent spot to be classified is determined by the actual light intensity of each fluorescent spot on the image to be classified and the predicted light intensity distribution data corresponding to each preset dye. Therefore, this method can determine the target dye of the fluorescent spot to be classified based on the image to be classified, which includes fluorescent spots corresponding to various preset dyes, thus solving the crosstalk problem of fluorescent spots to be classified between multiple channels.

[0161] In one embodiment, the predicted light intensity distribution data corresponding to each preset dye is determined based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image, including:

[0162] Based on the correlation of the spectroscopic parameters of the spectrometer and the correlation of the fluorescence data of each preset dye, the transmittance and reflectance of the reference fluorescence corresponding to each preset dye are determined. For any reference fluorescence image, image autocorrelation processing is performed on the reference fluorescence image to obtain autocorrelation data. Image correlation processing is performed on the first reference image and the second reference image to obtain correlation data. Based on each transmittance, each reflectance, their respective correlation data, and the correlation data, the predicted light intensity distribution data corresponding to each preset dye is determined.

[0163] In this embodiment, the control unit 108 acquires the wavelength of the reference fluorescence emitted by each preset dye under the excitation of a reference laser. For the reference fluorescence emitted by any preset dye, the control unit 108 calculates the reflectance of the reference fluorescence reflected by the spectrometer 114 and the transmittance of the reference fluorescence transmitted through the spectrometer 114 based on the wavelength of the reference fluorescence, the corresponding fluorescence data of each preset dye, and the corresponding spectroscopic parameters of the spectrometer 114. The corresponding fluorescence data of the preset dye includes the emission spectrum curve of the preset dye, and the emission spectrum curve of the preset dye includes the spectrum curve of the reference fluorescence. The emission spectrum curve is used to characterize the wavelength and relative intensity of the fluorescence emitted by the preset dye. Specifically, as shown in formulas (1) and (2) below.

[0164] Formula (1)

[0165] Formula (2)

[0166] in, In Indicates transmittance. In This indicates the dye category of the preset dye (i.e., the preset dye). ),Right now Indicates the preset dye The transmittance of the corresponding reference fluorescence, Indicates the preset dye The corresponding emission spectrum curves (including the spectrum curves corresponding to the reference fluorescence). This indicates the relationship between the incident wavelength (including reference fluorescence) and the transmittance of the spectrophotometer 114. Indicates the preset dye The reflectance of the corresponding reference fluorescence, This indicates the relationship between the incident wavelength (including reference fluorescence) and the transmittance of the spectrophotometer 114. It is a positive integer.

[0167] For any first reference image, the control unit 108 performs image autocorrelation processing on the first reference image to obtain autocorrelation data of the first reference image. For any second reference image, the control unit 108 performs image autocorrelation processing on the second reference image to obtain autocorrelation data of the second reference image. The control unit 108 performs image correlation processing on the first reference image and the second reference image to obtain correlation data between the reference images.

[0168] The control unit 108 calculates the predicted light intensity distribution data corresponding to each preset dye based on the transmittance and reflectance of each reference fluorescence, the autocorrelation data of the first reference image, the autocorrelation data of the second reference image, and the correlation data between reference images. Specifically, the control unit 108 calculates the predicted light intensity distribution data corresponding to each preset dye based on the transmittance and reflectance of each reference fluorescence, the autocorrelation data of the first reference image, the autocorrelation data of the second reference image, the correlation data between reference images, and the super-resolution optical fluctuation imaging (SOFI) algorithm. The super-resolution optical fluctuation imaging algorithm is specifically shown in the following formula (3).

[0169] Formula (3)

[0170] in, This represents the autocorrelation data of the second reference image. This represents the correlation data between reference images. This represents the autocorrelation data of the first reference image. In Indicates reflectivity, In This indicates the dye category of the preset dye (i.e., the preset dye). ), In express Power of 1 In Indicates transmittance. In This indicates the dye category of the preset dye (i.e., the preset dye). ), In express Power of 1 Indicates the preset dye Second-order autocorrelation data, Indicates the preset dye Light intensity distribution data, and All are positive integers.

[0171] In this embodiment, the predicted light intensity distribution data corresponding to each preset dye is determined by the corresponding relationship of the spectroscopic parameters of the spectroscopic device, the corresponding relationship of the fluorescence data of each preset dye, and the reference fluorescence image. This provides a prerequisite for the subsequent method of determining the target dye of the fluorescent spot to be classified based on the predicted light intensity distribution data and the actual light intensity.

[0172] In one embodiment, determining a target imaging strategy among preset imaging strategies based on the number of preset dyes includes:

[0173] When the number of preset dyes is equal to the second threshold, the second imaging strategy is used as the target imaging strategy.

[0174] In this embodiment, when the number of preset dyes equals a second threshold, the control unit 108 uses the second imaging strategy as the target imaging strategy. The second threshold is 3.

[0175] This embodiment provides a method for using a second imaging strategy as a target imaging strategy, thereby providing a prerequisite for a subsequent method for generating target fluorescence microscopic images based on the target imaging strategy.

[0176] To enable those skilled in the art to better understand this application, such as Figure 8 As shown, the present application will be described below through specific embodiments. In this embodiment, the sample is stained with four preset dyes (including AF633 dye, AF647 dye, AF680 dye and AF700 dye), the wavelength of the reference laser is 671nm, and the wavelength of the comparison laser is 643nm, as an example, and combined with, as shown below Figure 1 The fluorescence microscopy imaging system shown is described below.

[0177] exist Figure 8 In the first step, the dual cameras include a first acquisition device 116 and a second acquisition device 118. The control unit 108 controls the laser unit 102 to emit a reference laser with a wavelength of 671nm, and acquires a first initial fluorescence image acquired by the first acquisition device 116 and a second initial fluorescence image acquired by the second acquisition device 118. The control unit 108 performs image registration processing on the first and second initial fluorescence images to obtain a first reference image and a second reference image. It can be understood that since the sample is stained with four preset dyes, both the first and second initial fluorescence images record the fluorescence spots to be classified corresponding to the four preset dyes (i.e.,...). Figure 8 The four types of fluorescence imaging (i.e., the first and second reference images after image registration processing both contain fluorescent spots to be classified corresponding to the four preset dyes).

[0178] exist Figure 8 In the second step, the control unit 108 controls the laser unit 102 to emit a comparison laser with a wavelength of 643nm, and acquires a first comparison image corresponding to the first acquisition device 116 and a second comparison image corresponding to the second acquisition device 118. The comparison image corresponding to the 643nm comparison laser includes both the first and second comparison images.

[0179] exist Figure 8In the third step (equivalent to step 502), for any fluorescent spot to be classified, the control unit 108 compares the first light intensity and the second light intensity of the fluorescent spot to be classified to obtain a first comparison result. For any fluorescent spot to be classified, the control unit 108 matches a first candidate dye from each preset dye according to the first comparison result. Specifically, as shown... Figure 9 As shown, the boxes represent the baseline image, and the circles represent the fluorescent spots to be classified. The darker and higher the color within the circle, the stronger the light intensity; that is, the light intensity of a white circle < the light intensity of a gray circle < the light intensity of a black circle. This is understandable. Figure 9 This is merely an example and does not impose any limitations on the actual distribution location or actual light intensity of the fluorescent spots to be classified.

[0180] For the fluorescent spot a to be classified, the control unit 108 compares the first light intensity with the second light intensity of the fluorescent spot a and determines that the first light intensity of the fluorescent spot a is greater than the second light intensity. Since the first light intensity of the reference fluorescence emitted by AF680 dye and AF700 dye is greater than the second light intensity under excitation by the 671nm reference laser, the control unit 108 selects AF680 dye and AF700 dye as the first candidate dyes for the fluorescent spot a to be classified. Similarly, the control unit 108 selects AF680 dye and AF700 dye as the first candidate dyes for the fluorescent spot c to be classified. It can be understood that... Figure 9 The first reference image corresponding to AF680 and AF700 dyes only contains fluorescent spots a and c to be classified, merely to indicate that the first candidate dyes for fluorescent spots a and c include AF680 and AF700 dyes. In reality, the first reference image also includes fluorescent spots b and d to be classified. Similarly, Figure 9 , Figure 10 as well as Figure 11 The image corresponding to the preset dye also contains all the fluorescent spots to be classified corresponding to the preset dye, but they are not shown in the figure.

[0181] For the fluorescent spot b to be classified, the control unit 108 compares the first light intensity with the second light intensity of the fluorescent spot b and determines that the first light intensity of the fluorescent spot b is less than the second light intensity. Since the first light intensity of the reference fluorescence emitted by AF633 dye and AF647 dye is less than the second light intensity under the excitation of the 671nm reference laser, the control unit 108 selects AF633 dye and AF647 dye as the first candidate dyes for the fluorescent spot b to be classified. Similarly, the control unit 108 selects AF633 dye and AF647 dye as the first candidate dyes for the fluorescent spot d to be classified.

[0182] exist Figure 8In the fourth step (equivalent to steps 506 to 512), the control unit 108 uses the first reference image as the first image of the fluorescent spots a and c to be classified, and the second reference image as the first image of the fluorescent spots b and d to be classified. Since the first comparison image corresponds to the same acquisition device as the first image, the control unit 108 uses the first comparison image as the second image of the fluorescent spots a and c to be classified. Similarly, the control unit 108 uses the second comparison image as the second image of the fluorescent spots b and d to be classified. For any fluorescent spot to be classified, the control unit 108 compares the light intensity of the fluorescent spot in the first image with the light intensity in the second image to obtain a second comparison result. Based on the second comparison result, the control unit 108 determines a second candidate dye from the first candidate dyes.

[0183] Specifically, such as Figure 10 As shown, for the fluorescent spot a to be classified, the control unit 108 compares the light intensity of the fluorescent spot a in the first image with the light intensity in the second image, and determines that the light intensity of the fluorescent spot a in the first image is less than the light intensity in the second image. Since the contrast fluorescence emitted by the AF680 dye in the first image is less than the light intensity in the second image under the excitation of the 643nm reference laser, the control unit 108 uses the AF680 dye as the second candidate dye for the fluorescent spot a to be classified. Since the number of the second candidate dyes for the fluorescent spot a to be classified is less than or equal to the first threshold 1, the control unit 108 uses the AF680 dye as the target dye for the fluorescent spot a to be classified. Since the contrast fluorescence emitted by the AF700 dye in the first image is stronger than the light intensity in the second image under the excitation of the 643nm reference laser, similarly, the control unit 108 uses the AF700 dye as the target candidate dye for the fluorescent spot c to be classified.

[0184] like Figure 11 As shown, under the excitation of the reference laser at 643nm, the intensity of the contrast fluorescence emitted by the AF633 dye in the first image is less than that in the second image, and the intensity of the contrast fluorescence emitted by the AF647 dye in the first image is equal to that in the second image. Similarly, the control unit 108 uses the AF633 dye as the target candidate dye for the fluorescent spot b to be classified and the AF647 dye as the target candidate dye for the fluorescent spot c to be classified.

[0185] exist Figure 8 In the fifth step (equivalent to step 308), the control unit 108 performs image overlay processing on the first reference image and the second reference image to obtain the overlaid reference fluorescence image, and generates the target fluorescence microscopic image based on the overlaid reference fluorescence image and the target dye corresponding to each fluorescent spot to be classified.

[0186] It is understandable that in practical applications, although ideally one laser excites one preset dye, in reality, it is possible for one laser to excite multiple preset dyes. In this case, existing technologies struggle to distinguish which preset dye corresponds to the target fluorescent spot to be classified under that laser excitation (i.e., there is an inter-channel crosstalk problem). This method, however, can determine the target dye for each fluorescent spot to be classified based on a reference fluorescence image containing four preset dyes and a comparative fluorescence image containing the same four preset dyes. Therefore, this method can solve the inter-channel crosstalk problem in multi-channel imaging, thereby improving the effect of multi-channel imaging.

[0187] To enable those skilled in the art to better understand this application, such as Figure 12 As shown, the present application will be described below through specific embodiments. In this embodiment, the sample is stained with three preset dyes (including AF647 dye, AF680 dye, and AF700 dye), the wavelength of the reference laser is 656nm, and combined with... Figure 2 The fluorescence microscopy imaging system shown is described below.

[0188] exist Figure 8 In the first step, the dual cameras include a first acquisition device 116 and a second acquisition device 118. The control unit 108 controls the laser unit 102 to emit a reference laser with a wavelength of 656nm, and acquires a first initial fluorescence image acquired by the first acquisition device 116 and a second initial fluorescence image acquired by the second acquisition device 118. The control unit 108 performs image registration processing on the first and second initial fluorescence images to obtain a first reference image and a second reference image. It can be understood that since the sample is stained with three preset dyes, both the first and second initial fluorescence images record the fluorescence spots to be classified corresponding to the three preset dyes (i.e.,...). Figure 11 The three types of fluorescence imaging (i.e., the first and second reference images after image registration processing both contain fluorescence spots to be classified corresponding to the three preset dyes).

[0189] exist Figure 8In the second step (equivalent to step 404), for any preset dye emitting reference fluorescence, the control unit 108 calculates the reflectance of the reference fluorescence reflected by the spectrometer 114 and the transmittance of the reference fluorescence transmitted through the spectrometer 114, based on the wavelength of the reference fluorescence and the corresponding relationship between the spectrometer parameters of the spectrometer 114. For any first reference image, the control unit 108 performs image autocorrelation processing on the first reference image to obtain autocorrelation data of the first reference image. For any second reference image, the control unit 108 performs image autocorrelation processing on the second reference image to obtain autocorrelation data of the second reference image. The control unit 108 performs image correlation processing on the first reference image and the second reference image to obtain correlation data between the reference images.

[0190] The control unit 108 calculates the predicted light intensity distribution data corresponding to each preset dye based on the transmittance and reflectance of each reference fluorescence, the autocorrelation data of the first reference image, the autocorrelation data of the second reference image, the correlation data between the reference images, and the super-resolution optical fluctuation imaging (SOFI) algorithm. The control unit 108 performs image overlay processing on the first and second reference images to obtain the image to be classified. Based on the image position data of the fluorescence spot to be classified on the image to be classified (referred to as target position data for ease of distinction), the control unit 108 determines the predicted light intensity (referred to as reference light intensity for ease of distinction) corresponding to each preset dye at the target position data. The control unit 108 selects the reference light intensity closest to the actual light intensity of the fluorescence spot to be classified as the target light intensity, and designates the preset dye corresponding to the target light intensity as the target dye corresponding to the fluorescence spot to be classified.

[0191] In this embodiment, the target dye of each fluorescent spot to be classified can be determined based on the reference fluorescence image of the fluorescent spots to be classified corresponding to three preset dyes and the super-resolution microscopy algorithm based on optical fluctuation signals. Therefore, this method can solve the problem of crosstalk between channels in multi-channel imaging, thereby improving the effect of multi-channel imaging.

[0192] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0193] Based on the same inventive concept, this application also provides a fluorescence microscopy imaging apparatus for implementing the fluorescence microscopy imaging method described above. The solution provided by this apparatus is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the fluorescence microscopy imaging apparatus provided below can be found in the limitations of the fluorescence microscopy imaging method described above, and will not be repeated here.

[0194] In one embodiment, such as Figure 13 As shown, a fluorescence microscopy imaging device is provided, comprising:

[0195] The strategy determination module 1302 is used to determine the target imaging strategy among the preset imaging strategies based on the number of preset dyes.

[0196] The image acquisition module 1304 is used to acquire the reference fluorescence image corresponding to the reference laser according to the target imaging strategy; the reference laser is used to excite each preset dye to emit reference fluorescence, so that the reference fluorescence image includes the fluorescence spot to be classified corresponding to each preset dye.

[0197] The dye determination module 1306 is used to determine the target dye corresponding to each fluorescent spot to be classified in the preset dyes based on the target imaging strategy and the reference fluorescence image.

[0198] The image generation module is used to generate target fluorescence microscopic images based on the baseline fluorescence image and the target dye corresponding to each fluorescence spot to be classified.

[0199] In the aforementioned fluorescence microscopy imaging apparatus, based on the target imaging strategy and the reference fluorescence image, the target dye corresponding to each fluorescence spot to be classified in the reference fluorescence image can be determined from among preset dyes. The reference fluorescence image includes the fluorescence spots to be classified corresponding to each preset dye. Therefore, this method can determine the target dye for each fluorescence spot to be classified in the same reference fluorescence image, thus solving the crosstalk problem between channels and improving the imaging quality of the target fluorescence microscopy image.

[0200] In one embodiment, the preset imaging strategy includes a first imaging strategy and a second imaging strategy; the dye determination module 1306 is specifically used for:

[0201] When the target imaging strategy includes the first imaging strategy, for any comparison laser, the comparison fluorescence image corresponding to the comparison laser is acquired, and based on each comparison fluorescence image and the reference fluorescence image, the target dye corresponding to each fluorescent spot to be classified is determined among each preset dye; the comparison laser is used to excite each preset dye to emit comparison fluorescence, so that the comparison fluorescence image includes the comparison fluorescence spot corresponding to each preset dye; or,

[0202] When the target imaging strategy includes a second imaging strategy, the target dye corresponding to each fluorescent spot to be classified is determined among the preset dyes based on the correspondence of the spectroscopic parameters of the spectroscopic device, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image.

[0203] In one embodiment, the reference fluorescence image includes a first reference image and a second reference image; the dye determination module 1306 is specifically used for:

[0204] For any fluorescent spot to be classified, the first light intensity and the second light intensity of the fluorescent spot to be classified are compared to obtain a first comparison result, and a first candidate dye is determined from each preset dye according to the first comparison result; wherein, the first light intensity is the light intensity of the fluorescent spot to be classified on the first reference image, and the second light intensity is the light intensity of the fluorescent spot to be classified on the second reference image.

[0205] If the number of candidate dyes is less than or equal to the first threshold, the first candidate dye is used as the target dye corresponding to the fluorescent spot to be classified.

[0206] In one embodiment, the dye determination module 1306 is further configured to:

[0207] If the number of candidate dyes is greater than a first threshold, a first image is determined in the reference fluorescence image, and a second image is determined in each comparison fluorescence image; the first image and the second image are acquired using the same acquisition device.

[0208] The light intensity of the fluorescent spot to be classified in the first image is compared with the light intensity in the second image to obtain a second comparison result, and a second candidate dye is determined from the first candidate dyes based on the second comparison result.

[0209] If the number of dyes of the second candidate dye is greater than the first threshold, the first image is used as the first reference image, the second image is used as the second reference image, the second candidate dye is used as the new first candidate dye, and the process jumps to the step of determining the first image in the reference fluorescence image and determining the second image in each comparison fluorescence image until the number of dyes of the second candidate dye is less than or equal to the first threshold.

[0210] The second alternative dye is used as the target dye for the fluorescent spot to be classified.

[0211] In one embodiment, the dye determination module 1306 is specifically used for:

[0212] The first dye category is determined based on the first comparison result, the reference laser, and the preset correspondence; the preset correspondence is used to characterize the relationship between the light intensity comparison result, the laser, and the dye category.

[0213] Based on the first dye category, at least one first candidate dye is determined among the preset dyes;

[0214] Based on the comparison results, a second candidate dye is determined from the first candidate dyes, including:

[0215] The second dye category is determined based on the second comparison result, the first laser corresponding to the first image, the second laser corresponding to the second image, and the preset correspondence.

[0216] Based on the second dye category, a second alternative dye is determined from the first alternative dyes.

[0217] In one embodiment, the dye determination module 1306 is specifically used for:

[0218] For any fluorescent spot to be classified, if the first light intensity of the fluorescent spot to be classified is greater than or equal to the second light intensity, then the first reference image is used as the first image corresponding to the fluorescent spot to be classified; or,

[0219] If the first light intensity of the fluorescent spot to be classified is less than the second light intensity, then the second reference image is used as the first image corresponding to the fluorescent spot to be classified.

[0220] In one embodiment, the dye determination module 1306 is specifically used for:

[0221] Based on the correlation of the spectroscopic parameters of the spectrometer, the correlation of the fluorescence data of each preset dye, and the reference fluorescence image, the predicted light intensity distribution data of each preset dye is determined; the reference fluorescence image includes a first reference image and a second reference image.

[0222] The image to be classified is determined based on the first reference image and the second reference image;

[0223] Based on the actual light intensity of each fluorescent spot to be classified on the image to be classified, and the predicted light intensity distribution data corresponding to each preset dye, the target dye corresponding to each fluorescent spot to be classified is determined among the preset dyes.

[0224] In one embodiment, the dye determination module 1306 is specifically used for:

[0225] Based on the correlation of the spectroscopic parameters of the spectrometer and the correlation of the fluorescence data of each preset dye, the transmittance and reflectance of the reference fluorescence corresponding to each preset dye are determined.

[0226] For any reference fluorescence image, perform image autocorrelation processing on the reference fluorescence image to obtain autocorrelation data;

[0227] Image correlation processing is performed on the first and second reference images to obtain relevant data;

[0228] Based on each transmittance, each reflectance, their respective related data, and other relevant data, the predicted light intensity distribution data corresponding to each preset dye is determined.

[0229] In one embodiment, the strategy determination module 1302 is specifically used for:

[0230] When the number of preset dyes is equal to the second threshold, the second imaging strategy is used as the target imaging strategy.

[0231] Each module in the aforementioned fluorescence microscopy imaging device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0232] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a fluorescence microscopy imaging method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0233] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0234] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0235] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0236] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0237] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0238] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0239] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0240] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method of fluorescence microscopy imaging, characterized in that, The method includes: Based on the number of preset dyes, a target imaging strategy is determined among each preset imaging strategy, wherein the preset imaging strategy includes a first imaging strategy and a second imaging strategy. According to the target imaging strategy, a reference fluorescence image corresponding to the reference laser is obtained; the reference laser is used to excite each of the preset dyes to emit reference fluorescence, so that the reference fluorescence image includes the fluorescence spot to be classified corresponding to each of the preset dyes; When the target imaging strategy includes the first imaging strategy, for any comparison laser, a comparison fluorescence image corresponding to the comparison laser is acquired, and based on each comparison fluorescence image and the reference fluorescence image, the target dye corresponding to each fluorescent spot to be classified is determined among each of the preset dyes; the comparison laser is used to excite each of the preset dyes to emit comparison fluorescence, so that the comparison fluorescence image includes the comparison fluorescence spot corresponding to each of the preset dyes; or, when the target imaging strategy includes the second imaging strategy, based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each of the preset dyes, and the reference fluorescence image, the target dye corresponding to each fluorescent spot to be classified is determined among each of the preset dyes; Based on the baseline fluorescence image and the target dye corresponding to each of the fluorescence spots to be classified, a target fluorescence microscopic image is generated.

2. The method of claim 1, wherein, The reference fluorescence image includes a first reference image and a second reference image; the step of determining the target dye corresponding to each of the preset dyes among the compared fluorescence images and the reference fluorescence images includes: For any of the fluorescent spots to be classified, the first light intensity and the second light intensity of the fluorescent spot to be classified are compared to obtain a first comparison result, and a first candidate dye is determined from each of the preset dyes based on the first comparison result; wherein, the first light intensity is the light intensity of the fluorescent spot to be classified on the first reference image, and the second light intensity is the light intensity of the fluorescent spot to be classified on the second reference image. If the number of the first candidate dyes is less than or equal to the first threshold, the first candidate dye is used as the target dye corresponding to the fluorescent spot to be classified.

3. The method of claim 2, wherein, The method further includes: If the number of dyes in the first candidate dyes is greater than the first threshold, a first image is determined in the reference fluorescence image, and a second image is determined in each of the comparison fluorescence images; the first image and the second image are acquired using the same acquisition device. The light intensity of the fluorescent spot to be classified in the first image is compared with the light intensity in the second image to obtain a second comparison result, and a second candidate dye is determined from the first candidate dyes based on the second comparison result; If the number of dyes in the second candidate dye is greater than the first threshold, the first image is used as the first reference image, the second image is used as the second reference image, the second candidate dye is used as the new first candidate dye, and the process jumps to the step of determining the first image in the reference fluorescence image and determining the second image in each of the comparison fluorescence images, until the number of dyes in the second candidate dye is less than or equal to the first threshold. The second alternative dye is used as the target dye corresponding to the fluorescent spot to be classified.

4. The method according to claim 3, characterized in that, The step of determining the first candidate dye from each of the preset dyes based on the first comparison result includes: The first dye category is determined based on the first comparison result, the reference laser, and the preset correspondence; the preset correspondence is used to characterize the relationship between the light intensity comparison result, the laser, and the dye category. Based on the first dye category, at least one first candidate dye is determined among the preset dyes; The step of determining a second candidate dye from the first candidate dyes based on the comparison result includes: The second dye category is determined based on the second comparison result, the first laser corresponding to the first image, the second laser corresponding to the second image, and the preset correspondence. Based on the second dye category, a second candidate dye is determined from the first candidate dyes.

5. The method according to claim 3, characterized in that, Determining the first image from the reference fluorescence image includes: For any of the fluorescent spots to be classified, if the first light intensity of the fluorescent spot to be classified is greater than or equal to the second light intensity, then the first reference image is used as the first image corresponding to the fluorescent spot to be classified; or, If the first light intensity of the fluorescent spot to be classified is less than the second light intensity, then the second reference image is used as the first image corresponding to the fluorescent spot to be classified.

6. The method according to claim 1, characterized in that, The step of determining the target dye corresponding to each fluorescent spot to be classified among the preset dyes, based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image, includes: Based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image, the predicted light intensity distribution data corresponding to each preset dye is determined; the reference fluorescence image includes a first reference image and a second reference image. Based on the first reference image and the second reference image, determine the image to be classified; Based on the actual light intensity of each fluorescent spot to be classified on the image to be classified, and the predicted light intensity distribution data corresponding to each preset dye, the target dye corresponding to each fluorescent spot to be classified is determined among the preset dyes.

7. The method according to claim 6, characterized in that, The step of determining the predicted light intensity distribution data corresponding to each preset dye based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each preset dye, and the reference fluorescence image includes: Based on the correspondence of the spectroscopic parameters of the spectrometer and the correspondence of the fluorescence data of each preset dye, the transmittance and reflectance of the reference fluorescence corresponding to each preset dye are determined. For any of the aforementioned reference fluorescence images, image autocorrelation processing is performed on the reference fluorescence images to obtain autocorrelation data; Image correlation processing is performed on the first reference image and the second reference image to obtain relevant data; Based on the transmittance, reflectance, autocorrelation data, and correlation data, the predicted light intensity distribution data corresponding to each preset dye is determined.

8. The method according to claim 1, characterized in that, The step of determining the target imaging strategy among various preset imaging strategies based on the number of preset dyes includes: When the number of preset dyes is equal to the second threshold, the second imaging strategy is used as the target imaging strategy.

9. A fluorescence microscopy imaging device, characterized in that, The apparatus is used to perform the fluorescence microscopy imaging method according to any one of claims 1 to 8, comprising: The strategy determination module is used to determine a target imaging strategy among various preset imaging strategies based on the number of preset dyes. The preset imaging strategies include a first imaging strategy and a second imaging strategy. The image acquisition module is used to acquire a reference fluorescence image corresponding to the reference laser according to the target imaging strategy; the reference laser is used to excite each of the preset dyes to emit reference fluorescence, so that the reference fluorescence image includes the fluorescence spot to be classified corresponding to each of the preset dyes; A dye determination module is configured to, when the target imaging strategy includes the first imaging strategy, acquire a comparison fluorescence image corresponding to any comparison laser for any comparison laser, and determine the target dye corresponding to each of the preset dyes among the preset dyes based on each comparison fluorescence image and the reference fluorescence image; the comparison laser is used to excite each of the preset dyes to emit comparison fluorescence, so that the comparison fluorescence image includes the comparison fluorescence spot corresponding to each of the preset dyes; or, when the target imaging strategy includes the second imaging strategy, determine the target dye corresponding to each of the preset dyes among the preset dyes based on the correspondence of the spectroscopic parameters of the spectrometer, the correspondence of the fluorescence data corresponding to each of the preset dyes, and the reference fluorescence image. The image generation module is used to generate a target fluorescence microscopic image based on the reference fluorescence image and the target dye corresponding to each of the fluorescence spots to be classified.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images