Imaging methods, systems, apparatuses, computer devices, and storage media
By performing linear confocal imaging and wide-field imaging in the horizontal and vertical directions on a micromirror chip, combined with computational processing, the problem of image quality being affected by pinholes and positional deviations in confocal imaging technology has been solved, thereby improving the signal-to-noise ratio and imaging speed.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NINGBO INVIEW INTELLIGENT TECH CO LTD
- Filing Date
- 2022-02-17
- Publication Date
- 2026-07-31
AI Technical Summary
The image quality of existing confocal imaging technology is affected by the size and position deviation of the confocal aperture, which limits the signal-to-noise ratio and resolution. Furthermore, the calculation results of existing algorithms are related to the noise of the original image, resulting in poor image quality.
Linear confocal imaging in the horizontal and vertical directions, as well as wide-field imaging, are performed using multiple micromirrors on a micromirror chip. The results are then processed to obtain the line scan confocal imaging results.
It improves the accuracy of image signal acquisition, reduces information loss, enhances the signal-to-noise ratio and imaging speed, and improves image quality.
Smart Images
Figure CN116661118B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of microscopic imaging technology, and in particular to an imaging method, system, apparatus, computer equipment, and storage medium. Background Technology
[0002] With the continuous development of optical microscopy imaging technology, confocal microscopy imaging technology has emerged. Due to its high stability and adaptability to a wide variety of samples, confocal microscopy imaging technology occupies an important position in the field of life science microscopy imaging.
[0003] The imaging quality of existing confocal imaging techniques is heavily dependent on the size and position of the confocal aperture: a large aperture reduces the ability to remove background and improve the signal-to-noise ratio without improving resolution; a small aperture allows less fluorescence signal to pass through, also reducing the signal-to-noise ratio; and a misalignment of the aperture position can drastically reduce the detected fluorescence signal. Therefore, related techniques typically use various algorithms (such as deconvolution Wiener filtering, maximum entropy, and Gerchberg-Saxton algorithms) to process confocal images. However, the quality of the confocal image obtained by these algorithms is highly dependent on the imaging quality of the original image and is significantly affected by noise in the original image, resulting in poor confocal image quality. Summary of the Invention
[0004] Therefore, it is necessary to provide an imaging method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve imaging quality in response to the above-mentioned technical problems.
[0005] In a first aspect, this application provides an imaging method. The method is applied to an imaging system including a micromirror chip, the micromirror chip comprising a plurality of micromirrors; the method includes:
[0006] By using multiple micromirrors on the micromirror chip, a horizontal linear confocal imaging is performed on the target object to obtain a first imaging result of the target object;
[0007] By using multiple micromirrors on the micromirror chip, the target object is subjected to linear confocal imaging in the vertical direction to obtain a second imaging result of the target object;
[0008] Wide-field imaging of the target object is performed using multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object;
[0009] Based on the first imaging result, the second imaging result, and the wide-field imaging result, the line scan confocal imaging result of the target object is calculated.
[0010] In one embodiment, the step of performing horizontal linear confocal imaging of the target object using multiple micromirrors on the micromirror chip to obtain a first imaging result of the target object includes:
[0011] According to a preset horizontal order, multiple target rows are sequentially determined along the horizontal direction of the micromirrors on the micromirror chip;
[0012] For each target row on the micromirror chip, the micromirror of the target row is activated, and the micromirror of the row preceding the target row is deactivated, so that the micromirror of the target row illuminates the target object, and linear confocal imaging is performed on the target object to obtain the imaging result of the target row;
[0013] Based on the imaging results of the multiple target rows, a first imaging result of the target object is obtained.
[0014] In one embodiment, the step of performing vertical linear confocal imaging of the target object using multiple micromirrors on the micromirror chip to obtain a second imaging result of the target object includes:
[0015] According to a preset vertical order, multiple target columns are sequentially determined along the vertical direction of the micromirrors on the micromirror chip;
[0016] For each target column on the micromirror chip, the micromirror of the target column is activated and the micromirror of the column preceding the target column is deactivated, so that the micromirror of the target column illuminates the target object and performs linear confocal imaging on the target object to obtain the imaging result of the target column;
[0017] Based on the imaging results of the multiple target columns, a second imaging result of the target object is obtained.
[0018] In one embodiment, calculating the line scan confocal imaging result of the target object based on the first imaging result, the second imaging result, and the wide-field imaging result includes:
[0019] The target value is obtained by summing the first imaging result and the second imaging result.
[0020] The difference between the target value and the wide-field imaging result is taken as the line scan confocal imaging result of the target object.
[0021] In one embodiment, the method further includes:
[0022] Based on the first imaging result, the second imaging result is aligned to obtain the third imaging result corresponding to the second imaging result;
[0023] The first imaging result and the third imaging result are fused to obtain a preliminary fusion result;
[0024] Based on the preliminary fusion results, joint deconvolution calculation is performed to obtain the line scan confocal imaging results of the target object.
[0025] Secondly, this application also provides an imaging system, the system comprising: a micromirror chip, a laser light source module, an imaging module, and a data processing module; wherein, the micromirror chip comprises multiple micromirrors.
[0026] The laser source module is used to emit a laser source;
[0027] The micromirror is used to irradiate the target object with the laser light source, so that the imaging module performs linear confocal imaging of the target object in the horizontal direction to obtain a first imaging result of the target object; performs linear confocal imaging of the target object in the vertical direction to obtain a second imaging result of the target object; and performs wide-field imaging of the target object to obtain a wide-field imaging result of the target object.
[0028] The data processing module is used to calculate the line scan confocal imaging result of the target object based on the first imaging result, the second imaging result, and the wide field imaging result.
[0029] In one embodiment, the system further includes an imaging control module;
[0030] The imaging control module is used to sequentially determine multiple target rows along the horizontal direction of the micromirrors on the micromirror chip according to a preset horizontal order; for each target row on the micromirror chip, the micromirrors of the target row are activated and the micromirrors of the row preceding the target row are deactivated, so that the micromirrors of the target row illuminate the target object, and linear confocal imaging is performed on the target object to obtain the imaging result of the target row; based on the imaging results of the multiple target rows, a first imaging result of the target object is obtained.
[0031] The imaging control module is further configured to sequentially determine multiple target columns along the vertical direction of the micromirrors on the micromirror chip according to a preset vertical order; for each target column on the micromirror chip, activate the micromirrors of the target column and deactivate the micromirrors of the column preceding the target column, so that the micromirrors of the target column illuminate the target object, perform linear confocal imaging on the target object, and obtain the imaging result of the target column; and obtain a second imaging result of the target object based on the imaging results of the multiple target columns.
[0032] Thirdly, this application also provides an imaging device, the device comprising:
[0033] The first imaging unit is used to perform linear confocal imaging of the target object in the horizontal direction through multiple micromirrors on the micromirror chip to obtain the first imaging result of the target object;
[0034] The second imaging unit is used to perform vertical linear confocal imaging of the target object through multiple micromirrors on the micromirror chip to obtain a second imaging result of the target object;
[0035] A wide-field imaging unit is used to perform wide-field imaging on the target object through multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object;
[0036] The calculation unit is used to calculate the line scan confocal imaging result of the target object based on the first imaging result, the second imaging result, and the wide field imaging result.
[0037] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:
[0038] By using multiple micromirrors on the micromirror chip, a horizontal linear confocal imaging is performed on the target object to obtain a first imaging result of the target object;
[0039] By using multiple micromirrors on the micromirror chip, the target object is subjected to linear confocal imaging in the vertical direction to obtain a second imaging result of the target object;
[0040] Wide-field imaging of the target object is performed using multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object;
[0041] Based on the first imaging result, the second imaging result, and the wide-field imaging result, the line scan confocal imaging result of the target object is calculated.
[0042] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:
[0043] By using multiple micromirrors on the micromirror chip, a horizontal linear confocal imaging is performed on the target object to obtain a first imaging result of the target object;
[0044] By using multiple micromirrors on the micromirror chip, the target object is subjected to linear confocal imaging in the vertical direction to obtain a second imaging result of the target object;
[0045] Wide-field imaging of the target object is performed using multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object;
[0046] Based on the first imaging result, the second imaging result, and the wide-field imaging result, the line scan confocal imaging result of the target object is calculated.
[0047] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:
[0048] By using multiple micromirrors on the micromirror chip, a horizontal linear confocal imaging is performed on the target object to obtain a first imaging result of the target object;
[0049] By using multiple micromirrors on the micromirror chip, the target object is subjected to linear confocal imaging in the vertical direction to obtain a second imaging result of the target object;
[0050] Wide-field imaging of the target object is performed using multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object;
[0051] Based on the first imaging result, the second imaging result, and the wide-field imaging result, the line scan confocal imaging result of the target object is calculated.
[0052] The aforementioned imaging method, apparatus, computer equipment, storage medium, and computer program product, through multiple micromirrors on the micromirror chip, perform horizontal linear confocal imaging, vertical linear confocal imaging, and wide-field imaging on the target object, obtaining a first imaging result, a second imaging result, and a wide-field imaging result for the target object; based on the aforementioned imaging results, the line-scan confocal imaging result of the target object is calculated. Using the imaging method provided by the embodiments of the present invention, confocal line-scan laser illumination imaging can be performed in both the horizontal and vertical directions, and signals within the same field of view can be sampled multiple times in different directions. The final imaging result is calculated based on the sampling results, which can improve the accuracy of image signal acquisition, reduce information loss, improve the signal-to-noise ratio, increase imaging speed and spatial resolution, and improve the imaging quality of the final imaging result. Attached Figure Description
[0053] Figure 1 This is a structural diagram of an imaging system in one embodiment;
[0054] Figure 2 This is a flowchart illustrating an imaging method in one embodiment;
[0055] Figure 3This is a flowchart illustrating the steps for obtaining the first imaging result in one embodiment;
[0056] Figure 4 This is a flowchart illustrating the steps for obtaining the second imaging result in one embodiment;
[0057] Figure 5 This is a flowchart illustrating the steps for achieving a line scan confocal imaging result of a target object in one embodiment;
[0058] Figure 6 This is a schematic diagram of the imaging system in another embodiment;
[0059] Figure 7A This is a schematic diagram of the structure of module 1 in the imaging system in another embodiment;
[0060] Figure 7B This is a schematic diagram of the structure of module 2 in the imaging system in another embodiment;
[0061] Figure 7C This is a schematic diagram of the structure of module 3 in the imaging system in another embodiment;
[0062] Figure 7D This is a schematic diagram of the structure of module 4 in the imaging system in another embodiment;
[0063] Figure 8 This is a schematic diagram of a micromirror chip in one embodiment;
[0064] Figure 9A This is a schematic diagram illustrating the sequence of micromirror opening during horizontal imaging in an imaging system according to one embodiment.
[0065] Figure 9B This is a schematic diagram illustrating the order in which a sample is irradiated during horizontal imaging in one embodiment of an imaging system.
[0066] Figure 10A This is a schematic diagram illustrating the sequence of micromirror opening during vertical imaging in an imaging system in one embodiment.
[0067] Figure 10B This is a schematic diagram illustrating the order in which a sample is irradiated during vertical imaging in one embodiment of an imaging system.
[0068] Figure 11A This is a schematic diagram showing the micromirrors in the open state when performing wide-field imaging in an imaging system in one embodiment;
[0069] Figure 11B This is a schematic diagram illustrating the order in which the sample is irradiated when performing wide-field imaging in an imaging system in one embodiment.
[0070] Figure 12This is a structural block diagram of an imaging device in one embodiment;
[0071] Figure 13 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0072] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0073] The method provided in this application embodiment can be applied to the field of fluorescence microscopy imaging, particularly confocal microscopy imaging. The imaging method provided in this application embodiment can be applied to, for example... Figure 1 The imaging system 100 shown in the application environment includes a micromirror chip 200, a laser source module 300, an imaging module 400, an imaging control module 500, and a data processing module 600. The imaging module 400 communicates with the data processing module 600 via a network. The data processing module 600 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. The data processing module 600 can also be a standalone server or a server cluster consisting of multiple servers.
[0074] In this embodiment, as Figure 2 As shown, the imaging method includes the following steps:
[0075] Step 102: Using multiple micromirrors on the micromirror chip, perform linear confocal imaging of the target object in the horizontal direction to obtain the first imaging result of the target object.
[0076] The micromirror chip can be a digital micromirror device (DMD), which includes multiple pixel-level micromirrors (micromirror pixels). For example, it can be a micromirror chip composed of N×N micromirror pixels, meaning that the micromirror chip has N rows and N columns of micromirrors. This micromirror chip can generate line-scan confocal illumination and realize the function of a confocal pinhole. The target object can be a sample containing a fluorescently labeled substance.
[0077] Specifically, the imaging system can generate horizontal line-scan illumination light by opening multiple micromirrors on the micromirror chip and perform confocal imaging on the target object to obtain the first imaging result of the target object.
[0078] In one example, the imaging system may further include a sample placement cell and a detector. The sample placement cell can be a regular sample cell or a live cell culture dish configured to maintain cell viability, etc., with the target object placed in the sample placement cell. A laser source module generates a laser beam, and the imaging system, through micromirrors on the micromirror chip, can horizontally irradiate the corresponding position on the target object. This excites the fluorescently labeled material at the laser-irradiated position, producing fluorescence. The fluorescence is collected and reflected back to the micromirrors on the micromirror chip. Thus, the micromirrors on the micromirror chip reflect the fluorescence, enabling linear confocal imaging on the detector to obtain the first image of the target object.
[0079] Step 104: Using multiple micromirrors on the micromirror chip, linear confocal imaging is performed on the target object in the vertical direction to obtain the second imaging result of the target object.
[0080] Specifically, the imaging system can generate vertical line scan illumination light by opening multiple micromirrors on the micromirror chip and perform confocal imaging on the target object to obtain a second imaging result of the target object.
[0081] In one example, a laser source module generates a laser beam. The imaging system, through micromirrors on a micromirror chip, can vertically direct the laser beam to the corresponding location on the target object. This excites the fluorescently labeled material at the laser-irradiated location, causing it to fluoresce. The fluorescence is collected and reflected back to the micromirrors on the micromirror chip. The micromirrors then reflect the fluorescence, enabling linear confocal imaging on a detector to obtain a second image of the target object.
[0082] Step 106: Wide-field imaging of the target object is performed using multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object.
[0083] Specifically, the process of wide-field imaging of the target object can be as follows: the imaging system activates all micromirrors on the micromirror chip. When the laser source module generates laser light, all micromirrors on the micromirror chip can irradiate the target object with the laser. Fluorescent materials at all locations on the target object are excited by the laser, producing fluorescence. This fluorescence is collected and reflected back to all the activated micromirrors on the micromirror chip. The activated micromirrors then reflect the fluorescence, resulting in wide-field imaging on the detector, thus obtaining the wide-field imaging result of the target object.
[0084] Optionally, steps 102, 104, and 108 do not need to be executed in a specific order in actual application scenarios.
[0085] Step 108: Calculate the line scan confocal imaging result of the target object based on the first imaging result, the second imaging result, and the wide field imaging result.
[0086] Specifically, the imaging system calculates based on the first imaging result, the second imaging result, and the wide-field imaging result. Since the first imaging result is obtained by imaging with horizontal line-scan illumination light, it contains horizontal noise. Similarly, the second imaging result contains vertical noise, and the wide-field imaging result contains noise in all directions. Thus, since the sum of the horizontal and vertical noise images constitutes the omnidirectional noise image, the calculation process for the noise-free imaging result can be: the sum of the first and second imaging results, the difference between this sum and the wide-field imaging result, and the difference used as the line-scan confocal imaging result of the target object.
[0087] In the above imaging method, multiple micromirrors on the micromirror chip are used to perform horizontal linear confocal imaging, vertical linear confocal imaging, and wide-field imaging on the target object, obtaining a first imaging result, a second imaging result, and a wide-field imaging result for the target object. Based on the above imaging results, the line-scan confocal imaging result of the target object is calculated. Using the imaging method provided by this embodiment of the invention, confocal line-scan laser illumination imaging can be performed in both the horizontal and vertical directions, and signals within the same field of view can be sampled multiple times in different directions. The final imaging result is calculated based on the sampling results, which can improve the accuracy of image signal acquisition, reduce information loss, improve the signal-to-noise ratio, and improve the imaging quality of the final imaging result.
[0088] In one embodiment, such as Figure 3 As shown, the specific processing steps of step 104, "using multiple micromirrors on the micromirror chip to perform horizontal linear confocal imaging of the target object to obtain the first imaging result of the target object," include:
[0089] Step 202: Determine multiple target rows sequentially along the horizontal direction of the micromirrors on the micromirror chip according to a preset horizontal order.
[0090] Specifically, the micromirror chip can be a digital micromirror device (DMD), which includes multiple pixel-level micromirrors (micromirror pixels). For example, it can be a micromirror chip composed of N×N micromirror pixels, that is, there are N rows and N columns of micromirrors on the micromirror chip. In this way, multiple target rows on the micromirror chip can be determined sequentially according to the arrangement order of the first row, the second row, etc., according to a preset horizontal order.
[0091] Step 204: For each target row on the micromirror chip, activate the micromirror of the target row and deactivate the micromirror of the row preceding the target row, so that the micromirror of the target row illuminates the target object and performs linear confocal imaging on the target object to obtain the imaging result of the target row.
[0092] Specifically, for each target row on the micromirror chip, the imaging system turns on the micromirrors of that target row at the current moment and turns off the micromirrors of other rows, so that only the micromirrors of the target row are in the open state at the current moment. In this way, the open micromirrors can receive the laser emitted by the laser source module and illuminate the target location of the target object. The fluorescent material marked at the target location of the target object will fluoresce. The generated fluorescence is collected by the imaging system and reflected back to the micromirrors on the target row of the micromirror chip. The micromirrors on the target row can reflect the fluorescence, and linear confocal imaging is performed on the detector to obtain the imaging result of the target row.
[0093] The x and y coordinates of the target position of the target object correspond to the x and y coordinates of the target row position on the micromirror chip.
[0094] Step 206: Based on the imaging results of multiple target rows, obtain the first imaging result of the target object.
[0095] The multiple target rows are composed of all the rows on the micromirror chip. The imaging system combines the multiple imaging results from the multiple target rows to obtain the first imaging result of the target object.
[0096] In one example, the imaging system places the target object in the sample cell. The system then activates the first row of micromirrors on the micromirror chip, allowing them to illuminate the target object and perform linear confocal imaging, obtaining the first row's image result. Next, the system activates the second row of micromirrors while simultaneously deactivating the first row, allowing the second row to illuminate the target object and perform linear confocal imaging, obtaining the second row's image result. This process is repeated until the last row of micromirrors is activated and the row preceding it is deactivated, allowing the last row to illuminate the target object and perform linear confocal imaging, obtaining the last row's image result. By performing linear confocal imaging on the target object using the micromirror chip, multiple rows of image results are obtained and combined to obtain the first imaging result.
[0097] In another example, the target rows can include two or three rows, and the specific number of rows can be determined according to the actual application scenario.
[0098] In this embodiment, by opening the micromirrors sequentially in the horizontal direction and performing confocal line scanning laser irradiation imaging in the horizontal direction in a line scanning manner, the accuracy of signal acquisition is improved, information loss is reduced, the signal-to-noise ratio is improved, and the imaging quality is improved to a certain extent.
[0099] In one embodiment, such as Figure 4 As shown, the specific processing steps of step 106, "using multiple micromirrors on the micromirror chip to perform vertical linear confocal imaging of the target object to obtain a second imaging result of the target object," include:
[0100] Step 302: Determine multiple target columns sequentially along the vertical direction of the micromirrors on the micromirror chip according to a preset vertical order.
[0101] The preset vertical order can be determined by sequentially determining multiple target columns on the micromirror chip according to the arrangement order of the first and second columns of the micromirror chip.
[0102] Specifically, the imaging system can sequentially determine multiple target columns in a vertical order, along the vertical direction of the micromirrors arranged on the micromirror chip.
[0103] Step 304: For each target column on the micromirror chip, activate the micromirror of the target column and deactivate the micromirror of the column preceding the target column, so that the micromirror of the target column illuminates the target object and performs linear confocal imaging on the target object to obtain the imaging result of the target column.
[0104] Specifically, for each target column on the micromirror chip, the imaging system opens the micromirrors of that target column at the current moment and closes the micromirrors of other columns, so that only the micromirrors of the target column are open at the current moment. In this way, the open micromirrors can receive the laser emitted by the laser source module and illuminate the target location of the target object. The fluorescent material marked at the target location of the target object will fluoresce. The generated fluorescence is collected by the imaging system and reflected back to the micromirrors on the target column of the micromirror chip. The micromirrors on the target column can reflect the fluorescence, performing linear confocal imaging on the detector to obtain the imaging result of the target column.
[0105] The x and y coordinates of the target position of the target object correspond to the x and y coordinates of the target column position on the micromirror chip.
[0106] Step 306: Based on the imaging results of multiple target columns, obtain the second imaging result of the target object.
[0107] The multiple target columns are composed of all columns on the micromirror chip. The imaging system combines multiple imaging results from these multiple target columns to obtain a second imaging result of the target object.
[0108] In one example, the imaging system places the target object in the sample cell. The system then activates the first column of micromirrors on the micromirror chip, allowing them to illuminate the target object and perform linear confocal imaging, obtaining the first column's image. Next, the system activates the second column of micromirrors on the chip while simultaneously deactivating the first column, allowing the second column to illuminate the target object and perform linear confocal imaging, obtaining the second column's image. This process is repeated until the last column of micromirrors on the chip is activated and the column preceding it is deactivated, allowing the last column to illuminate the target object and perform linear confocal imaging, obtaining the last column's image. By performing linear confocal imaging on the target object using the micromirror chip, multiple columns of image results are obtained, which are then combined to obtain the second imaging result.
[0109] In another example, the target column can include two or three columns, and the specific number of columns can be determined according to the actual application scenario.
[0110] In this embodiment, by opening the micromirrors sequentially in the vertical direction and performing confocal line scanning laser irradiation imaging in the vertical direction in a line scanning manner, the accuracy of signal acquisition is improved, information loss is reduced, the signal-to-noise ratio is improved, and the imaging quality is improved to a certain extent.
[0111] In one embodiment, such as Figure 5 As shown, the specific processing steps of step 108, "calculate the line scan confocal imaging result of the target object based on the first imaging result, the second imaging result, and the wide-field imaging result," include:
[0112] Step 402: Summate the first imaging result and the second imaging result to obtain the target value.
[0113] Step 404: The difference between the target value and the wide-field imaging result is used as the line scan confocal imaging result of the target object.
[0114] Specifically, the line scan confocal imaging result of the target object is calculated using the following formula:
[0115] I1=I signal +I background-x ,
[0116] I2=I signal +I background-y ,
[0117] I3 = I signal +I background ,
[0118] I signal=I1+I2-I3,
[0119] Wherein, I1 is the first imaging result, I2 is the second imaging result, I3 is the wide-field imaging result, and I... signal It is the line scan confocal imaging result of the target object, I background-x It is a noisy image in the horizontal direction, I background-y It is a noisy image in the vertical direction.
[0120] In this embodiment, by employing a line-scan confocal imaging method, the imaging speed is significantly improved compared to point-scan confocal imaging. Furthermore, confocal imaging avoids the mirror control stability and image distortion problems associated with resonant scanning imaging. For example, when processing 512x512 resolution images, the imaging speed can be increased to 30 f / s. By using line-scan confocal imaging in different directions and performing addition and subtraction calculations on the results of three imaging sessions, the effect of removing background light and improving the imaging signal-to-noise ratio can be achieved.
[0121] In one possible implementation, the method further includes:
[0122] Based on the first imaging result, the second imaging result is aligned to obtain the third imaging result corresponding to the second imaging result; the first imaging result and the third imaging result are fused to obtain the preliminary fusion result; based on the preliminary fusion result, joint deconvolution calculation is performed to obtain the line scan confocal imaging result of the target object.
[0123] Specifically, the imaging system can perform joint deconvolution calculations on the first imaging result (SideA) and the second imaging result (SideB). The specific calculation process can be as follows: Align the second imaging result according to the first imaging result. First, calculate the normalized cross-correlation value between the first and second imaging results. Then, align the second imaging result based on this normalized cross-correlation value to obtain the aligned second imaging result, i.e., the third imaging result. The matrix data after alignment (aligning the second imaging result to the first imaging result) can be SideA (the first imaging result) and SideB' (the third imaging result). In this way, the first and second imaging results can be fused to obtain a preliminary fusion result. The specific fusion process can be as follows: Calculate the sum of the first and third imaging results, and perform a weighted calculation on the sum according to preset weights to obtain the preliminary fusion result. For example, the preliminary fusion result can be calculated using the following formula:
[0124] Result0=1 / 2*(SideA+SideB').
[0125] In this way, the imaging system can perform joint deconvolution calculations based on the preliminary fusion results to obtain the line scan confocal imaging results of the target object. The specific calculation process can be achieved using the following formula:
[0126] For i = 1, 2, ..., N
[0127] ResultA=Resulti-1*BlurA(SideA / BlurA(Resulti-1));
[0128] Result i =ResultA*BlurB(SideB' / BlurB(ResultA))
[0129] Among them, Result i It refers to the calculation result of joint deconvolution.
[0130] In this embodiment, by performing joint deconvolution calculations, the signal-to-noise ratio and resolution of the target object (sample) are simultaneously improved in each arrangement direction.
[0131] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0132] Based on the same inventive concept, this application also provides an imaging system for implementing the imaging method described above. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations in one or more imaging system embodiments provided below can be found in the limitations of the imaging method described above, and will not be repeated here.
[0133] In one embodiment, such as Figure 1 As shown, an imaging system 100 is provided, including: a micromirror chip 200, a laser light source module 300, an imaging module 400, and a data processing module 600; wherein, the micromirror chip includes multiple micromirrors:
[0134] Laser source module 300 is used to emit laser light.
[0135] A micromirror is used to illuminate a target object with a laser source, so that the imaging module 400 performs linear confocal imaging of the target object in the horizontal direction to obtain a first imaging result of the target object; performs linear confocal imaging of the target object in the vertical direction to obtain a second imaging result of the target object; and performs wide-field imaging of the target object to obtain a wide-field imaging result of the target object.
[0136] The data processing module 600 is used to calculate the line scan confocal imaging result of the target object based on the first imaging result, the second imaging result, and the wide field imaging result.
[0137] In one example, such as Figure 6 As shown, the imaging system may include modules 1, 2, 3, 4, and 5. Module 1 is the main body of the microscope; module 2 is the laser source module 300; module 3 is the confocal illumination module; module 4 is the confocal detection module; and module 5 is the control module. Module 1 includes an objective lens (OBJ); module 2 includes a laser driver, laser 1, laser 2, laser 3, an acousto-optic filter (AOTF), and a coupling lens; module 3 includes a digital micromirror array (DMD) and a switchable spatial filter; and module 5 includes a workstation, image detector control (to cameras), and stage driver. Modules 2 and 3 in this imaging system are connected via single-mode fiber (SM fiber).
[0138] Specifically, such as Figure 7A The diagram shows detailed illustrations of the components included in Module 1. 1-1 is the microscope objective; 1-2 is the microscope tube; 1-3 and 1-4 are reflecting mirrors; 1-5 is the microscope platform; and 1-6 is the sample cell, which can be a standard sample cell or a live cell culture dish designed to maintain cell viability. The microscope platform can be manually operated, electrically operated, or piezoelectric.
[0139] like Figure 7BThe diagram shows the detailed components of module 2 (Laser driver). 2-1 is the laser; the wavelength and number of lasers are determined based on the actual application scenario and are not limited in this invention. 2-2 is the laser control system, which controls the laser's on / off state, output intensity, and synchronization with other components. 2-3 is the laser coupling device (Coupling Lens), which, through optical path design, couples the outputs of several lasers into a single-mode fiber. 2-4 is the single-mode fiber.
[0140] like Figure 7C The diagram shows detailed illustrations of the components included in module 3. 3-1 is a fiber optic collimating lens, which collimates the laser output from a single-mode fiber. 3-2 is a dichroic mirror, which reflects the illumination light and transmits the fluorescence emitted by the sample. 3-3 is a digital micromirror device (DMD, also known as micromirror chip 200), used to generate line-scan confocal illumination light and to realize the function of a confocal pinhole. 3-3 consists of NxN micromirror pixels arranged as shown in the diagram. Figure 8 As shown, Figure 8 The illustrations shown are for illustrative purposes only. The number of pixels N is not the same as that of the actual parts and can be determined according to the actual application scenario. This invention does not limit this. 3-4 and 3-5 are relay lenses; 3-6 and 3-7 are plane mirrors; 3-8 is a pattern selectable spatial filter, which can further select the line scan confocal illumination structure to improve the illumination quality.
[0141] like Figure 7D The diagram shows detailed illustrations of the components included in Module 4. 4-1 and 4-2 are relay lenses; 4-2 is optional and can be omitted or replaced with appropriate beam-splitting components to separate fluorescence based on properties such as intensity and wavelength, thereby improving imaging speed or expanding imaging capabilities. 4-3 is a fluorescence filter that filters out laser light while allowing fluorescence to pass through. 4-5 is an imaging sensor, which can be SCMOS, EMCCD, etc. Module 5 consists of an image workstation, component controllers, etc., which will not be described further here.
[0142] In one example, in conjunction with the imaging system described above, the specific execution process of the imaging method provided in this embodiment of the invention may include:
[0143] Step 1, as follows Figure 9AAs shown, a row of micromirrors are open. Micromirrors marked with triangles are in the open state, while micromirrors in other positions are in the closed state. Micromirrors in the open state can irradiate the corresponding position of the sample with laser light. The fluorescent labeling material at the corresponding position of the sample irradiated by the laser light will be excited and then produce fluorescence. The fluorescence is collected by objective lens 1-1, passes through tube lenses 1-2, 3-5, 3-7, 3-8, 3-6, and 3-4, and is reflected back to digital micromirror 3-3. After being reflected by the micromirror in the open position on the digital micromirror chip, it passes through 3-2 and finally passes through module 4 to be imaged on the detector. Fluorescence on the focal plane of the sample will illuminate the micromirrors in the digital micromirror chip that are in the open state and be reflected into the imaging module 4. Meanwhile, fluorescence on the defocus plane will illuminate the micromirrors in the digital micromirror chip that are in the closed state, deviate from the imaging module and be absorbed. Therefore, the micromirrors in the digital micromirror chip that are in the closed state can both modulate the illumination laser and act as a confocal aperture. Compared with traditional confocal systems, this eliminates the tedious process of collimating the aperture and avoids the problem of image quality degradation caused by inaccurate aperture collimation.
[0144] The imaging control module 500 will turn on sequentially along the horizontal direction (the direction of the micromirrors in the diagram showing the "on" sequence) according to a preset horizontal order, and turn off the micromirrors in the previous row. Thus, as... Figure 9B As shown, the sample is also irradiated by the confocal line scanning laser (the triangular marks indicate the corresponding positions of the irradiated sample), and imaging is performed sequentially. When the confocal line scanning imaging process is completed, the first imaging result is recorded.
[0145] Step 2, as follows Figure 10A As shown, a row of micromirrors is opened; those marked with triangles are in the open state, illuminating the corresponding positions on the sample and performing confocal imaging. The imaging control module 500 opens sequentially along the vertical direction (following the "on" order of the micromirrors in the diagram) according to a pre-defined vertical sequence, and closes the previous row simultaneously. Figure 10B As shown, the sample is also irradiated by the confocal line scanning laser (the triangular marks indicate the corresponding positions of the irradiated sample), and imaging is performed sequentially. When the confocal line scanning imaging process is completed, the second imaging result is recorded.
[0146] Step 3, as follows Figure 11A As shown, all micromirrors on the micromirror chip are in the open state; those marked with a triangle are in the open state, such as... Figure 11B As shown, wide-field imaging was performed on the sample (the triangular markers indicate the corresponding positions of the sample that were illuminated), and the wide-field imaging results were recorded.
[0147] Step 4, data processing. There are two methods for data processing:
[0148] In one possible implementation, background noise in one direction can be removed from the sample obtained by line scan confocal imaging, so it can be calculated using the following formula:
[0149] I1 = Isignal + Ibackground - x
[0150] I2 = Isignal + Ibackground - y
[0151] I3 = Isignal + Ibackground
[0152] Isignal = I1 + I2 - I3.
[0153] In this embodiment, all background light during the sample imaging process can be removed through calculation, which greatly improves the imaging speed while maintaining the imaging signal-to-noise ratio compared with point scanning confocal imaging technology.
[0154] In another possible implementation, the imaging quality of the sample in line scan confocal imaging can be improved.
[0155] The first and second imaging results are subjected to a linear scanning mode in two directions. The image signals in the x and y directions are then subjected to joint-deconvolution calculation to improve the imaging quality.
[0156] Because this method uses line scanning imaging, the number of samples required is 2*n when the image has n*n pixels, while the time required by traditional confocal microscopy is n*n. Therefore, the time is shortened and the efficiency is improved. In summary, joint-deconvolution can shorten the sampling time, improve sampling efficiency, and increase resolution.
[0157] The imaging method provided in this invention can improve imaging speed. It adopts a line scanning imaging method, which greatly improves the imaging speed compared with the point scanning confocal imaging method. The DMD is used as a confocal illumination and detection aperture, avoiding the image quality degradation caused by incorrect aperture size selection and position misalignment. The algorithm is highly efficient and improves the resolution of the imaging results in multiple directions.
[0158] In one embodiment, the imaging system further includes an imaging control module 500; the imaging control module 500 is used to sequentially determine multiple target rows along the horizontal direction of the micromirrors on the micromirror chip according to a preset horizontal order; for each target row on the micromirror chip, the micromirrors of the target row are activated and the micromirrors of the row preceding the target row are deactivated, so that the micromirrors of the target row illuminate the target object, and linear confocal imaging is performed on the target object to obtain the imaging result of the target row; based on the imaging results of the multiple target rows, a first imaging result of the target object is obtained.
[0159] The imaging control module 500 is also used to sequentially determine multiple target columns according to a preset vertical order along the vertical direction of the micromirrors on the micromirror chip; for each target column on the micromirror chip, the micromirrors of the target column are activated and the micromirrors of the column preceding the target column are deactivated, so that the micromirrors of the target column illuminate the target object and perform linear confocal imaging on the target object to obtain the imaging result of the target column; and based on the imaging results of multiple target columns, a second imaging result of the target object is obtained.
[0160] Based on the same inventive concept, this application also provides an imaging apparatus for implementing the imaging method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more imaging apparatus embodiments provided below can be found in the limitations of the imaging method described above, and will not be repeated here.
[0161] In one embodiment, such as Figure 12 As shown, an imaging device 601 is provided, including: a first imaging unit 602, a second imaging unit 603, a wide-field imaging unit 604, and a computing unit 605, wherein:
[0162] The first imaging unit 602 is used to perform linear confocal imaging of the target object in the horizontal direction through multiple micromirrors on the micromirror chip to obtain the first imaging result of the target object.
[0163] The second imaging unit 603 is used to perform linear confocal imaging of the target object in the vertical direction through multiple micromirrors on the micromirror chip to obtain a second imaging result of the target object.
[0164] The wide-field imaging unit 604 is used to perform wide-field imaging of the target object through multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object.
[0165] The calculation unit 605 is used to calculate the line scan confocal imaging result of the target object based on the first imaging result, the second imaging result, and the wide field imaging result.
[0166] Each module in the aforementioned imaging device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0167] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 13 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores imaging-related data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements an imaging method.
[0168] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0169] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.
[0170] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0171] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0172] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.
[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. An imaging method, characterized in that, The method is applied to an imaging system including a micromirror chip, the micromirror chip including a plurality of micromirrors; the method includes: By using multiple micromirrors on the micromirror chip, a horizontal linear confocal imaging is performed on the target object to obtain a first imaging result of the target object; By using multiple micromirrors on the micromirror chip, the target object is subjected to linear confocal imaging in the vertical direction to obtain a second imaging result of the target object; Wide-field imaging of the target object is performed using multiple micromirrors on the micromirror chip to obtain the wide-field imaging result of the target object; The first imaging result and the second imaging result are summed to obtain the target value; the difference between the target value and the wide-field imaging result is taken as the line scan confocal imaging result of the target object. The step of performing horizontal linear confocal imaging of the target object using multiple micromirrors on the micromirror chip to obtain a first imaging result of the target object includes: According to a preset horizontal order, multiple target rows are sequentially determined along the horizontal direction of the micromirrors on the micromirror chip; for each target row on the micromirror chip, the micromirrors of the target row are activated and the micromirrors of the row preceding the target row are deactivated, so that the micromirrors of the target row illuminate the target object, and linear confocal imaging is performed on the target object to obtain the imaging result of the target row; based on the imaging results of the multiple target rows, a first imaging result of the target object is obtained.
2. The method according to claim 1, characterized in that, The step of performing vertical linear confocal imaging of the target object using multiple micromirrors on the micromirror chip to obtain a second imaging result of the target object includes: According to a preset vertical order, multiple target columns are sequentially determined along the vertical direction of the micromirrors on the micromirror chip; For each target column on the micromirror chip, the micromirror of the target column is activated and the micromirror of the column preceding the target column is deactivated, so that the micromirror of the target column illuminates the target object and performs linear confocal imaging on the target object to obtain the imaging result of the target column; Based on the imaging results of the multiple target columns, a second imaging result of the target object is obtained.
3. The method according to claim 1, characterized in that, The method further includes: Based on the first imaging result, the second imaging result is aligned to obtain the third imaging result corresponding to the second imaging result; The first imaging result and the third imaging result are fused to obtain a preliminary fusion result; Based on the preliminary fusion results, joint deconvolution calculation is performed to obtain the line scan confocal imaging results of the target object.
4. An imaging system, characterized in that, The system includes: a micromirror chip, a laser source module, an imaging module, a data processing module, and an imaging control module; wherein, the micromirror chip includes multiple micromirrors. The laser source module is used to emit a laser source; The micromirror is used to irradiate the target object with the laser light source, so that the imaging module performs linear confocal imaging of the target object in the horizontal direction to obtain a first imaging result of the target object; performs linear confocal imaging of the target object in the vertical direction to obtain a second imaging result of the target object; and performs wide-field imaging of the target object to obtain a wide-field imaging result of the target object. The data processing module is used to sum the first imaging result and the second imaging result to obtain a target value; and to use the difference between the target value and the wide-field imaging result as the line scan confocal imaging result of the target object. The imaging control module is used to sequentially determine multiple target rows along the horizontal direction of the micromirrors on the micromirror chip according to a preset horizontal order; for each target row on the micromirror chip, the micromirrors of the target row are activated and the micromirrors of the row preceding the target row are deactivated, so that the micromirrors of the target row illuminate the target object, and linear confocal imaging is performed on the target object to obtain the imaging result of the target row; based on the imaging results of the multiple target rows, a first imaging result of the target object is obtained.
5. The system according to claim 4, characterized in that, The imaging control module is further configured to sequentially determine multiple target columns along the vertical direction of the micromirrors on the micromirror chip according to a preset vertical order; for each target column on the micromirror chip, activate the micromirrors of the target column and deactivate the micromirrors of the column preceding the target column, so that the micromirrors of the target column illuminate the target object, perform linear confocal imaging on the target object, and obtain the imaging result of the target column; Based on the imaging results of the multiple target columns, a second imaging result of the target object is obtained.
6. The system according to claim 4, characterized in that, The data processing module is also used for: Based on the first imaging result, the second imaging result is aligned to obtain a third imaging result corresponding to the second imaging result; the first imaging result and the third imaging result are fused to obtain a preliminary fusion result; based on the preliminary fusion result, joint deconvolution calculation is performed to obtain the line scan confocal imaging result of the target object.
7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.