Scanning imaging method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a scanning imaging method and device, electronic equipment and computer readable storage medium, and relate to the technical field of optical microscopic imaging. The method comprises: scanning a sample in at least two directions by line focusing light, collecting signals generated by the sample each time excited by the line focusing light to generate scanning images in the at least two directions respectively, and the at least two directions are symmetrically distributed; performing pixel redistribution on the scanning images in the at least two directions respectively to obtain scanning images after pixel redistribution; and performing image fusion on the scanning images in the at least two directions after pixel redistribution to obtain a fused image as an imaging result of the sample. Embodiments of the present application solve the problems of slow imaging speed, poor imaging effect and high imaging cost in related microscopic imaging technology.

Description

Technical Field

[0001] This application relates to the field of optical microscopy imaging technology, and more specifically, to a scanning imaging method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Confocal microscopy is an important imaging technique in fields such as life sciences, semiconductor detection, and materials science. However, to obtain high-resolution imaging results that break the diffraction limit in confocal microscopy, the size of the spatial pinhole must be extremely small. However, an excessively small pinhole can lead to signal loss at the focal plane and a decrease in the image signal-to-noise ratio. Therefore, achieving high resolution that breaks the diffraction limit in confocal microscopy is virtually impossible.

[0003] Scanning imaging microscopy is a type of microscope that combines confocal microscopy with pixel reassignment. It can achieve the best resolution of confocal microscopy without sacrificing photon efficiency. Unlike confocal microscopy, which uses point detectors, scanning imaging microscopy uses array detectors to record the spatial distribution of signals at each scanning position. Furthermore, it redistributes and combines the signals at each focal point through pixel reassignment, ultimately achieving high resolution that breaks the diffraction limit with high photon efficiency.

[0004] However, related scanning imaging microscopy techniques suffer from problems such as slow imaging speed, poor imaging quality, and high implementation costs. Summary of the Invention

[0005] This application provides a scanning imaging method, apparatus, electronic device, computer-readable storage medium, and computer program product, which solves the problems of slow imaging speed, high complexity, and high imaging cost in microscopic imaging technology.

[0006] According to a first aspect of the embodiments of this application, a scanning imaging method is provided, the method comprising:

[0007] The sample is scanned in two directions by a line-focused light, and the signal generated by the sample each time it is excited by the line-focused light is collected to generate scan images in two directions, which are symmetrically distributed.

[0008] Pixels are redistributed in at least two directions of the scanned image to obtain the scanned image after pixel redistribution;

[0009] Image fusion is performed on the scanned image after pixel redistribution in at least two directions, and the fused image is used as the imaging result of the sample.

[0010] In one possible implementation, the scanning module of the scanning imaging device includes:

[0011] The determining unit is used to determine the direction of the line-focused light;

[0012] A control unit is configured to control the sample to pass through the line-focused light in the focal plane in at least two orientations, in a direction perpendicular to the direction of the line-focused light; the at least two orientations are perpendicular to each other.

[0013] In another possible implementation, the fusion module of the scanning imaging device includes:

[0014] The decomposition unit is used to perform wavelet decomposition on the scanned images after pixel redistribution in the at least two directions to obtain high-frequency information of the scanned images in the at least two directions.

[0015] The fusion unit is used to perform image fusion on the scanned images in at least two directions based on the high-frequency information of the scanned images in the two directions.

[0016] In yet another possible implementation, the scanning imaging fusion module further includes:

[0017] A matrix is ​​determined for each of the scanned images after pixel redistribution in at least two directions, and each element value in the matrix is ​​used to characterize the pixel value of the corresponding pixel in the scanned image after pixel redistribution.

[0018] The matrix calculation unit is used to take the mean of two matrices as the matrix of the fused image in the first iteration process;

[0019] An iterative unit is used to obtain the matrix of the target fused image in at least two directions in each iteration process, based on the matrix of the fused image in the current iteration process, the matrix corresponding to the scanned image after pixel redistribution in at least two directions, and the point spread function of the scanned image in at least two directions.

[0020] The result determination unit is used to determine the image corresponding to the mean result of the matrix of the target fusion image in at least two directions in the current iteration process, if the resolution in the at least two directions is consistent with the scanned image after pixel redistribution in the at least two directions, and the image corresponding to the mean result of the matrix of the target fusion image in the at least two directions is used as the imaging result of the sample.

[0021] If the resolution of the image corresponding to the mean result of the matrix of the target fusion image in the at least two directions is inconsistent with the resolution of the scanned image after pixel redistribution in the at least two directions, the mean result of the matrix of the target fusion image in the at least two directions of the current iteration process shall be used as the matrix of the fusion image to participate in the next iteration process.

[0022] In yet another possible implementation, the scanning module of the scanning imaging device includes:

[0023] An imaging unit is used to collect the signal generated by the sample by the line-focusing light through the objective lens and image it onto the array detector to obtain scan images of the sample in at least two directions.

[0024] The array detector includes a linear array camera or an area array camera.

[0025] In another possible implementation, the line-focused light is obtained by passing a laser beam through a preset lens group; the preset lens group includes at least: a beam expander lens group, a cylindrical lens, a relay lens, and an objective lens.

[0026] According to a second aspect of the embodiments of this application, a scanning imaging apparatus is provided, the apparatus comprising:

[0027] The scanning module is used to scan a sample in two directions using a line-focused light, and to acquire the signal generated by the sample each time it is excited by the line-focused light to generate scan images in at least two directions, wherein the at least two directions are perpendicular to each other.

[0028] The pixel redistribution module is used to redistribute pixels in the scanned images in at least two directions to obtain the pixel-redistributed scanned images.

[0029] The fusion module is used to perform image fusion on the scanned image after pixel redistribution in at least two directions, and use the fused image as the imaging result of the sample.

[0030] According to a third aspect of the present application, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, wherein the processor executes the program to implement the steps of the scanning imaging method provided in the first aspect.

[0031] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the scanning imaging method provided in the first aspect.

[0032] According to a fifth aspect of the present application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, wherein when a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, causing the computer device to perform steps implementing the scanning imaging method as provided in the first aspect.

[0033] The beneficial effects of the technical solutions provided in this application are:

[0034] In the scanning imaging method provided in this application embodiment, line scanning of the sample is performed using line-focused light, reducing the time required for scanning the sample. Simultaneously, continuous image acquisition can be achieved at a higher speed, improving the system's imaging speed. Pixel redistribution is performed on the scanned images in at least two directions, achieving resolution improvement in at least two scanning directions. Image fusion of at least two images with improved resolution in their respective scanning directions further enhances the resolution, resulting in a clearer scanning imaging result. This solves the problems of slow scanning imaging speed, poor imaging effect, high complexity, and high cost in related technologies. It significantly improves the resolution of the images obtained from scanning imaging while ensuring scanning imaging efficiency and cost-effectiveness. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0036] Figure 1 This is a schematic diagram of the system architecture for implementing the scanning imaging method provided in an embodiment of this application;

[0037] Figure 2 A schematic flowchart of a scanning imaging method provided in an embodiment of this application;

[0038] Figure 3 This application provides a schematic diagram of the optical path for a scanning imaging process.

[0039] Figure 4a This is a schematic diagram illustrating the process of pixel redistribution in a scanned image in the X direction, as provided in an embodiment of this application.

[0040] Figure 4b This is a schematic diagram illustrating pixel redistribution of a scanned image in the X direction, as provided in an embodiment of this application.

[0041] Figure 5a This is a schematic diagram illustrating the process of pixel redistribution in a scanned image in the Y direction, as provided in an embodiment of this application.

[0042] Figure 5bThis is a schematic diagram illustrating pixel redistribution of a scanned image in the Y direction, provided as an embodiment of this application.

[0043] Figure 6 This is a schematic diagram illustrating an image fusion process for scanning images using iterative deconvolution, provided as an embodiment of this application.

[0044] Figure 7 A schematic flowchart illustrating a scanning imaging method applied to biological cell scanning imaging, provided in an embodiment of this application;

[0045] Figure 8 This is a schematic diagram of the structure of a scanning imaging device provided in an embodiment of this application;

[0046] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0048] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0050] First, let's introduce and explain several terms used in this application:

[0051] Laser: Light emitted by stimulated emission of atoms, hence the name "laser." Laser light occurs when electrons in an atom absorb energy, jump from a lower energy level to a higher energy level, and then fall back down, releasing energy in the form of photons. The resulting beam of stimulated photons (laser) exhibits highly consistent optical properties. Therefore, compared to ordinary light sources, lasers have better monochromaticity, directionality, and brightness.

[0052] Linear array detectors primarily convert X-rays into visible light before displaying them on a computer. The detector is one of the key components in a radiation imaging system; its performance is closely related to the final specifications of the imaging system. Therefore, detector design is a crucial aspect of imaging system design.

[0053] Focal plane: also known as the front focal plane or object focal plane, in optical terminology, that is, the plane that is perpendicular to the principal optical axis of the system is called the first focal plane.

[0054] Super-resolution: This refers to improving the resolution of an original image through hardware or software methods. The process of obtaining a high-resolution image from a series of low-resolution images is called super-resolution reconstruction. The core idea of ​​super-resolution reconstruction is to trade temporal bandwidth (acquiring multiple frames of the same scene) for spatial resolution, realizing the conversion from temporal resolution to spatial resolution.

[0055] Deconvolution: Deconvolution is a mathematical method that reverses the blurring process of an image by estimating the optical system. In short, it attempts to find an original image that, after passing through the optical system, is as similar as possible to the actually observed image. It is typically performed in the digital domain using software algorithms as part of a suite of microscopy image processing techniques.

[0056] The common approach assumes the optical path through the instrument is optically ideal, and the imaging process can be described as: convolving the imaging object with the point spread function (PSF), where convolution is a mathematical operation describing the superposition effect of the two. If the PSF can be determined, its inverse or complementary function is calculated, and the acquired image is deconvolved with it. The result is the original, undistorted image. It's important to note that in actual imaging, image reconstruction via deconvolution is affected by noise, etc. If the image signal is weak and noise is high, the reconstructed image from deconvolution is prone to distortion.

[0057] Image fusion refers to the process of combining image data of the same target acquired from multiple sources using image processing and computer technology. This process maximizes the extraction of useful information from each source and ultimately synthesizes the combined image into a high-quality image. This aims to improve the utilization of image information, enhance the accuracy and reliability of computer interpretation, increase the spatial resolution of the original image, and fuse multi-scale and multi-type information.

[0058] Point Spread Function (PSF): The point spread function (PSF) is a mathematical function that represents the response of an optical system to a point light source. In an optical system, when a point light source passes through a lens or other optical element, it forms a blurred spot on the image plane. The PSF describes the shape and distribution of this spot. By performing a mathematical convolution operation with an actually observed image, we can simulate the blurring effect of an optical system on an image—imagine a blurred spot formed on the image by a point light source after passing through the optical system. The shape and size of the PSF have a significant impact on image sharpness and the visibility of details.

[0059] Line scan cameras: Typical applications of line scan cameras are the inspection of continuous materials, such as metals, plastics, paper, and fibers. The object being inspected typically moves at a constant speed, and one or more cameras are used to continuously scan it line by line to achieve uniform inspection of its entire surface. The images can be processed line by line or as a multi-line area scan image.

[0060] Area scan camera: Area scan cameras mainly use continuous, area scanning light to detect products. They can acquire a complete target image at once and can perform image acquisition in a timely manner.

[0061] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0062] Figure 1 This is a schematic diagram of a system architecture for implementing scanning imaging provided in an embodiment of this application. The system architecture includes a terminal 120 and a server 140.

[0063] Terminal 120 installs and runs an application that supports scanning imaging. Terminal 120 is used to scan samples using line-focused light. Terminal 120 is also used to acquire signals generated by the samples through an array detector and generate scanned images. Terminal 120 is also used to perform pixel reallocation on the scanned images and image fusion on the scanned images.

[0064] Terminal 120 is connected to server 140 via a wireless network or a wired network.

[0065] Server 140 includes at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Illustratively, server 140 includes a processor 144 and a memory 142, the memory 142 including a display module 1421, a control module 1422, and a receiving module 1423. Server 140 provides background services for applications supporting scanning imaging. Optionally, server 140 performs the primary computing task, and terminal 120 performs secondary computing tasks; or, server 140 performs secondary computing tasks, and terminal 120 performs primary computing tasks; or, server 140 and terminal 120 collaborate in a distributed computing architecture.

[0066] Optionally, the device type of the terminal includes at least one of the following: POS terminal, smartphone, tablet computer, e-book reader, Moving Picture Experts Group Audio Layer III (MP3) player, Moving Picture Experts Group Audio Layer IV (MP4) player, laptop computer, and desktop computer.

[0067] Those skilled in the art will understand that the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.

[0068] In related technologies, one scanning imaging microscope employs a combination of point-by-point scanning and a single-photon avalanche diode array or an array camera to acquire the spatial distribution of each scanning electrical signal, followed by pixel redistribution. However, point-by-point scanning suffers from low parallelism and slow imaging speed, while single-photon avalanche diode arrays have high combination complexity, and array cameras have low area utilization, resulting in a low overall image quality. Another approach utilizes spatial light modulation devices or rotating lens arrays to simultaneously generate multiple focal points for multi-point synchronous excitation. This is combined with array camera detection to record the spatial information of images under multiple focal excitations in parallel, followed by pixel redistribution in optical or digital form. While parallel excitation of multiple focal points improves imaging speed, enabling faster image acquisition within a single field of view, large-scale imaging (beyond a single field of view) requires stitching together different fields of view, which still limits imaging speed and increases the complexity of coupling different fields of view. Furthermore, generating multi-focal excitation involves high optical path complexity and requires high coupling precision from optical devices, resulting in high overall complexity and implementation cost.

[0069] To overcome the aforementioned problems of the prior art, this application provides a scanning imaging method applied to a terminal, which may be one of the above-mentioned methods. Figure 1 Terminal 120 in the middle, such as Figure 2 As shown, the method includes:

[0070] S101, the sample is scanned in at least two directions by line-focused light, and the signal generated by the sample each time it is excited by the line-focused light is collected to generate scan images in at least two directions, which are symmetrically distributed in at least two directions.

[0071] In the embodiments of this application, the sample is scanned from at least two directions, and the scanning directions are symmetrically distributed. The symmetry can be mirror symmetry, axial symmetry, and rotational symmetry.

[0072] In one example, the control line focused laser scans the sample from two directions, with an angle of 90 degrees between adjacent directional vectors. The two directions are the X and Y directions, i.e., the horizontal axis and the vertical axis.

[0073] In another example, the control line focused excitation light scans the sample from three directions, with the angle between adjacent directional vectors being 60 degrees; the control line focused excitation light scans the sample from four directions, with the angle between adjacent directional vectors being 45 degrees.

[0074] In the embodiments of this application, the samples include, but are not limited to: biological samples, such as bacteria, cells, cell clusters and cell tissues, non-biological samples and organic matter, etc.

[0075] In this embodiment, the line-focused light is obtained by passing the excitation light through a preset lens group; the preset lens group includes at least: a beam expander lens group, a cylindrical lens, a relay lens, and an objective lens.

[0076] In this embodiment, the signal generated by the sample being excited by the focused light is a photoelectric signal. After the photoelectric signal is collected by the array detector, it is converted into a digital signal for storage and processing, thereby obtaining a scanned image.

[0077] In the embodiments of this application, the excitation light can be a laser or an LED light. It should be noted that the excitation light used in the scanning imaging process is determined according to the application scenario and actual situation. This application does not specifically limit the excitation light.

[0078] Please refer to Figure 3The figure exemplifies the optical path diagram of the scanning process in this embodiment. As shown, the excitation light is expanded by the beam-expanding lens group 301 and then compressed into a linear distribution, i.e., line-focused light, by the cylindrical lens 302. After passing through the relay lens group 303, the reflector 304, and the objective lens 305, it finally forms illumination on the focal plane. After the line-focused light illuminates the sample on the stage 306, the signal generated by the sample excited by the line-focused light is recorded and imaged by the camera 308 of the array detector after passing through the objective lens 305 and the imaging lens 307. By compressing the laser into line-focused light through the lens group to scan the sample, the optical path is simple and low-cost. Moreover, by using line scanning, i.e., scanning the sample with line-focused light, the problem of low parallelism and slow imaging speed of point-by-point scanning is solved, thereby improving the area utilization of the array camera and enabling continuous image acquisition at a higher speed. This greatly improves the imaging efficiency.

[0079] This application provides a possible implementation method, which involves determining the direction of the line-focused light; controlling the sample to pass through the line-focused light in the focal plane in a direction perpendicular to the direction of the line-focused light, with at least two placement orientations; and the two placement orientations being vertically distributed.

[0080] This application provides a possible implementation method in which the signal generated by the sample being excited by the line-focusing light is collected by the objective lens and imaged onto the array detector to obtain scan images of the sample in at least two directions.

[0081] In one example, the signal generated by the laser excitation of the sample is collected by the objective lens and imaged onto the array detector, which can be a high-speed linear array camera or a planar array camera with only a few windows.

[0082] In one example, keeping the line-focused illumination constant, the sample is placed on a displacement stage. The displacement stage moves the sample in coordination with the camera's fastest line frame, enabling continuous line scanning of the sample by the line-focused light in one direction. The sample is then rotated 90 degrees, keeping the line-focused light and camera window orientation constant. The displacement stage moves the sample in coordination with the camera's fastest line frame, enabling continuous line scanning of the sample by the line-focused light in another direction, thus obtaining scanned images in both directions.

[0083] In one example, the camera pixel size is 6.5μm, the fastest line frame is 22kHz (8-line window), and the linear movement speed of the stage is set to 143mm / s to achieve high-speed linear scanning of the sample. Setting the stage speed based on the camera's fastest line frame ensures maximum utilization of the array camera during scanning, enabling faster image acquisition within a single field of view and significantly improving imaging efficiency.

[0084] S102, perform pixel redistribution on the scanned images in at least two directions respectively, and obtain the scanned images after pixel redistribution.

[0085] In the embodiments of this application, after the scanning imaging microscope records the spatial distribution of the signal at each scanning position of the sample using an array detector, that is, after generating the scanning image, the signal at each scanning position is redistributed and combined in a pixel redistribution manner.

[0086] In this embodiment of the application, the pixels of the scanned images in at least two directions are redistributed according to the redistribution factor m. When m is 0.5, the scanned images can achieve the best resolution improvement in at least two directions.

[0087] In one example, the signal generated during line scanning of the sample, at each scan position r s The pixels at each location are redistributed to the detection plane using an allocation factor m, which means the pixels of each location are redistributed. The corresponding image formation process is represented by the following formula:

[0088]

[0089] Where o(r) represents the sample located at spatial coordinate r, and h ex (r′-r s ) indicates that it is located at r s The distribution of the scanning light, its relative position to the sample located at r′ in space is r′-r s . To obtain the signal distribution for scanning imaging. a r represents the location of the signal in the image. a -r s This represents the change in signal position relative to the scanning light position, where m corresponds to the allocation factor; the detected signal is redistributed into the image r. a =mr+(1-m)r s The above formula can also be simplified to the following:

[0090] I(r a )=o*[h ex *h em ]

[0091] It can be seen that the imaging result of sample o is determined by the focused scanning beam h. ex and imaging signal h em Decision: Scan beam h ex and imaging signal h em It can be represented by a Gaussian function, as follows:

[0092]

[0093]

[0094] Where σ corresponds to the standard deviation of the Gaussian function distribution. The standard deviation σ of the scanned signal. ex The standard deviation σ of the detected signal is determined by the optical system NA and the scanning light wavelength. em The point spread function of the system can be further expressed as follows, determined by the optical system NA and the wavelength of the probe signal light:

[0095]

[0096] Therefore, the system corresponds to the standard deviation σ PR From σ em Together with m, we decided:

[0097]

[0098] The formula derivation shows that when m takes the following values:

[0099]

[0100] σ PR The minimum value can be obtained, which corresponds to the system having the highest resolution.

[0101]

[0102] Since both the scanning light and the probe signal are determined by the system NA, and in a scenario where the scanning wavelength and the probe signal wavelength are close, i.e. σ ex = em Calculations show that the system achieves optimal resolution when m = 0.5, corresponding to... In related fluorescence microscopy imaging, the optimal resolution of the system is directly determined by the detection signal σ = σ em The decision is made. Comparison shows that, through the process of redistributing the probe signal using an allocation factor m (also known as pixel redistribution), the resolution of the scanning imaging microscope surpasses that of traditional optical systems; the resolution is significantly improved. This represents a several-fold improvement, achieving super-resolution.

[0103] Please refer to Figure 4a The figure exemplifies a schematic diagram of the X-direction scanned image before pixel redistribution, as shown in the embodiment of this application. Before pixel redistribution, the image resolution is low and the blurriness is high. Because there is a positional deviation between the signal distribution in the scanned image and the actual scanning light during the scanning imaging process, it is necessary to redistribute each pixel of the scanned image. Please refer to... Figure 4bThe example illustrates a schematic diagram of the X-direction scan image after pixel redistribution in an embodiment of this application. Based on the redistribution factor m (m is optimally 0.5), the pixel redistribution of the X-direction scan image improves the resolution of the X-direction scan image, thereby obtaining a scan image with super-resolution information in the X-direction.

[0104] Please refer to Figure 5a The figure exemplifies a schematic diagram of the scanned image in the Y direction before pixel redistribution according to an embodiment of this application. As shown, before pixel redistribution of the scanned image in the Y direction, the image resolution is low and the blurriness is high. Because there is a positional deviation between the signal distribution in the scanned image and the actual scanning light during the scanning imaging process, it is necessary to redistribute each pixel of the scanned image. Please refer to... Figure 5b The example illustrates a schematic diagram of the scanning image in the Y direction after pixel redistribution according to an embodiment of this application. Based on the redistribution factor m (m is optimally 0.5), pixel redistribution of the scanning image in the Y direction improves the resolution of the scanning image in the Y direction, thereby obtaining a scanning image with super-resolution information in the Y direction.

[0105] By redistributing pixels along the scanning direction of the scanned image, resolution improvements can be achieved in at least two scanning directions, significantly enhancing the resolution of the scanned imaging results.

[0106] S103, perform image fusion on the scanned image after pixel redistribution in at least two directions, and use the fused image as the imaging result of the sample.

[0107] In the embodiments of this application, image fusion can be performed on scanned images after pixel redistribution in at least two directions by averaging in the spatial domain or frequency domain.

[0108] This application provides a possible implementation method, which involves performing wavelet decomposition on the scanned images after pixel redistribution in at least two directions to obtain high-frequency information of the scanned images in at least two directions; and performing image fusion on the scanned images in at least two directions based on the high-frequency information of the scanned images in at least two directions.

[0109] In one example, the X-direction and Y-direction scan images after pixel redistribution possess super-resolution information in the X and Y directions, respectively. Wavelet decomposition is performed on the X-direction and Y-direction scan images after pixel redistribution to obtain the high-frequency information LHx in the X-direction of the X-direction scan image after pixel redistribution and the high-frequency information HLy in the Y-direction scan image after pixel redistribution. The super-resolution information is contained in the decomposed high-frequency information. Based on the obtained high-frequency information LHx in the X-direction and high-frequency information HLy in the Y-direction, inverse wavelet decomposition is performed to obtain a fused image that integrates the high-frequency information LHx in the X-direction and the high-frequency information HLy in the Y-direction, thereby obtaining an image with super-resolution information in both the X and Y directions.

[0110] This application embodiment also provides a possible implementation method, which involves determining the matrices corresponding to the scanned images after pixel redistribution in at least two directions, where each element value in the matrix represents the pixel value of the corresponding pixel in the scanned image after pixel redistribution; using the mean of the at least two matrices as the matrix of the fused image in the first iteration process; in each iteration process, based on the matrix of the fused image in this iteration process, the matrices corresponding to the scanned images after pixel redistribution in at least two directions, and the point spread function of the scanned images in at least two directions, obtaining the matrix of the target fused image in at least two directions for this iteration process; if the current iteration is determined... If the image corresponding to the mean result of the matrix of the target fusion image in at least two directions of the iteration process has the same resolution in at least two directions as the scanned image after pixel redistribution in at least two directions, the image corresponding to the mean result of the matrix of the target fusion image in at least two directions is used as the imaging result of the sample; if it is determined that the image corresponding to the mean result of the matrix of the target fusion image in at least two directions has the same resolution in at least two directions as the scanned image after pixel redistribution in at least two directions, the mean result of the matrix of the target fusion image in at least two directions of the current iteration process is used as the matrix of the fusion image participating in the next iteration process.

[0111] In one example, the matrices corresponding to the X and Y scan images after pixel redistribution are imgx and imgy, respectively. Image fusion is performed on the X and Y scan images after pixel redistribution using iterative deconvolution. For the specific iterative process, please refer to [link to relevant documentation]. Figure 6 As shown.

[0112] S201, the mean of the matrices imgx and imgy corresponding to the scanned images in the X and Y directions after pixel redistribution is used as the matrix E0 of the fused image in the first iteration process, as shown in the following formula:

[0113]

[0114] S202, through iterative deconvolution operations, in each iteration, the matrix E of the fused image is... k (k = 0, 1, 2, ..., N, k is the iteration number) Combined with the matrix imgx corresponding to the scanned image and the point spread function PSF of the scanned image in the X direction. x and the transpose of the point spread function The matrix E of the scanned image in the X direction after deconvolution is obtained. x The matrix E of the fused images k Combining the matrix imgy corresponding to the scanned image in the Y direction after pixel redistribution, and the point spread function PSF of the scanned image in the Y direction. y and the transpose of the point spread function The matrix E of the scanned image in the Y direction after deconvolution is obtained. y The matrix E of the scanned image in the X direction x The matrix E of the scanned image in the Y direction. y The mean result is used as the matrix E of the newly generated fused image. k+1 The formula for the above operation is as follows:

[0115]

[0116]

[0117]

[0118] S203, determine whether the resolution of the newly generated fused image in the X direction is consistent with the X-direction scan image after pixel redistribution, and whether the resolution of the newly generated fused image in the Y direction is consistent with the Y-direction scan image after pixel redistribution. If they are inconsistent, return to S202 and output the matrix E. k+1 This matrix serves as the fused image for the next iteration.

[0119] S204, under the condition of consistency, output matrix E k+1 The corresponding image is used as the final fused image to obtain the final imaging result of the sample.

[0120] By using joint deconvolution, the resolvable distance between two adjacent points in the scanned image is further shortened. Through multiple iterations, the resolution of the scanned image can be improved to varying degrees, ultimately resulting in a super-resolution scanned imaging result.

[0121] Please refer to Figure 7 The figure illustrates an exemplary flowchart of the scanning imaging method provided in this application for biological cell scanning imaging. The specific process is as follows:

[0122] S301: The 491nm laser source is turned on. After passing through the beam expander lens group, the laser is compressed into a linear distribution by the cylindrical lens, and then passes through the relay lens and objective lens to form a linear illumination, i.e., linear focused light. The linear focused light is kept constant while scanning biological cells. During the scanning process, the displacement stage for placing biological cells is moved in the X or Y direction. The array detector records the signal excited by the sample after being scanned by the linear focused light. Based on the collected signal, the scanning images of biological cells in the X and Y directions are generated.

[0123] S302, based on the redistribution factor, the pixels of the scanned image of the biological cell in the X direction are redistributed to obtain a scanned image with super-resolution information in the X direction, and the pixels of the scanned image of the biological cell in the Y direction are redistributed to obtain a scanned image with super-resolution information in the Y direction.

[0124] S303 performs image fusion processing on the scanned image with super-resolution information in the X direction and the scanned image with super-resolution information in the Y direction to obtain the scanning imaging results of biological cells.

[0125] The scanning method provided in this application reduces the time required for scanning samples by compressing laser light into line-focused light using a laser lens group. This achieves faster image acquisition within a single field of view, improves the utilization rate of the area scan camera, increases the imaging speed, and significantly improves the efficiency of scanning imaging. Based on a redistribution factor, pixels are redistributed in at least two directions of the scanned image, improving the resolution of the scanned image in each of those directions, resulting in scanned images with super-resolution information in at least two directions. Joint deconvolution further shortens the resolvable distance between adjacent points in the scanned image. Multiple iterations can improve the resolution of the scanned image to varying degrees, ultimately yielding a super-resolution scanning imaging result. This further improves the resolution of the scanning imaging result, solving the problems of slow scanning imaging speed, poor imaging effect, high complexity, and high cost in related technologies. It significantly improves the resolution of the images obtained from scanning imaging while maintaining scanning imaging efficiency and cost.

[0126] This application provides a scanning imaging device, such as... Figure 8 As shown, the scanning imaging device 80 may include: a scanning module 801, a redistribution module 802, and a fusion module 803, wherein,

[0127] The scanning module 801 is used to scan the sample in two directions using a line-focused light, and to collect the signal generated by the sample each time it is excited by the line-focused light to generate scan images in at least two directions, the two directions being perpendicular to each other;

[0128] The redistribution module 802 is used to redistribute pixels in scanned images in at least two directions respectively, and obtain the scanned images after pixel redistribution.

[0129] The fusion module 803 is used to perform image fusion on the scanned image after pixel redistribution in at least two directions, and use the fused image as the imaging result of the sample.

[0130] The apparatus in this application embodiment can execute the scanning imaging method provided in this application embodiment. The implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0131] Furthermore, in one possible implementation, the scanning module of the scanning imaging device includes:

[0132] The determining unit is used to determine the direction of the line-focused light;

[0133] A control unit is used to control the samples to pass through the line-focused light in the focal plane in at least two orientations, in a direction perpendicular to the direction of the line-focused light; the at least two orientations are symmetrically distributed.

[0134] In another possible implementation, the fusion module of the scanning imaging device includes:

[0135] The decomposition unit is used to perform wavelet decomposition on the scanned image after pixel redistribution in at least two directions to obtain high-frequency information of the scanned image in at least two directions.

[0136] The fusion unit is used to perform image fusion on scanned images in at least two directions based on high-frequency information from scanned images in at least two directions.

[0137] In yet another possible implementation, the fusion module for scanning imaging also includes:

[0138] Determine the matrices corresponding to the scanned images after pixel redistribution in two directions respectively. Each element value in the matrix is ​​used to represent the pixel value of the corresponding pixel in the scanned image after pixel redistribution.

[0139] A matrix calculation unit is used to take the mean of at least two matrices as the matrix of the fused image in the first iteration process;

[0140] The iterative unit is used to obtain the matrix of the target fused image in at least two directions in each iteration process, based on the matrix of the fused image in this iteration process, the matrix of the scanned image after pixel redistribution in at least two directions, and the point spread function of the scanned image in at least two directions.

[0141] The result determination unit is used to determine the image corresponding to the mean result of the matrix of the target fusion image in at least two directions in the current iteration process, which has the same resolution in at least two directions as the scanned image after pixel redistribution in at least two directions, and to take the image corresponding to the mean result of the matrix of the target fusion image in at least two directions as the imaging result of the sample.

[0142] If the resolution of the image corresponding to the mean result of the matrix of the target fusion image in at least two directions is inconsistent with the resolution of the scanned image after pixel redistribution in at least two directions, the mean result of the matrix of the target fusion image in at least two directions in this iteration process will be used as the matrix of the fusion image to participate in the next iteration process.

[0143] In yet another possible implementation, the scanning module of the scanning imaging device includes:

[0144] The imaging unit is used to collect the signal generated by the line-focusing light excitation of the sample through the objective lens and image it onto the array detector to obtain scan images of the sample in at least two directions.

[0145] The array detector includes linear array cameras or area array cameras.

[0146] In another possible implementation, the line-focused light is obtained by passing the laser through a preset lens group; the preset lens group includes at least: a beam expander lens group, a cylindrical lens, a relay lens, and an objective lens.

[0147] This application provides an electronic device (computer device / equipment / system) including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a scanning imaging method. Compared with related technologies, the scanning method provided in this application reduces the time required to scan the sample by compressing the laser into a line-focused beam using a laser lens group, achieving faster image acquisition within a single field of view, improving the utilization rate of the area scan camera, increasing the imaging speed of scanning imaging, and greatly improving the efficiency of scanning imaging. Based on a redistribution factor, pixels are redistributed in at least two directions of the scanned image, improving the resolution of the scanned image in each of the at least two directions, resulting in scanned images with super-resolution information in at least two directions. Joint deconvolution further shortens the resolvable distance between adjacent points in the scanned image. Through multiple iterations, the resolution of the scanned image can be improved to varying degrees, ultimately obtaining a scanning imaging result with super-resolution. This further improves the resolution of the scanning imaging result, solving the problems of slow scanning imaging speed, poor imaging effect, high complexity, and high cost in related technologies. It significantly improves the resolution of the images obtained from scanning imaging while ensuring scanning imaging efficiency and cost.

[0148] In one alternative embodiment, an electronic device is provided, such as Figure 9 As shown, Figure 9 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0149] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0150] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0151] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0152] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0153] The electronic device package may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0154] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, the scanning method provided in this application reduces the time required to scan samples by compressing the laser into a line-focused beam using a laser lens group, achieving faster image acquisition within a single field of view, improving the utilization rate of the area array camera, increasing the imaging speed of scanning imaging, and greatly improving the efficiency of scanning imaging. Based on a redistribution factor, pixels are redistributed in at least two directions of the scanned images, improving the resolution of the scanned images in each of the at least two directions, resulting in scanned images with super-resolution information in at least two directions. Joint deconvolution further shortens the resolvable distance between adjacent points in the scanned image, and multiple iterations can improve the resolution of the scanned image to varying degrees, ultimately obtaining a super-resolution scanning imaging result. This further improves the resolution of the scanning imaging result, solving the problems of slow scanning imaging speed, poor imaging effect, high complexity, and high cost in related technologies. It significantly improves the resolution of the images obtained from scanning imaging while ensuring scanning imaging efficiency and cost.

[0155] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0156] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve:

[0157] The scanning method provided in this application reduces the time required for scanning samples by compressing laser light into line-focused light using a laser lens group. This achieves faster image acquisition within a single field of view, improves the utilization rate of the area scan camera, increases the imaging speed, and significantly improves the efficiency of scanning imaging. Based on a redistribution factor, pixels are redistributed in at least two directions of the scanned image, improving the resolution of the scanned image in each of those directions, resulting in scanned images with super-resolution information in at least two directions. Joint deconvolution further shortens the resolvable distance between adjacent points in the scanned image. Multiple iterations can improve the resolution of the scanned image to varying degrees, ultimately yielding a super-resolution scanning imaging result. This further improves the resolution of the scanning imaging result, solving the problems of slow scanning imaging speed, poor imaging effect, high complexity, and high cost in related technologies. It significantly improves the resolution of the images obtained from scanning imaging while maintaining scanning imaging efficiency and cost.

[0158] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0159] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0160] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A scanning imaging method, characterized in that, include: The sample is scanned in at least two directions by a line-focused light, and the signal generated by the sample each time it is excited by the line-focused light is collected to generate scan images in at least two directions, wherein the at least two directions are symmetrically distributed. Pixels are redistributed in the scanned images of at least two directions respectively to obtain the scanned images after pixel redistribution; Image fusion is performed on the scanned image after pixel redistribution in at least two directions, and the fused image is used as the imaging result of the sample; The image fusion of the scanned image after pixel redistribution in at least two directions includes: Wavelet decomposition is performed on the scanned images after pixel redistribution in at least two directions to obtain the high-frequency information of the scanned images in at least two directions; Based on the high-frequency information of the scanned images in at least two directions, image fusion is performed on the scanned images in at least two directions; The step of performing image fusion on the scanned images after pixel redistribution in at least two directions, and using the fused image as the imaging result of the sample, includes: A matrix is ​​determined for each of the scanned images after pixel redistribution in at least two directions, and each element value in the matrix is ​​used to characterize the pixel value of the corresponding pixel in the scanned image after pixel redistribution. The mean of at least two matrices is used as the matrix of the fused image in the first iteration. In each iteration, the matrix of the target fused image in at least two directions is obtained based on the matrix of the fused image in this iteration, the matrix of the scanned image after pixel redistribution in at least two directions, and the point spread function of the scanned image in at least two directions. If it is determined that the image corresponding to the mean result of the matrix of the target fusion image in at least two directions in the current iteration process is consistent with the resolution of the scanned image after pixel redistribution in the at least two directions, the image corresponding to the mean result of the matrix of the target fusion image in the at least two directions is taken as the imaging result of the sample. If the resolution of the image corresponding to the mean result of the matrix of the target fusion image in the at least two directions is inconsistent with the resolution of the scanned image after pixel redistribution in the at least two directions, the mean result of the matrix of the target fusion image in the at least two directions of the current iteration process shall be used as the matrix of the fusion image to participate in the next iteration process.

2. The scanning imaging method according to claim 1, characterized in that, The scanning of the sample in at least two directions using line-focused light includes: Determine the direction of the focused line light; The sample is controlled to pass through the line-focused light in the focal plane in at least two different orientations, in a direction perpendicular to the direction of the line-focused light; the at least two orientations are symmetrical to each other.

3. The scanning imaging method according to claim 1, characterized in that, The step of acquiring the signal generated by the sample each time it is excited by the focused light to generate scan images in at least two directions includes: The signal generated by the sample being excited by the line-focused light is collected by the objective lens and imaged onto the array detector to obtain scan images of the sample in at least two directions. The array detector includes a linear array camera or an area array camera.

4. The scanning imaging method according to any one of claims 1 to 3, characterized in that, The line-focused light is obtained by passing a laser beam through a preset lens group; The preset lens group includes at least: a beam expander lens group, a cylindrical lens, a relay lens, and an objective lens.

5. A scanning imaging device, characterized in that, include: A scanning module is used to scan a sample in at least two directions using a line-focused light, and to acquire the signal generated by the sample each time it is excited by the line-focused light to generate scan images in at least two directions, wherein the at least two directions are symmetrically distributed. The pixel redistribution module is used to redistribute pixels in the scanned images of the at least two directions respectively, and obtain the scanned images after pixel redistribution. The fusion module is used to perform image fusion on the scanned image after pixel redistribution in at least two directions, and use the fused image as the imaging result of the sample; The image fusion of the scanned image after pixel redistribution in at least two directions includes: Wavelet decomposition is performed on the scanned images after pixel redistribution in at least two directions to obtain the high-frequency information of the scanned images in at least two directions; Based on the high-frequency information of the scanned images in at least two directions, image fusion is performed on the scanned images in at least two directions; The step of performing image fusion on the scanned images after pixel redistribution in at least two directions, and using the fused image as the imaging result of the sample, includes: A matrix is ​​determined for each of the scanned images after pixel redistribution in at least two directions, and each element value in the matrix is ​​used to characterize the pixel value of the corresponding pixel in the scanned image after pixel redistribution. The mean of at least two matrices is used as the matrix of the fused image in the first iteration. In each iteration, the matrix of the target fused image in at least two directions is obtained based on the matrix of the fused image in this iteration, the matrix of the scanned image after pixel redistribution in at least two directions, and the point spread function of the scanned image in at least two directions. If it is determined that the image corresponding to the mean result of the matrix of the target fusion image in at least two directions in the current iteration process is consistent with the resolution of the scanned image after pixel redistribution in the at least two directions, the image corresponding to the mean result of the matrix of the target fusion image in the at least two directions is taken as the imaging result of the sample. If the resolution of the image corresponding to the mean result of the matrix of the target fusion image in the at least two directions is inconsistent with the resolution of the scanned image after pixel redistribution in the at least two directions, the mean result of the matrix of the target fusion image in the at least two directions of the current iteration process shall be used as the matrix of the fusion image to participate in the next iteration process.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images