Focusing method, device and equipment

Abstract

The invention provides a focusing method, device and equipment. The method comprises the following steps: determining pixel offset between a red channel image and a green channel image of a tissue slice; the red channel image and the green channel image are a red channel image and a green channel image of a tissue slice in a ghosting image captured by the color image sensor when a red light filter, a green light filter and double optical wedges in the focusing module are modulated at the same time; determining an out-of-focus distance according to the pixel offset and a preset corresponding relation; the preset corresponding relation is the corresponding relation between the pixel offset and the defocus distance; and focusing according to the out-of-focus distance. According to the method provided by the embodiment of the invention, the problem that positive and negative defocusing directions cannot be distinguished through a single-frame image in a phase shift detection automatic focusing technology based on image monochromatic double-hole modulation is solved, and the problem of time waste caused by high-speed switching of an illumination light source in an automatic focusing technology based on red-green double-LED illumination is solved; and meanwhile, the problem that a phase shift detection automatic focusing technology based on pupil segmentation images is relatively small in focusing range is solved.

Description

Technical Field

[0001] This invention relates to the field of automatic microscopic imaging technology, and in particular to a focusing method, apparatus, and device. Background Technology

[0002] A tissue slide scanner uses an electric displacement platform to mechanically scan the entire area of ​​a tissue slide and stitches the images together to form a complete digital slide. It is the hardware foundation of modern automated microscopic imaging systems.

[0003] In related technologies, slide scanners use an electrically driven displacement platform to scan samples and rely on autofocus technology to find the correct focus position. Autofocus technology is a crucial factor affecting the scanning efficiency and focusing accuracy of slide scanners, directly influencing the scanning speed and the clarity of the slide image. Therefore, how to perform efficient and accurate autofocus is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] This invention provides a focusing method, apparatus, and device. By acquiring red-green ghost images of tissue sections when the red light filter, green light filter, and dual light wedges in the focusing module are simultaneously modulated, and calculating the pixel offset between the red channel image and the green channel image in the ghost image, the defocus position of the tissue section can be quickly and accurately determined based on the pre-calibrated linear relationship between the pixel offset between the red and green channel images and the defocus distance, thus realizing single-frame real-time automatic focusing of the tissue section scanner.

[0005] This invention provides a focusing method, comprising the following steps.

[0006] Determine the pixel offset between the red channel image and the green channel image of the tissue slice; the red channel image and the green channel image are the red channel image and the green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, the green light filter and the dual light wedge in the focusing module are simultaneously modulated; The defocus distance is determined based on the pixel offset and a preset correspondence; the preset correspondence is the correspondence between pixel offset and defocus distance. Focusing is performed based on the stated defocus distance.

[0007] According to a focusing method provided by the present invention, determining the pixel offset between the red channel image and the green channel image of a tissue slice includes: The pixel offset between the red channel image and the green channel image of the tissue slice is determined based on the cross-correlation registration algorithm.

[0008] According to a focusing method provided by the present invention, determining the defocus distance based on the pixel offset and a preset correspondence includes: Color crosstalk correction is performed on the red channel and green channel images of tissue slices in the ghost images captured by the color image sensor to obtain the corrected red channel image and the corrected green channel image; The defocus distance is determined based on the pixel offset between the corrected red channel image and the corrected green channel image and the preset correspondence.

[0009] According to a focusing method provided by the present invention, the step of performing color crosstalk correction on the red channel image and green channel image of a tissue slice in a ghost image captured by a color image sensor to obtain a corrected red channel image and a corrected green channel image includes: When the red light filter, green light filter and dual light wedge in the focusing module are simultaneously modulated, the first red channel image and the first green channel image of the tissue slice sample in the first ghost image captured by the color image sensor are obtained. Acquire the second red channel image of a tissue slice sample in an image captured by the color sensor when only the red light filter is working; Acquire a second green channel image of a tissue slice sample in an image captured by a color sensor when only the green light filter is working; The color crosstalk coefficient is determined based on the first red channel image, the first green channel image, the second red channel image, and the second green channel image. Based on the color crosstalk coefficient, color crosstalk correction is performed on the red channel image and green channel image of the tissue slice in the ghost image captured by the color image sensor to obtain the corrected red channel image and the corrected green channel image.

[0010] According to a focusing method provided by the present invention, determining the color crosstalk coefficient based on the first red channel image, the first green channel image, the second red channel image, and the second green channel image includes: The color crosstalk coefficient is determined based on the following method: in, and Indicates the color crosstalk coefficient; M and N represent the image size; This represents the first red channel image; This represents the first green channel image; This represents the second red channel image; This represents the second green channel image.

[0011] According to a focusing method provided by the present invention, the method further includes: The preset correspondence is determined based on the pixel offset of the tissue slice sample at different defocus distances.

[0012] The present invention also provides a focusing device, comprising the following modules: The first determining module is used to determine the pixel offset between the red channel image and the green channel image of the tissue slice; the red channel image and the green channel image are the red channel image and the green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, the green light filter and the dual light wedge in the focusing module are simultaneously modulated. The second determining module is used to determine the defocus distance based on the pixel offset and a preset correspondence; the preset correspondence is the correspondence between pixel offset and defocus distance. A focusing module is used to focus based on the defocus distance.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the focusing method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the focusing method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the focusing method as described above.

[0016] The focusing method, apparatus, and device provided by this invention acquire red-green ghost images of tissue sections when the red light filter, green light filter, and dual wedges in the focusing module are simultaneously modulated, and calculate the pixel offset between the red channel image and the green channel image in the ghost image. Then, based on a pre-calibrated linear relationship between the pixel offset between the red and green channel images and the defocus distance, the defocus position of the tissue section can be quickly and accurately determined, realizing real-time autofocus in a single frame using a tissue section scanner. The method of this application overcomes the problem that image-based monochrome dual-aperture modulation phase-shift detection autofocus technology cannot distinguish positive and negative defocus directions from a single frame image, solves the time waste caused by high-speed switching of the illumination source in autofocus technology based on red-green dual-LED illumination, and compensates for the small focusing range of phase-shift detection autofocus technology based on pupil segmentation images. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the focusing method provided by the present invention.

[0019] Figure 2 This is a schematic diagram of the focusing module provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the sample defocus position and pixel offset provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the color crosstalk correction results provided by the present invention.

[0022] Figure 5 This is a schematic diagram of pixel offset and defocus distance provided by the present invention.

[0023] Figure 6 This is the second flowchart of the focusing method provided by the present invention.

[0024] Figure 7 This is a schematic diagram of the focusing device provided by the present invention.

[0025] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] The following is combined with Figures 1-8 The focusing method, apparatus, and device of the present invention are described.

[0028] To facilitate a clearer understanding of the technical solutions of the various embodiments of this application, some technical content related to the various embodiments of this application will be introduced first.

[0029] A slide scanner uses an electrically driven displacement platform to mechanically scan the entire area of ​​a tissue section and stitches the images together to form a single digitized slide. It forms the hardware foundation of modern automated microscopic imaging systems. The surface height of samples to be imaged, such as pathological tissue sections, is often uneven. When traditional optical microscopes acquire images from different fields of view, the shallow depth of field causes the microscope system to defocus, resulting in blurred images. Therefore, repeated fine focusing is required to obtain clear, high-quality images, leading to low efficiency. A slide scanner uses an electrically driven displacement platform to scan the sample and relies on autofocus technology to find the correct focus position. Autofocus technology is a crucial factor affecting the scanning efficiency and focusing accuracy of a slide scanner, directly influencing the scanning speed and the clarity of the slide image. Many research institutions both domestically and internationally are dedicated to the research of autofocus technology. Based on different focusing principles, autofocus technology can be mainly divided into two categories: active focusing technology and passive focusing technology.

[0030] The core idea of ​​active autofocus technology is to convert the defocus amount into a detectable signal change (usually a photoelectric detection signal), thereby establishing a one-to-one correspondence between the defocus amount and the signal. Active autofocus technology mainly relies on the principle of laser reflection for autofocus, and common methods include the eccentric beam method, Foucault knife method, astigmatism method, and critical angle method. However, during image acquisition, factors such as external environmental vibrations, insufficient stability of the system's mechanical structure, and fluctuations in ambient temperature can all cause axial drift in the laser system, leading to defocusing of the imaging system, resulting in image blurring and affecting focusing accuracy and imaging quality. Furthermore, this method is difficult to handle morphological changes in tissue sections on glass slides; therefore, it will only be briefly discussed in this invention. Passive autofocus technology mainly utilizes digital image processing principles to evaluate and analyze defocused images acquired by high-speed CCDs, and is divided into two main categories based on different analysis methods: Depth of Focus (DFF) and Depth of Defocus (DFD). Currently, the most widely used method in the Depth of Focus method is the focus map-based autofocus method. This method acquires image sequences for each pre-selected region along the axial (Z-axis) direction and evaluates the sharpness of each frame using an image quality assessment function to determine the extreme point of the assessment function and obtain the optimal focus position. Although this method performs well in terms of accuracy and robustness and has become the mainstream autofocus technology in commercial slicing scanners, its computational overhead is high, making it unable to meet real-time requirements and resulting in a long time consumption for the whole-slice image acquisition process. The defocus depth method does not require pre-acquiring a focus map and can calculate the defocus displacement of the current scanning field of view in real time. It mainly includes the following six types: dual-camera continuous scanning, beam splitter array, tilting camera, deep learning, dual-LED illumination, and phase shift detection. The dual-camera continuous scanning method continuously acquires three images by adding an additional autofocus imaging optical path to calculate the optimal focus position of the system, supporting autofocus in the continuous scanning mode of the motorized displacement platform; the beam splitter array method also requires an additional autofocus imaging optical path, introducing different optical path differences by splitting the beam, so that the acquired images contain image information of different defocus positions, and the optimal focus position can be determined based on the image contrast. Compared to autofocus technology based on continuous scanning of dual cameras, this method reduces the need to acquire three images to a single image by introducing a beam splitter array, saving acquisition time. The autofocus method based on a tilting camera is similar in principle to the methods based on continuous scanning of dual cameras and beam splitter arrays. The difference is that the horizontally placed camera is replaced by a tilted camera. The focal plane positions of the samples acquired in different spatial regions on the tilting camera are different. Therefore, the optimal focus position can be found by calculating the image contrast of the cross section on the tilting camera.This method only requires acquiring a single image for autofocus, resulting in a simpler optical path and lower hardware costs. Deep learning-based autofocus methods fall into two categories: the first predicts the defocus displacement based on the acquired defocus image, then controls an axial electric displacement platform to move to the focus position and acquires the corresponding focus image. The second directly outputs a virtual focus image based on the input defocus image. This method has simpler hardware requirements but requires further work to improve its robustness and generalization ability to handle different types of samples. The dual-LED illumination-based autofocus method utilizes LED illumination samples at different angles to convert axial defocus displacement into lateral image shift. By pre-calibrating the linear relationship between axial defocus displacement and lateral image shift, the axial defocus position of the current scanning field of view can be calculated in real-time during slice scanning based on the pixel shift of the defocus image, thus achieving autofocus. The advantage of this method is single-frame real-time autofocus, requiring no additional autofocus optical path and only one camera. The drawback is that high-speed switching between red-green LED illumination and bright-field illumination is required during sample scanning, increasing the scanning time. The basic idea of ​​phase-shift detection autofocus technology based on monochromatic dual-aperture modulation is to convert the axial defocus of the sample into a spatial translation of the image in the lateral direction. Due to the specific linear relationship between the two, the distance of spatial translation can be calculated using an image registration-based algorithm, thereby inferring the corresponding axial defocus. This method can achieve real-time autofocus with only a single image; however, when the sample has equal defocus displacement in both positive and negative directions, the camera will display a ghost image with the same spatial translation. In this case, it will be impossible to distinguish between the positive and negative defocus directions.

[0031] Among related technologies: 1. Phase-shift detection autofocus technology based on monochromatic dual-aperture modulation will produce ghost images with identical spatial translations on the camera when the sample has equal defocus displacement in both positive and negative directions. In this case, it will be impossible to distinguish between positive and negative defocus directions. 2. Real-time autofocus technology based on colored LED illumination requires high-speed switching between red and green LED illumination and bright-field illumination during sample scanning, increasing scanning time. 3. Phase-shift detection autofocus technology based on pupil segmentation images achieves pixel-level offset by introducing a single optical wedge. However, the small offset distance between the ghost images limits the autofocus range.

[0032] Figure 1 This is one of the flowcharts illustrating the focusing method provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Determine the pixel offset between the red channel image and the green channel image of the tissue slice; the red channel image and the green channel image are the red channel image and the green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, the green light filter and the dual light wedge in the focusing module are simultaneously modulated.

[0033] Specifically, most commercially available slicing scanners on the market currently use autofocus technology based on focus maps, which suffers from long autofocus time and poor autofocus accuracy.

[0034] To address the aforementioned issues, this embodiment first determines the pixel offset between the red channel image and the green channel image of the tissue slice. The red channel image and green channel image are the red channel image and green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, green light filter, and dual optical wedges in the focusing module are simultaneously modulated. For example, as shown... Figure 2 As shown, the focusing module in this embodiment comprises an integrated device consisting of a beam splitter, a teleconverter, an optical wedge prism, a red light filter, a green light filter, a tube lens, and a color image sensor. The beam splitter divides the light from the microscope objective into two parts: one part enters the color image sensor 1 to acquire the focused image, and the other part enters the autofocus device to calculate the defocus distance. The teleconverter (using a 4f system here) delays the Fourier plane image of the microscope objective beyond the objective lens. A double-beam wedge prism is placed on the back focal plane of the second lens in the teleconverter. The double-beam wedge prism consists of two symmetrical optical wedge prisms with the same angle, equidistant from the central optical axis, which linearly deflect the incident light rays, thereby introducing a pixel shift closely related to the defocus direction in the image. Red and green light filters are attached to the inclined surfaces of the left and right optical wedge prisms, respectively. These filters filter the bright-field illumination image, resulting in a red-green ghost image on the color image sensor. The telescope focuses the image of the front optical system, color image sensor 1 is used to acquire the collimated image, and the image acquired by color image sensor 2 is used to calculate the defocus distance.

[0035] In other words, this application uses a dual-beam wedge prism, a red light filter, and a green light filter to simultaneously modulate the light entering the autofocus device. The images generated by the red light filter and the green light filter will respectively enter the red channel and green channel of the color image sensor. When the sample is in the foveated position, the images in the red and green channels of the color image sensor 2 will experience a slight pixel shift, such as... Figure 3 As shown in (b); when the sample is in a negative out-of-focus position, the red channel image produces a small pixel shift, such as Figure 3 As shown in (a), when the sample is in a positive out-of-focus position, the red and green channel images produce a large pixel shift, such as... Figure 3 As shown in (c).

[0036] Step 102: Determine the defocus distance based on the pixel offset and the preset correspondence; the preset correspondence is the correspondence between pixel offset and defocus distance.

[0037] Specifically, since the pixel offset between the red and green channel images of the color image sensor 2 is linearly related to the defocus distance of the sample, by pre-calibrating the linear relationship between the two, the defocus position of the sample can be obtained by calculating the pixel offset between the red and green channel images of the color image sensor 2 in real time.

[0038] Step 103: Focus according to the defocus distance.

[0039] Specifically, after determining the defocus distance based on the pixel offset and the preset correspondence, the axial electric displacement platform can be controlled to reach the specified focus position to complete automatic focusing, and the color image sensor 1 acquires the focus image.

[0040] It should be noted that the autofocus method in this application introduces a dual-beam wedge prism into the Fourier surface of the imaging optical path to achieve linear deflection of incident light rays in the optical path. By analyzing the pixel shift in the ghosting region of the image, the positive and negative directions of defocus are accurately determined, overcoming the problem that phase-shift detection autofocus technology based on monochromatic dual-aperture modulation of images cannot distinguish the positive and negative defocus directions in a single frame image. Moreover, the autofocus method of this application only requires bright-field illumination and does not require switching of the illumination source, solving the time waste problem caused by the high-speed switching of the illumination source in autofocus technology based on red-green dual-LED illumination. In addition, by introducing a dual-beam wedge prism into the Fourier surface of the imaging optical path, the autofocus method of this application effectively compensates for the shortcomings of small pixel shift and limited autofocus range caused by the use of a single beam wedge in phase-shift detection autofocus technology based on pupil segmentation images, realizing real-time autofocus in a single frame of a slice scanner.

[0041] The method described in the above embodiments acquires a red-green ghost image of a tissue section when the red light filter, green light filter, and dual wedges in the focusing module are simultaneously modulated, and calculates the pixel offset between the red channel image and the green channel image in the ghost image. Based on a pre-calibrated linear relationship between the pixel offset between the red and green channel images and the defocus distance, the defocus position of the tissue section can be quickly and accurately determined, achieving real-time autofocus in a single frame using a tissue section scanner. The method in this application overcomes the problem that image-based monochrome dual-aperture modulation phase-shift detection autofocus technology cannot distinguish between positive and negative defocus directions in a single frame image, solves the time waste caused by high-speed switching of the illumination source required in red-green dual-LED illumination autofocus technology, and compensates for the small focusing range of image-based pupil segmentation phase-shift detection autofocus technology.

[0042] In some embodiments, determining the pixel offset between the red channel image and the green channel image of a tissue slice includes: The pixel offset between the red channel image and the green channel image of the tissue slice is determined based on the cross-correlation registration algorithm.

[0043] Specifically, the cross-correlation registration algorithm is a frequency domain-based image registration algorithm. The translation of an image in the spatial domain corresponds to a phase change in the frequency domain. Therefore, by comparing the phase information of two images, the pixel offset between them can be uniquely determined. and This algorithm is more accurate than methods that directly compare pixels in the spatial domain for registration, especially for two images with only rigid translation.

[0044] Assumption and These represent the red and green channel images that need to be registered, respectively. This can be seen as a reason Translation To obtain, the following conditions must be met: = The relative offset between the images to be registered is estimated by calculating the cross-correlation function between them. The cross-correlation registration formula is as follows: in and It's an attempt at offset, sign It is a cross-correlation operation. This represents the attempt to find the offset that maximizes the cross-correlation value. The pixel offsets of the red and green channel images can then be calculated using this cross-correlation registration algorithm. and In this application, the offset is only generated in the horizontal direction (a single direction), therefore only pixel offset exists. ,in .

[0045] In some embodiments, determining the defocus distance based on a pixel offset and a preset correspondence includes: Color crosstalk correction is performed on the red channel and green channel images of tissue slices in the ghost images captured by the color image sensor to obtain the corrected red channel image and the corrected green channel image; The defocus distance is determined based on the pixel offset between the corrected red channel image and the corrected green channel image and the preset correspondence.

[0046] Specifically, when only the red light filter is working (i.e., when the green light filter side is blocked), the green channel of the color image sensor still has a certain intensity signal. Similarly, when the red light filter side is blocked, and only the green light filter is working, the red channel of the color image sensor also has a certain response. This is because the response band of the spectral filters inside the color camera is not singular; the camera's red channel can respond to part of the green light signal, and the green channel can also respond to part of the red light signal, leading to color crosstalk between the red and green channels. In this embodiment, by performing color crosstalk correction on the acquired sample color images, the accuracy of the pixel offset calculation for the two channels can be effectively guaranteed, eliminating color crosstalk errors between the red and green channels.

[0047] In the methods described above, color image sensors suffer from spectral crosstalk, where the red channel weakly responds to green light and the green channel weakly responds to red light. Without correction, this crosstalk causes irrelevant signals to be mixed in the red and green channel images, resulting in significant errors in the directly calculated pixel offset. This application corrects the color crosstalk in the red and green channel images of tissue slices in the ghosted images captured by the color image sensor. This provides noise-free and accurate input for subsequent derivation of the defocus distance based on pixel offset, avoiding "offset misjudgment" caused by crosstalk and achieving accurate determination of the defocus distance.

[0048] In some embodiments, color crosstalk correction is performed on the red channel image and green channel image of a tissue slice in a ghost image captured by a color image sensor to obtain corrected red channel image and corrected green channel image, including: When the red light filter, green light filter and dual light wedge in the focusing module are simultaneously modulated, the first red channel image and the first green channel image of the tissue slice sample in the first ghost image captured by the color image sensor are obtained. Acquire the second red channel image of a tissue slice sample in an image captured by the color sensor when only the red light filter is working; Acquire a second green channel image of a tissue slice sample in an image captured by a color sensor when only the green light filter is working; The color crosstalk coefficient is determined based on the first red channel image, the first green channel image, the second red channel image, and the second green channel image. Based on the color crosstalk coefficient, color crosstalk correction is performed on the red channel image and green channel image of the tissue slice in the ghost image captured by the color image sensor to obtain the corrected red channel image and the corrected green channel image.

[0049] Specifically, the color crosstalk model in the embodiments of this application can be represented as follows: in and These are the red and green channel images in the ghost image captured by the color image sensor when the red and green light filters and the dual light wedges are modulated simultaneously, namely the first red channel image and the first green channel image. These are pixel coordinates in two-dimensional space. It means that the color sensor captures the red channel image in the image only when the red light filter is working, that is, the second red channel image; It is when the green light filter is working that the color sensor captures the green channel image in the image, that is, the second green channel image. and These are the color crosstalk coefficients. Assume the image size is... Then the color crosstalk coefficient can be written as: The color crosstalk coefficient is obtained from the above formula. and Therefore, the color-corrected red and green channel images can be described as follows: in, and These represent the red and green channel images, respectively, after color crosstalk correction processing, in the ghost image acquired by the color image sensor 2 under the simultaneous modulation of red and green light filters and color dual light wedges.

[0050] For example, first, the light wedge on the side with the green light filter is blocked, and the color sensor 2 acquires a sample image when only the red light filter is working, and then extracts the red channel image of that image. This is referred to as the red channel reference image. Similarly, by blocking the light wedge on the side of the red light filter, color sensor 2 acquires a sample image when only the green light filter is working, and extracts the green channel image from that image. This is referred to as the green channel reference image. Next, the color image sensor 2 acquires a ghost image when the red and green light filters and dual optical wedges are simultaneously modulated, and extracts the red channel image of this ghost image. With green channel image Finally, the extracted red channel reference image Green channel reference image and red channel image Green channel images Substituting into the color crosstalk model, the color crosstalk coefficients are solved. and .

[0051] Optionally, to verify the effectiveness of the color crosstalk model's correction strategy, the color image sensor 2 acquires a ghost image at a certain defocus position when the red and green light filters and the dual optical wedges are simultaneously modulated. Then, the red and green channel images of the ghost image are extracted, and the calculated crosstalk coefficient is used... and Color crosstalk correction is performed on the red and green channel images, resulting in two corrected red and green channel images, as shown below. Figure 4 The color crosstalk correction results are shown. Figure 4 In the color crosstalk correction results, (a) represents the color channel ghosting image when the red and green filters and the double wedge are simultaneously modulated at a certain defocus position; (b1) represents the red channel image before crosstalk correction; (b2) represents the red channel image after crosstalk correction; (c1) represents the green channel image before crosstalk correction; and (c2) represents the green channel image after crosstalk correction.

[0052] The method in the above embodiment corrects the red and green channel images of tissue slices in the ghost image captured by the color image sensor according to the color crosstalk coefficient, thereby eliminating the interference of color crosstalk on pixel offset calculation and solving the problem of inaccurate focusing caused by color crosstalk in the prior art.

[0053] In some embodiments, a preset correspondence is determined based on the pixel offset of the tissue slice sample at different defocus distances.

[0054] Specifically, before determining the defocus distance based on pixel offset and preset correspondence, this application first needs to calibrate the relationship curve between the relative pixel offset and defocus distance between the red channel and green channel images in the ghost image acquired by the color image sensor 2 under the simultaneous modulation of red and green light filters and dual light wedges.

[0055] Optionally, the axial electric displacement platform is first controlled to move near the relative focus position to acquire ghost images of the sample. Specifically, the sample is first moved to the focus position, and a ghost image at that position is acquired. Then, the displacement platform is controlled to acquire ghost images near the focus position, i.e., moving from a -10μm defocus position to a +10μm defocus position, with a step interval of 2μm, acquiring ghost images at the corresponding positions, resulting in a total of 11 ghost images, including the one acquired at the focus position. Next, red and green channel images are extracted from the ghost images, and color crosstalk correction is performed using the estimated color crosstalk correction coefficients. Then, a cross-correlation image registration algorithm is used on the color crosstalk corrected red and green channel images to calculate the pixel offset ∆x of the red and green channel images. Using the pixel offset calculated at the focus position as a reference, the red and green channel pixel offsets calculated at other positions are subtracted from it to obtain the relative pixel offset x. Finally, based on the sample ghost images acquired at different defocus positions and combined with the pixel offset at the corresponding positions, a calibration curve between the sample defocus distance and the relative pixel offset can be established.

[0056] For example, Figure 5 (a) Shows sample images acquired at different defocus positions, where the area indicated by the red box corresponds to the ghosting image when the red and green filters are simultaneously modulated with the dual optical wedges. When the sample is at the positive defocus position, the red and green channel images will have a large pixel shift, such as... Figure 5 As shown in (a1); when the sample is in near-focus, the red and green channel images will have a slight pixel shift, such as Figure 5 As shown in (a2); when the sample is in a negative out-of-focus position, the red and green channel images will have a small pixel shift, such as Figure 5 As shown in (a3). Furthermore, the pixel offset between the red and green channel images increases with the increase of the sample's defocus displacement, and the two have a linear relationship. Figure 5 In (b), the red markers represent the calibration curves showing the relative pixel offsets between the sample defocus distance and the red and green channel images after color crosstalk correction. Based on this calibration relationship, the corresponding defocus distance can be quickly calculated from the offset measured in the image during subsequent sample scanning, thereby achieving real-time and efficient autofocus. In contrast, the blue markers show the calibration results without color crosstalk correction. The relationship between the defocus distance and the pixel offsets between the red and green channels is significantly more discrete, and the fitting error is also larger, further verifying the effectiveness of crosstalk correction in suppressing error fluctuations and improving the consistency of defocus estimation.

[0057] The method described above provides a reliable mathematical basis for real-time defocus calculation by establishing a precise linear mapping between pixel offset and defocus distance. This effectively solves the problems of long scanning time and low efficiency of commercial slide scanners, and meets the actual needs of rapid acquisition of full-slide digital images in clinical pathology diagnosis.

[0058] For example, such as Figure 6 As shown in the figure, this application provides a focusing method, the specific process of which is as follows: Step 1: Construct the autofocus microscopic imaging system used in this invention.

[0059] Step 2: Estimate the color crosstalk coefficient.

[0060] First, by blocking the light wedge on the green light filter side, meaning that only the red light filter is working, the green channel of the color image sensor still has a certain intensity signal. Similarly, by blocking the light wedge on the red light filter side, the red channel of the color image sensor also exhibits a certain response when only the green light filter is working. Therefore, the model for color crosstalk can be expressed as: (1) (2) in and These are the red and green channel images in the ghost image captured by the color image sensor when the red and green light filters and the dual light wedges are modulated simultaneously. These are pixel coordinates in two-dimensional space. It means that the color sensor captures the red channel image in the image only when the red light filter is working; It means that the color sensor captures the green channel image in the image only when the green light filter is working. and These are the color crosstalk coefficients. Assume the image size is... Then the color crosstalk coefficient can be written as: (3) (4) The color crosstalk coefficient is obtained from the above formula. and Therefore, the color-corrected red and green channel images can be described as follows: (5) (6) in, and These represent the red and green channel images, respectively, after color crosstalk correction processing, in the ghost image acquired by the color image sensor 2 under the simultaneous modulation of red and green light filters and color dual light wedges.

[0061] Step 3: Calibrate the relationship curve between the relative pixel offset and defocus distance between the red channel and the green channel images in the ghost image acquired by the color image sensor 2 under the simultaneous modulation of red and green light filters and dual light wedges.

[0062] The specific operation involves simultaneously modulating the red and green light filters and the dual optical wedges, controlling a single-axis electric displacement platform to move the sample near the focal point and acquire ghost images. Specifically, the sample is moved from a defocus distance of -10µm to a defocus distance of +10µm, with a step interval of 2µm, while ghost images are acquired through the color image sensor 2. Following step 2, color crosstalk correction is performed on the red and green channel images in the ghost image captured by the color image sensor 2, and a cross-correlation registration algorithm is used to calculate the pixel offset of the red and green channel images.

[0063] Cross-correlation registration is a frequency-domain-based image registration algorithm. Image translation in the spatial domain corresponds to phase change in the frequency domain. Therefore, by comparing the phase information of two images, the pixel offset between them can be uniquely determined. and This algorithm is more accurate than methods that directly compare pixels in the spatial domain for registration, especially for two images with only rigid translation.

[0064] Assumption and These represent the red and green channel images that need to be registered, respectively. This can be seen as a reason Translation To obtain, the following conditions must be met: = (7) The relative offset between the images to be registered is estimated by calculating the cross-correlation function between them. The cross-correlation registration formula is as follows: (8) in and It's an attempt at offset, sign It is a cross-correlation operation. This represents the attempt to find the offset that maximizes the cross-correlation value. The pixel offsets of the red and green channel images can then be calculated using this cross-correlation registration algorithm. and The method proposed in the patent only produces offset in the horizontal direction (a single direction), therefore only pixel offset exists. ,in .

[0065] Finally, using the pixel offset calculated at the focus position as a benchmark, the pixel offsets of the red and green channels at other positions are subtracted from it to obtain the relative pixel offset. Relative pixel offset and the corresponding axial defocus distance The data was plotted as a scatter plot, and then curve fitting was performed to obtain the linear parameters. and The fitting formula is as follows: (9) Step 4: Starting from the first field of view, the automatic microscopic imaging system simultaneously modulates the red and green light filters and the dual light wedges, and the color image sensor 2 acquires a ghost image.

[0066] Step 5: Extract the red and green channels of the ghost image separately, and use the color crosstalk coefficient. and Correct the red and green channel images.

[0067] Step 6: Calculate the pixel offsets of the red and green channel images using a cross-correlation image registration algorithm. And convert it into a relative pixel offset. .

[0068] Step 7: Based on the pre-calibrated linear relationship between the axial defocus distance and the relative pixel offset of the image. Obtain the axial defocus distance .

[0069] Step 8: Based on the calculation results in Step 7, control the axial electric displacement platform to reach the designated focusing position, complete autofocus, and the color image sensor 1 acquires the focused image. That is, for the field of view at a certain defocus position, control the color image sensor 2 of the autofocus device to acquire a ghost image while simultaneously modulating the red and green light filters and the dual optical wedges. Then, perform color crosstalk correction on the red and green channel images of this ghost image, and calculate the pixel offset of the red and green channel images using a cross-correlation phase registration algorithm. Next, convert this to a relative pixel offset as described above, and quickly calculate the axial defocus distance using a pre-calibrated curve showing the relationship between the axial defocus distance and the relative pixel offset of the red and green channel images.

[0070] Step 9: Control the XY motorized displacement platform to move to the next field of view, and repeat steps 4-9 until the entire field of view is scanned. That is, based on the calculated axial defocus distance, control the axial motorized displacement platform to reach the designated focusing position, complete autofocus, and the color image sensor 1 acquires the focused image. The XY displacement platform moves to the next field of view position until the entire slice scan is completed.

[0071] The focusing device provided by the present invention is described below. The focusing device described below can be referred to in correspondence with the focusing method described above. The focusing device of the embodiments of this application is as follows: Figure 7 As shown, it includes: The first determining module 710 is used to determine the pixel offset between the red channel image and the green channel image of the tissue slice; the red channel image and the green channel image are the red channel image and the green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, the green light filter and the dual light wedge in the focusing module are simultaneously modulated. The second determining module 720 is used to determine the defocus distance based on the pixel offset and a preset correspondence; the preset correspondence is the correspondence between pixel offset and defocus distance. The focusing module 730 is used to focus based on the defocus distance.

[0072] Figure 8 A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a focusing method. This method includes: determining the pixel offset between the red channel image and the green channel image of the tissue slice; the red channel image and the green channel image are the red channel image and the green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, green light filter, and dual light wedge in the focusing module are simultaneously modulated; determining the defocus distance according to the pixel offset and a preset correspondence; the preset correspondence is the correspondence between the pixel offset and the defocus distance; and focusing according to the defocus distance.

[0073] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the focusing method provided by the above methods. The method includes: determining the pixel offset between the red channel image and the green channel image of the tissue slice; the red channel image and the green channel image are the red channel image and the green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, the green light filter and the dual light wedge in the focusing module are simultaneously modulated; determining the defocus distance according to the pixel offset and a preset correspondence; the preset correspondence is the correspondence between the pixel offset and the defocus distance; and focusing according to the defocus distance.

[0075] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the focusing method provided by the above methods. The method includes: determining the pixel offset between a red channel image and a green channel image of a tissue slice; the red channel image and green channel image being the red channel image and green channel image of the tissue slice in a ghost image captured by a color image sensor when the red light filter, green light filter, and dual light wedge in the focusing module are simultaneously modulated; determining a defocus distance based on the pixel offset and a preset correspondence; the preset correspondence being the correspondence between the pixel offset and the defocus distance; and focusing based on the defocus distance.

[0076] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0077] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A focusing method, characterized in that, include: Determine the pixel offset between the red channel and green channel images of the tissue section; The red channel image and green channel image are the red channel image and green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, green light filter and dual light wedge in the focusing module are simultaneously modulated. The defocus distance is determined based on the pixel offset and a preset correspondence; the preset correspondence is the correspondence between pixel offset and defocus distance. Focusing is performed based on the stated defocus distance.

2. The focusing method according to claim 1, characterized in that, The determination of the pixel offset between the red channel image and the green channel image of the tissue slice includes: The pixel offset between the red channel image and the green channel image of the tissue slice is determined based on the cross-correlation registration algorithm.

3. The focusing method according to claim 1, characterized in that, The step of determining the defocus distance based on the pixel offset and the preset correspondence includes: Color crosstalk correction is performed on the red channel and green channel images of tissue slices in the ghost images captured by the color image sensor to obtain the corrected red channel image and the corrected green channel image; The defocus distance is determined based on the pixel offset between the corrected red channel image and the corrected green channel image and the preset correspondence.

4. The focusing method according to claim 3, characterized in that, The step of performing color crosstalk correction on the red and green channel images of tissue slices in the ghost image captured by the color image sensor to obtain corrected red and green channel images includes: When the red light filter, green light filter and dual light wedge in the focusing module are simultaneously modulated, the first red channel image and the first green channel image of the tissue slice sample in the first ghost image captured by the color image sensor are obtained. Acquire the second red channel image of a tissue slice sample in an image captured by the color sensor when only the red light filter is working; Acquire a second green channel image of a tissue slice sample in an image captured by a color sensor when only the green light filter is working; The color crosstalk coefficient is determined based on the first red channel image, the first green channel image, the second red channel image, and the second green channel image. Based on the color crosstalk coefficient, color crosstalk correction is performed on the red channel image and green channel image of the tissue slice in the ghost image captured by the color image sensor to obtain the corrected red channel image and the corrected green channel image.

5. The focusing method according to claim 4, characterized in that, The step of determining the color crosstalk coefficient based on the first red channel image, the first green channel image, the second red channel image, and the second green channel image includes: The color crosstalk coefficient is determined based on the following method: in, and Indicates the color crosstalk coefficient; M and N represent the image size; This represents the first red channel image; This represents the first green channel image; This represents the second red channel image; This represents the second green channel image.

6. The focusing method according to any one of claims 1-5, characterized in that, The method further includes: The preset correspondence is determined based on the pixel offset of the tissue slice sample at different defocus distances.

7. A focusing device, characterized in that, include: The first determining module is used to determine the pixel offset between the red channel image and the green channel image of the tissue slice; The red channel image and green channel image are the red channel image and green channel image of the tissue slice in the ghost image captured by the color image sensor when the red light filter, green light filter and dual light wedge in the focusing module are simultaneously modulated. The second determining module is used to determine the defocus distance based on the pixel offset and a preset correspondence; the preset correspondence is the correspondence between pixel offset and defocus distance. A focusing module is used to focus based on the defocus distance.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the focusing method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the focusing method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the focusing method as described in any one of claims 1 to 6.

Images