System and method for detecting focal plane of infinite-distance microscopic system

Abstract

The invention relates to a system for detecting a focal plane of an infinite-distance microscopic system. The system comprises an objective lens, a spectroscope, a tube lens, an image acquisition module and an image processing module. The objective lens collects light reflected by an object and converts the light into parallel light, and the spectroscope separates the parallel light into two parts which are respectively transmitted to the tube lens and the image acquisition module. The first tube lens receives part of light and focuses the light on an image focal plane, the second tube lens in the image acquisition module receives the other part of light and focuses the light on the digital camera photosensitive chip, and the digital camera photosensitive chip is obliquely mounted to ensure light focusing of a specific field of view. And the image processing module processes the digital image through partitions and analyzes the definition curve graph, and calculates an object plane defocusing amount to determine an optimal focal plane position.

Description

Technical Field

[0001] The present invention relates to the field of microscope technology, and more specifically to a system and method for detecting the focal plane of an infinite microscope system. Background Art

[0002] With the continuous development of optical technology, autofocus technology is increasingly used in imaging systems. Although the theory and methods of autofocus technology are developing rapidly, the passive method of autofocusing requires multiple image acquisitions and the analysis and processing of image clarity, which takes too long. It is necessary to find a new fast and efficient focus detection method to reduce the focus detection time and improve the focusing speed. Summary of the invention

[0003] The present invention provides a system for detecting the focal plane of an infinity microscope system, which can improve the financial management ability and acceptance rate of customers and reduce the risk factor. At the same time, by reducing the risk factor, the financial management ability and acceptance rate of customers can be improved inversely, thereby finding a profit balance point and improving business profitability.

[0004] The present invention provides a system for detecting the focal plane of an infinite microscope system, comprising an objective lens, a beam splitter, a first tube lens, an image acquisition module and an image processing module, wherein:

[0005] The objective lens is used to collect light reflected by an object located on the object focal plane and convert it into parallel light to be emitted to infinity;

[0006] The beam splitter is arranged behind the objective lens, and is used to separate the parallel light into two parts and guide them to the first tube lens and the image acquisition module respectively;

[0007] The first tube lens is arranged in the direction of the projection light of the beam splitter, and is used to receive part of the projection light separated by the beam splitter, and converge it onto the first image focal plane;

[0008] The image acquisition module comprises a second tube lens and a digital camera photosensitive chip which are sequentially arranged in the direction of the reflected light of the beam splitter, wherein the second tube lens is used to receive another part of the reflected light separated by the beam splitter and focus it on the second image focal plane, and the digital camera photosensitive chip is tiltedly arranged on the second image focal plane; the digital camera photosensitive chip adopts a tilted design, so that only the light of a specific field of view can be focused on the chip, and other fields of view are defocused due to the tilt of the chip;

[0009] The image processing module is used to receive the digital image signal transmitted by the digital camera photosensitive chip, and perform image partition processing, and perform image processing on the field of view image obtained after the partition processing. The image processing includes using a preset algorithm to evaluate the image clarity and draw a clarity curve diagram, confirming the position corresponding to the imaging partition with the highest clarity through the clarity curve diagram, obtaining the image plane defocus amount through the relationship between the imaging partition position and the tilt angle of the digital camera photosensitive chip, and calculating the object plane defocus amount based on the relationship between the image plane defocus amount and the object plane defocus amount with respect to the magnification to determine the optimal focal plane position.

[0010] The present invention also provides a method for detecting the focal plane of an infinite microscope system, based on the aforementioned system for detecting the focal plane of an infinite microscope system, characterized in that it comprises the following steps:

[0011] Light collection steps: Use an objective lens to collect light reflected from an object on the object focal plane and convert it into parallel light that shoots toward infinity;

[0012] Light separation step: Use a beam splitter to separate the light in the parallel light path into two beams: projected light and reflected light;

[0013] First tube lens processing step: the first tube lens receives the projection light distributed by the beam splitter and focuses the projection light on the image side focal plane of the first tube lens;

[0014] Second tube lens processing step: focusing the reflected light onto a photosensitive chip of a digital camera through a second tube lens, wherein the reflected light includes a plurality of sub-field light beams;

[0015] Image acquisition step: the sub-viewing field light beam is projected onto the digital camera photosensitive chip through the second tube lens, each sub-viewing field light beam forms a corresponding image on the digital camera photosensitive chip, and the multiple view fields respectively cover different areas of the digital camera photosensitive chip. The digital camera photosensitive chip is tilted so that one of the view fields is focused on the digital camera photosensitive chip and the other view fields are out of focus. The digital camera photosensitive chip transmits the digital image signal to the image processing module;

[0016] Partition processing steps: the digital camera photosensitive chip is divided into regions according to different resolutions, and is divided into coarse equal partitions and fine equal partitions respectively, wherein the coarse equal partition is larger than the fine equal partition, and the digital camera photosensitive chip is divided into regions with a resolution between the coarse equal partition and the fine equal partition to form overlapping partitions, and the overlapping partitions cover adjacent coarse equal partitions and fine equal partitions;

[0017] Image processing steps: The image processing module uses a preset algorithm to evaluate the clarity of each field of view on the partitioned image obtained after the partitioning process, and draws a clarity curve diagram. The position of the field of view with the highest clarity is confirmed by the clarity curve diagram, and the position of the partition corresponding to the field of view is determined. If the peak position of the image clarity curve is on the central field of view, the position of the object plane is the focal plane of the objective lens. If the peak position of the image clarity curve is not on the central field of view, the image plane defocus is obtained through the relationship between the partition position and the tilt angle of the digital camera photosensitive chip. The object plane defocus is calculated based on the relationship between the image plane defocus and the object plane defocus with respect to the magnification. The objective lens is moved a corresponding distance based on the solved object plane defocus, and the preset algorithm is used again to evaluate the clarity of each field of view. The above process is repeated until the peak position of the image clarity curve is on the central field of view. At this time, the object plane is the focal plane of the objective lens.

[0018] Preferably, in the image processing step, the image plane defocus amount is obtained by the relationship between the partition position and the tilt angle of the digital camera photosensitive chip, which is expressed as d 像 =L s × sinθ, where L s is the field of view position, and θ is the tilt angle of the digital camera photosensitive chip.

[0019] Preferably, in the image processing step, the object plane defocus amount is calculated according to the relationship between the image plane defocus amount and the object plane defocus amount with respect to the magnification, and is expressed as d 物 =d 像 ÷β 2 , where d 像 is the image plane defocus amount, and β is the magnification of the system.

[0020] Preferably, in the image processing step, the image processing module uses a preset algorithm to evaluate the clarity of the partitioned image obtained after the partitioning process, and the preset algorithm is to use Laplacian algorithm, wavelet transform, Fourier transform or contrast method to evaluate the clarity of the image in the partition.

[0021] Preferably, in the image processing step, the image processing includes using a variational autoencoder model VAE_c to process the image data in the coarse partition, and the variational autoencoder model VAE_c includes an encoder and a decoder

[0022] The encoder is represented as: Among them, μ c (x c )and Generated by the encoder network to map the input image x c To the latent space z c ;

[0023] The decoder is represented as: For the latent space z c Reconstruct image x c , optimized to capture a wide range of features of an image;

[0024] The image processing also includes using a variational autoencoder model VAE_f to process image data in the fine partitions, and the variational autoencoder model VAE_c includes an encoder and a decoder

[0025] The encoder is represented as: Among them, μ f (x f )and Generated by the encoder network to map the input image x c To the latent space z c ;

[0026] The decoder is represented as: For the latent space z f Reconstruct image x f , improve the details and texture accuracy of the image;

[0027] In overlapping partitions, the model outputs of the coarse and fine partitions use the weighted sum method to obtain the information of the edge overlap, expressed as: combined =ω c μ′ c (z c )+ω f μ′ f (z f ), where ω c and ω f are the decoding output weight factors of the coarse and fine partitions respectively.

[0028] Preferably, the network of the coarse equally partitioned variational autoencoder includes at least three convolutional layers, the convolution kernel size is 5x5, and the number of channels of the convolutional layers is 32, 64 and 128 respectively; the convolution operation of the lth layer in the variational autoencoder model VAE_f is expressed as: in is the convolution kernel of layer l, is the bias term;

[0029] The network of the coarse equally partitioned variational autoencoder includes at least five convolutional layers, the convolution kernel size is 3x3, and the number of channels of the convolutional layers is 64, 128, 256 and 512 respectively; the convolution operation of the lth layer in the variational autoencoder model VAE_c is expressed as: in is the convolution kernel of layer l, is the bias term.

[0030] Preferably, the weight factor ω c and ω f Use gradient descent to optimize to minimize the reconstruction error after fusion, which includes the following steps:

[0031] Objective function definition: Define the objective function J(ω) of the image reconstruction error after fusion c ,ω f ), specifically expressed as:

[0032] J(ω c ,ω f )=||ω c μ′ c (z c )+ω f μ′ f (z f )-x true || 2

[0033] Among them, μ′ c (z c ) and μ′ f (z f ) are the decoded outputs from the coarse and fine partition variational autoencoder models, respectively, x true is the target image, and the goal is to find ω that minimizes the reconstruction error c and ω f .

[0034] Gradient calculation: Calculate the objective function with respect to v c and v f The gradient is:

[0035]

[0036] Among them, the gradient and Denotes J with respect to ω c and ω f sensitivity to change;

[0037] The rule for updating the weight factor using the gradient descent method is expressed as:

[0038]

[0039] Among them, η is the learning rate, which is used to control the update step size;

[0040] Initialize the weight factor ω c and ω f , expressed as ω c =ωf = 0.5; Apply the gradient descent method to update the weight factor rule for multiple iterations, each iteration using a new set of μ′ obtained from the data set f (z f ), μ′ c (z c ) and x true To calculate the gradient and update the weights, iterate until the reconstruction error J(ω c ,ω f ) reaches the preset value.

[0041] Preferably, in the image processing step, the image plane defocus amount is determined according to the tilt angle and field of view position of the digital camera photosensitive chip, and then the object plane defocus amount is calculated, and the focal length is adjusted in real time through the machine learning model to optimize the imaging effect; the establishment of the machine learning model specifically includes the following steps:

[0042] Training data preparation: Collect images with different focal length settings and their corresponding clarity scores. The clarity scores are obtained using the Laplacian algorithm and are expressed as:

[0043]

[0044] Among them, x i represents the i-th image, y i is the corresponding focal length setting, d i is the clarity score obtained by the Laplacian algorithm;

[0045] Model training: Using the regression model f(y i ; θ) to learn the relationship between focal length y and image clarity d, where θ is a model parameter; the least squares method is used to train the regression model to minimize the difference between the predicted clarity and the actual clarity, expressed as:

[0046]

[0047] Real-time clarity assessment: Capture images in real time and assess their clarity, expressed as:

[0048] d current =Laplacian(capture())

[0049] Use the optimization algorithm to make slight adjustments to the focal length to find the best focus, expressed as:

[0050]

[0051] Among them, α is the learning rate, Represents the gradient of the objective function with respect to the focal length, which is used to update the current focal length to improve image clarity.

[0052] Preferably, the step of obtaining a clarity score by Laplacian algorithm comprises:

[0053] Convolution operation: Apply the Laplacian kernel to the original image I and calculate the Laplacian value of each pixel, expressed as: I Laplacian =L*I, where * represents the convolution operation and I represents the original image;

[0054] Calculate variance: The sharpness can be evaluated by calculating the variance of the Laplacian image, expressed as:

[0055] Sharpness=Var(I Laplacian )

[0056] The variance is calculated as:

[0057]

[0058] where x i isI Laplacian The pixel value in , μ is x i is the average value of , where n is the total number of pixels.

[0059] The beneficial effect of the present invention is that the present invention divides parallel light into two beams, projection light and reflected light, through a beam splitter, and processes them independently through a first tube lens and a second tube lens, thereby increasing the imaging flexibility of the system. The independent operation of the first tube lens and the second tube lens allows the system to optimize the focus and imaging quality on different imaging paths at the same time, adapting to a wider range of application requirements. Each sub-field of view light beam emitted from the beam splitter lens corresponds to a coarse equal partition on the photosensitive chip, and an overlapping area is designed at the edge of the coarse equal partition as a fine equal partition. This design not only enhances the focusing accuracy of the image, but also ensures the integrity of the image information, especially in the edge area, effectively reduces information loss, and improves the continuity and detail performance of the imaging. Through the tilt design of the digital camera photosensitive chip, the system can achieve optimal focus on a specific field of view, while other fields of view are appropriately adjusted as needed. This design enables the system to adjust the focus according to different imaging requirements, and is suitable for a variety of occasions requiring high-precision focusing and wide field of view coverage. The system uses a variety of image clarity evaluation technologies such as Laplacian algorithm, wavelet transform, Fourier transform and contrast method through the image processing module to accurately evaluate the image quality of each partition, and accurately control the focus plane through the clarity curve. By calculating and adjusting the relationship between the image plane defocus and the object plane defocus, the system can dynamically adjust the imaging parameters to ensure that the optimal focal plane position can be obtained at different magnifications. The system integrates image processing algorithms and automatic focusing technology, which can automatically analyze and adjust imaging parameters, greatly reducing the difficulty of operation and improving operating efficiency and imaging speed. This is particularly beneficial for scientific research and industrial applications that require fast, efficient and frequent adjustment of focal length and imaging parameters. Therefore, it can be said that the present invention improves the performance and application value of the microscope system, enabling it to play an important role in multiple fields such as scientific research, medical diagnosis, and industrial detection, and meet the imaging challenges of higher precision and wider needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work:

[0061] Figure 1 It is a schematic diagram of a focusing embodiment 1 of a system for detecting a focal plane of an infinity microscope system according to an embodiment of the present invention;

[0062] Figure 2 It is a clarity curve diagram of a first embodiment of a system for detecting a focal plane of an infinite microscope system according to an embodiment of the present invention;

[0063] Figure 3This is a schematic diagram of a second focusing embodiment of a system for focal plane detection of an infinite microscope system according to an embodiment of the present invention;

[0064] Figure 4 This is a clarity curve diagram of a second embodiment of a system for focal plane detection of an infinite microscope system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the following will be described clearly and completely in combination with the technical solution in the embodiments of the present invention. Obviously, the described embodiments are partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0066] like Figure 1-4 As shown, the present invention provides a system for detecting the focal plane of an infinite microscope system, comprising an objective lens 1, a beam splitter 2, a first tube lens 3, an image acquisition module 4 and an image processing module 5, wherein:

[0067] Objective lens 1 is used to collect the light reflected by the object on the object focal plane and convert it into parallel light to be emitted to infinity;

[0068] The beam splitter 2 is arranged behind the objective lens 1, and is used to separate the parallel light into two parts and guide them to the first tube lens 3 and the image acquisition module 4 respectively;

[0069] The first tube lens 3 is arranged in the direction of the projected light of the beam splitter 2, and is used to receive part of the projected light separated by the beam splitter 2, and converge it onto the first image focal plane;

[0070] The image acquisition module 4 includes a second tube lens 41 and a digital camera photosensitive chip 42 which are sequentially arranged in the direction of the reflected light from the beam splitter 2. The second tube lens 41 is used to receive another part of the reflected light separated by the beam splitter 2 and focus it on the second image focal plane. The digital camera photosensitive chip 42 is tiltedly arranged on the second image focal plane. The digital camera photosensitive chip adopts a tilted design, so that only the light of a specific field of view can be focused on the chip, and other fields of view are defocused due to the tilt of the chip.

[0071] The image processing module 5 is used to receive the digital image signal transmitted by the digital camera photosensitive chip 42, and perform image partition processing, and perform image processing on the field of view image obtained after the partition processing. The image processing includes using a preset algorithm to evaluate the image clarity and draw a clarity curve diagram, confirming the position corresponding to the imaging partition with the highest clarity through the clarity curve diagram, and obtaining the image plane defocus amount through the relationship between the imaging partition position and the inclination angle of the digital camera photosensitive chip 42. According to the relationship between the image plane defocus amount and the object plane defocus amount with respect to the magnification, the object plane defocus amount is calculated to determine the optimal focal plane position.

[0072] like Figure 1-4 As shown, the reflected light formed by the parallel light passing through the beam splitter 2 is divided into a plurality of sub-viewing light beams, and the sub-viewing light beams enter the second tube lens 41 and are focused on the digital camera photosensitive chip 42.

[0073] Also included is a method for detecting the focal plane of an infinite microscope system, based on the aforementioned system for detecting the focal plane of an infinite microscope system, comprising the following steps:

[0074] Light collection steps: Use objective lens 1 to collect the light reflected by the object on the object focal plane, and convert it into parallel light to be emitted to infinity;

[0075] Light separation step: Use a beam splitter 2 to separate the light in the parallel light path into two beams: projected light and reflected light;

[0076] First tube lens processing step: the first tube lens 3 receives the projection light distributed by the beam splitter 2, and focuses the projection light on the image side focal plane of the first tube lens 3;

[0077] Second tube lens processing step: focusing the reflected light onto the digital camera photosensitive chip 42 through the second tube lens 41, the reflected light includes a plurality of sub-field light beams;

[0078] Image acquisition step: the sub-viewing field light beam is projected onto the digital camera photosensitive chip 42 through the second tube lens 41, and each sub-viewing field light beam forms a corresponding image on the digital camera photosensitive chip 42. The multiple view fields respectively cover different areas of the digital camera photosensitive chip 42. The digital camera photosensitive chip 42 is tilted so that one of the view fields is focused on the digital camera photosensitive chip 42, and the other view fields are out of focus. The digital camera photosensitive chip 42 transmits the digital image signal to the image processing module 5;

[0079] Partition processing step: divide the digital camera photosensitive chip 42 into regions according to different resolutions, and divide them into coarse equal partitions and fine equal partitions, wherein the coarse equal partitions are larger than the fine equal partitions, and divide the digital camera photosensitive chip 42 into regions with a resolution between the coarse equal partitions and the fine equal partitions to form overlapping partitions, and the overlapping partitions cover adjacent coarse equal partitions and fine equal partitions;

[0080] Image processing steps: The image processing module 5 uses a preset algorithm to evaluate the clarity of each field of view of the partitioned image obtained after the partitioning processing, and draws a clarity curve diagram. The position of the field of view with the highest clarity is confirmed by the clarity curve diagram, and the position of the partition corresponding to the field of view is determined. If the peak position of the image clarity curve is on the central field of view, the position of the object plane is the focal plane of the objective lens 1 at this time. If the peak position of the image clarity curve is not on the central field of view, the image plane defocus amount is obtained through the relationship between the partition position and the inclination angle of the digital camera photosensitive chip 42. The object plane defocus amount is calculated according to the relationship between the image plane defocus amount and the object plane defocus amount with respect to the magnification. The objective lens 1 is moved a corresponding distance according to the solved object plane defocus amount, and the preset algorithm is used again to evaluate the clarity of each field of view. The above process is repeated until the peak position of the image clarity curve is on the central field of view. At this time, the object plane is the focal plane of the objective lens 1.

[0081] In the image processing step, the image plane defocus amount is obtained by the relationship between the partition position and the tilt angle of the digital camera photosensitive chip 42, which is expressed as d 像 =L s × sinθ.

[0082] In the image processing step, the object plane defocus amount is calculated based on the relationship between the image plane defocus amount and the object plane defocus amount and the magnification, and is expressed as d 物 =d 像 ÷β 2 , where β is the magnification of the system.

[0083] The present invention divides parallel light into two beams, projection light and reflected light, through a beam splitter, and processes them independently through a first tube lens and a second tube lens, thereby increasing the imaging flexibility of the system. The independent operation of the first tube lens and the second tube lens allows the system to optimize the focus and imaging quality on different imaging paths at the same time, and adapt to a wider range of application requirements. Each sub-field of view beam emitted from the beam splitter lens corresponds to a coarse equal partition on the photosensitive chip, and an overlapping area is designed at the edge of the coarse equal partition as a fine equal partition. This design not only enhances the focusing accuracy of the image, but also ensures the integrity of the image information, especially in the edge area, effectively reduces information loss, and improves the continuity and detail performance of the imaging. Through the tilt design of the digital camera photosensitive chip, the system can achieve optimal focus on a specific field of view, while other fields of view are appropriately adjusted as needed. This design enables the system to adjust the focus according to different imaging requirements, and is suitable for a variety of occasions requiring high-precision focusing and wide field of view coverage. The system uses a variety of image clarity evaluation technologies such as Laplacian algorithm, wavelet transform, Fourier transform or contrast method through an image processing module to accurately evaluate the image quality of each partition, and accurately control the focus plane through a clarity curve diagram. By calculating and adjusting the relationship between the image plane defocus and the object plane defocus, the system can dynamically adjust the imaging parameters to ensure that the optimal focal position can be obtained at different magnifications. The system integrates image processing algorithms and autofocus technology, and can automatically analyze and adjust imaging parameters, greatly reducing the difficulty of operation and improving operating efficiency and imaging speed. This is particularly beneficial for scientific research and industrial applications that require fast, efficient and frequent adjustment of focal length and imaging parameters. Therefore, it can be said that the present invention improves the performance and application value of the microscope system, enabling it to play an important role in multiple fields such as scientific research, medical diagnosis, and industrial detection, and meet the challenges of imaging with higher precision and wider needs.

[0084] According to the above scheme, embodiment 1 of the present invention focuses on the specific successful cases, such as Figure 1 As shown, the system configuration and operation process:

[0085] Objective lens: Located at the front end of the system, it is used to capture the light reflected by the object on its object focal plane and convert it into a parallel beam of light that shoots to infinity.

[0086] The beam splitter is placed behind the objective lens and is responsible for separating the parallel light into two parts, one of which is directed to the first tube lens and the other to the image acquisition module. The first tube lens receives part of the projected light distributed by the beam splitter and focuses it on the image focal plane of the tube lens. The second tube lens receives the reflected light from the beam splitter and focuses it precisely onto the photosensitive chip. Since the photosensitive chip of the digital camera is tilted, Figure 1 Only the light from the center field of view is focused to the center of the photosensitive chip of the tilted digital camera, and the rest of the field of view is out of focus.

[0087] Partition the digital camera photosensitive chip. Specifically, divide the digital camera photosensitive chip into 5X5 small windows as a rough partition. Divide the digital camera photosensitive chip into 2X2 small windows as a fine partition. Divide the digital camera photosensitive chip into 3X3 small windows as an overlapping partition. This area includes both the rough partition and a part of the fine partition. Therefore, the central field of view is field of view 2 at this time.

[0088] Under standard operating conditions, the object is precisely placed on the focal plane of the objective lens to ensure that the light emitted from the objective lens is completely parallel. The parallel light is correctly distributed through the beam splitter, and the second tube lens is precisely adjusted to ensure that the central field of view light is focused on the center of the photosensitive chip. Figure 2 It corresponds to Figure 1 The clarity curves drawn by the collected images with different clarity levels, the horizontal axis is the field of view position, the vertical axis is the image clarity, it can be clearly seen Figure 2 The field of view with the largest image clarity is 2 (i.e., the center field of view), and then the position corresponding to the partition is confirmed. The image plane defocus is obtained through the relationship between the partition position and the tilt angle of the digital camera photosensitive chip. The object plane defocus can be obtained based on the relationship between the image plane defocus and the object plane defocus with respect to the magnification. The image processing module receives clear image data and calculates that the object plane defocus is zero, without additional adjustment.

[0089] Embodiment 2 of the present invention is a focusing adjustment process, such as Figure 3 As shown, the object is not on the focal plane of the objective lens. The light passes through the objective lens through the beam splitter to the image acquisition module. The light passes through the second tube lens and is focused in front of the digital camera photosensitive chip. At this time, the image plane of the second tube lens is in front of the digital camera photosensitive chip. Figure 3 Only one field of view of the middle light is focused on the digital camera photosensitive chip after tilt processing through the second tube lens, and the other fields of view are out of focus. Therefore, the object is not accurately placed on the focal plane of the objective lens, resulting in the light emitted from the objective lens not being completely parallel, which in turn affects the focusing quality of the image.

[0090] System adjustment process:

[0091] After the light passes through the beam splitter, part of the light fails to be accurately focused on the predetermined position of the photosensitive chip when it is focused by the second tube lens. The image data received by the image processing module shows that the focus is moved forward or part of the field of view is out of focus. Figure 4 For the corresponding Figure 3 The clarity curves drawn by the collected images with different clarity levels, the horizontal axis is the field of view position, the vertical axis is the image clarity, it can be clearly seen Figure 4The position of the field of view with the largest clarity is 1, and then the position corresponding to the partition is confirmed. The image plane defocus is obtained through the relationship between the partition position and the tilt angle of the digital camera photosensitive chip. The object plane defocus can be obtained based on the relationship between the image plane defocus and the object plane defocus with respect to the magnification. Through real-time image clarity evaluation, the image processing module calculates the image plane defocus. The object plane defocus is calculated using the image plane defocus and the system magnification β. According to the object plane defocus, the system automatically adjusts the focal length setting to correct the focus deviation and achieve focus optimization.

[0092] Through these two embodiments, it can be seen that the system can not only achieve precise focusing under ideal conditions, but also has the ability to correct focusing deviations through algorithms and automatic adjustment mechanisms under non-ideal conditions, which greatly improves the practicality and flexibility of the system.

[0093] In the image processing step of this embodiment, the image processing includes using a variational autoencoder model VAE_c to process the image data in the coarse partition. The variational autoencoder model VAE_c includes an encoder and a decoder

[0094] The encoder is represented as: Among them, μ c (x c )and Generated by the encoder network to map the input image x c To the latent space z c ;

[0095] The decoder is represented as: For the latent space z c Reconstruct image x c , optimized to capture a wide range of features of an image;

[0096] The image processing also includes using a variational autoencoder model VAE_f to process image data in the fine partitions, and the variational autoencoder model VAE_c includes an encoder and a decoder

[0097] The encoder is represented as: Among them, μ f (x f )and Generated by the encoder network to map the input image x c To the latent space z c ;

[0098] The decoder is represented as: For the latent space z f Reconstruct image x f, improve the details and texture accuracy of the image;

[0099] In overlapping partitions, the model outputs of the coarse and fine partitions use the weighted sum method to obtain the information of the edge overlap, expressed as: combined =ω c μ′ c (z c )+ω f μ′ f (z f ), where ω c and ω f are the decoding output weight factors of the coarse and fine partitions respectively.

[0100] The VAE model of coarse and equal partitions (VAE_c) can capture the macroscopic features and extensive structures of the image, which helps to maintain the overall consistency and contextual information of the image. The VAE model of fine and equal partitions (VAE_f) focuses on the microscopic details and textures of the image, improving the resolution and detail performance of the image, especially at the edges and details of the image. By fusing image data from coarse and fine equal partitions in overlapping partitions, the representation of the overall structure and local details can be effectively balanced, and the dynamic range of the image can be optimized.

[0101] The present invention adjusts the weight factor ω in the weighted sum method c and ω f , achieve optimal image expression in different scenarios, and thus provide the best image quality in different applications.

[0102] In this embodiment, the image data to be processed is firstly collected and allocated to coarse equal divisions or fine equal divisions according to the position of the image content.

[0103] For roughly partitioned images, use the VAE_c encoder Map the image to the latent space z c , capturing macro features. For finely divided images, use the encoder of VAE_f Map the image to the latent space z f , focusing on capturing details and textures. Then using the decoder From the latent space z c Reconstruct the image of the roughly equal partition and optimize the overall structure. Use the decoder From the latent space z f Reconstruct the image of fine and equal partitions to improve the detail performance. In overlapping partitions, the system uses the weighted sum method x combined =ω c μ′ c (z c )+ω f μ′ f (zf ), fuse the images of coarse and fine equal partitions, and adjust ω c and ω f To optimize the fusion effect and ensure a natural transition of the edge part. The final fused image is presented to the user or stored for further analysis. The system can adjust the weight factors and processing parameters based on user feedback or automatic analysis results to meet different imaging requirements.

[0104] The VAE-based image processing strategy can adapt to different image characteristics, such as different lighting conditions, different imaging device characteristics, and different properties of the target object. By intelligently adjusting the weighting factors and VAE parameters, the system can provide customized image processing services while maintaining efficient processing speed, meeting the high standards for image quality in fields such as scientific research and clinical diagnosis.

[0105] In this embodiment, the network of the coarse equally partitioned variational autoencoder includes at least three convolutional layers, the convolution kernel size is 5x5, and the number of channels of the convolutional layers is 32, 64 and 128 respectively; the convolution operation of the lth layer in the variational autoencoder model VAE_f is expressed as: in is the convolution kernel of layer l, is the bias term;

[0106] The network of the coarse equally partitioned variational autoencoder includes at least five convolutional layers, the convolution kernel size is 3x3, and the number of channels of the convolutional layers is 64, 128, 256 and 512 respectively; the convolution operation of the lth layer in the variational autoencoder model VAE_c is expressed as: in is the convolution kernel of layer l, is the bias term.

[0107] In this embodiment, the weight factor ω c and ω f Use gradient descent to optimize to minimize the reconstruction error after fusion, which includes the following steps:

[0108] Objective function definition: Define the objective function J(ω) of the image reconstruction error after fusion c ,ω f ), specifically expressed as:

[0109] J(ω c ,ω f )=||ω c μ′ c (z c )+v f μ′ f (z f )-x true || 2

[0110] Among them, μ′ c (z c ) and μ′ f (z f ) are the decoded outputs from the coarse and fine partition variational autoencoder models, respectively, x true is the target image, and the goal is to find ω that minimizes the reconstruction error c and ω f .

[0111] The larger convolution kernels and wider receptive fields of the coarse partitions help the model effectively capture the overall image features and contextual information while maintaining computational efficiency, which is suitable for rapid analysis of background and structure. The smaller convolution kernels and multi-layer structures of the fine partitions enable the model to deeply analyze the microscopic details of the image without adding too much computational burden, especially in application scenarios that require high precision, such as cell structures and material surfaces in medical and scientific research.

[0112] Different network architectures and parameter settings provide optimized processing paths for different types of image data, allowing the system to flexibly respond to a variety of needs from extensive medical imaging to sophisticated scientific research. The ability to automatically adjust focus and image fusion significantly improves operational efficiency and reduces the need for manual intervention, while improving overall imaging performance by ensuring optimal processing of images in each partition. Therefore, it provides strong support for real-time image processing, especially in clinical environments that require fast and accurate diagnosis, as well as scientific research fields that require extremely high image quality. In summary, the design of the dual variational autoencoder model not only improves the accuracy and efficiency of image processing, but also provides reliable support for demanding imaging applications through intelligent image processing technology.

[0113] Gradient calculation: Calculate the objective function with respect to ω c and ω f The gradient is:

[0114]

[0115] Among them, the gradient and Denotes J with respect to ω c and ω f sensitivity to change;

[0116] The rule for updating the weight factor using the gradient descent method is expressed as:

[0117]

[0118] Among them, η is the learning rate, which is used to control the update step size;

[0119] Initialize the weight factor ω c and ω f , expressed as ω c =ω f = 0.5; Apply the gradient descent method to update the weight factor rule for multiple iterations, each iteration using a new set of μ′ obtained from the data set f (z f ), μ′ c (z c ) and x true To calculate the gradient and update the weights, iterate until the reconstruction error J(ω c ,ω f ) reaches the preset value.

[0120] By fine-tuning the weighting factors, the system is able to effectively balance the image contribution between coarse and fine partitions, thereby enhancing microscopic details while maintaining the macroscopic structure of the image. The model is able to dynamically adjust the weighting factors based on real-time image data to adapt to different imaging conditions and needs. The automatic optimization process reduces the reliance on manual adjustments by professionals, thereby improving the efficiency and convenience of operation. By using gradient descent and automatically adjusting weights, the system can use its computing resources more efficiently. This means that the speed and quality of image processing can be improved while maintaining or even reducing computing costs. Combining information from different coarse and fine partitions can enhance the system's image parsing capabilities, especially when processing images with complex textures and boundaries. This method of comprehensively utilizing information from different partitions can better handle noise and inconsistencies in the image, thereby improving the accuracy and usability of the final image.

[0121] In this embodiment, in the image processing step, the image plane defocus amount is determined according to the tilt angle and field of view position of the digital camera photosensitive chip, and then the object plane defocus amount is calculated, and the focal length is adjusted in real time through the machine learning model to optimize the imaging effect; the establishment of the machine learning model specifically includes the following steps:

[0122] Training data preparation: Collect images with different focal length settings and their corresponding clarity scores. The clarity scores are obtained using the Laplacian algorithm and are expressed as:

[0123]

[0124] Among them, x i represents the i-th image, y i is the corresponding focal length setting, d i is the clarity score obtained by the Laplacian algorithm;

[0125] Model training: Using the regression model f(y i; θ) to learn the relationship between focal length y and image clarity d, where θ is a model parameter; the least squares method is used to train the regression model to minimize the difference between the predicted clarity and the actual clarity, expressed as:

[0126]

[0127] Real-time clarity assessment: Capture images in real time and assess their clarity, expressed as:

[0128] d current =Laplacian(capture())

[0129] Use the optimization algorithm to make slight adjustments to the focal length to find the best focus, expressed as:

[0130]

[0131] Among them, α is the learning rate, Represents the gradient of the objective function with respect to the focal length, which is used to update the current focal length to improve image clarity.

[0132] By precisely adjusting the focal length to match the sharpness calculated in real time, the system can automatically and accurately find the optimal focus position. By automating the process, the reliance on operators is significantly reduced, the imaging process is accelerated, and image quality issues caused by human error are reduced. The system continuously evaluates the sharpness of the captured image through the Laplacian algorithm to monitor image quality in real time. The instant feedback mechanism enables the imaging device to respond quickly to changes, which is suitable for dynamic or rapidly changing imaging environments. Using the relationship model learned from experimental data, the system can predict and optimize imaging results based on historical data and statistical methods, rather than relying solely on a single observation. As more imaging data is captured and analyzed, the machine learning model can continue to learn and optimize, improving its prediction accuracy and adaptability. This continuous learning ability makes the system smarter and more efficient over time.

[0133] In this embodiment, the step of obtaining a clarity score by using the Laplacian algorithm includes:

[0134] Convolution operation: Apply the Laplacian kernel to the original image I (or the image information of the edge overlap portion of the overlapping partitions obtained in the above embodiment), and calculate the Laplacian value of each pixel, which is expressed as: I Laplacian =L*I, where * represents the convolution operation and I represents the original image;

[0135] Calculate variance: The sharpness can be evaluated by calculating the variance of the Laplacian image, expressed as:

[0136] Sharpness=Var(I Laplacian )

[0137] The variance is calculated as:

[0138]

[0139] where x i isI Laplacian The pixel value in , μ is x i is the average value of , where n is the total number of pixels.

[0140] Therefore, by precisely adjusting the focus and continuously monitoring the image clarity, the method can ensure that the microscopic imaging system is in the best focal position as quickly as possible. The functions of automatically adjusting the focus and evaluating the clarity reduce the operator's workload and human errors in operation, and improve the degree of automation of the imaging process. Through partition processing and clarity evaluation, the system can analyze the image quality of each field of view in detail and automatically select the clearest field of view. This optimization method significantly improves the quality of the final image. The present invention adapts to changing observation conditions by real-time monitoring and rapid adjustment of the focus, so that it can quickly adapt to different environments and sample changes, ensuring that the imaging quality is always optimal. The partition processing method enables the system to not only provide an overall high-definition image, but also pay attention to details when necessary, especially in overlapping partitions by finely adjusting the weighting factors to optimize the processing of image edges and seam areas.

[0141] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A focal plane detection system for an infinity microscope system, characterized in that: It comprises an objective lens (1), a beam splitter (2), a first tube lens (3), an image acquisition module (4) and an image processing module (5), wherein: The objective lens (1) is used to collect light reflected by an object located on the object focal plane and convert it into parallel light to be emitted to infinity; The beam splitter (2) is arranged behind the objective lens (1) and is used to separate the parallel light into two parts and guide them to the first tube lens (3) and the image acquisition module (4) respectively; The first tube lens (3) is arranged in the direction of the projected light of the beam splitter (2), and is used to receive part of the projected light separated by the beam splitter (2), and converge it onto the first image focal plane; The image acquisition module (4) comprises a second tube lens (41) and a digital camera photosensitive chip (42) which are sequentially arranged in the direction of the reflected light of the beam splitter (2); the second tube lens (41) is used to receive another part of the reflected light separated by the beam splitter (2) and focus it on a second image focal plane; the digital camera photosensitive chip (42) is tiltedly arranged on the second image focal plane; the digital camera photosensitive chip adopts a tilted design, so that only light in a specific field of view can be focused on the chip, and other fields of view are defocused due to the tilt of the chip; The image processing module (5) is used to receive a digital image signal transmitted by a digital camera photosensitive chip (42), and perform image partition processing, and perform image processing on the field of view image obtained after the partition processing, wherein the image processing includes using a preset algorithm to evaluate the image clarity and draw a clarity curve diagram, confirming the position corresponding to the imaging partition with the highest clarity through the clarity curve diagram, calculating the image plane defocus amount through the relationship between the imaging partition position and the tilt angle of the digital camera photosensitive chip (42), and calculating the object plane defocus amount based on the relationship between the image plane defocus amount and the object plane defocus amount with respect to the magnification to determine the optimal focal plane position.

2. A method for detecting the focal plane of an infinite microscope system, based on the system for detecting the focal plane of an infinite microscope system according to claim 1, characterized in that: The following steps are involved: Light collection step: using an objective lens (1) to collect light reflected from an object on the object focal plane, and converting it into parallel light that is emitted to infinity; Light separation step: using a beam splitter (2) to separate the light in the parallel light path into two beams, a projected light beam and a reflected light beam; First tube lens processing step: the first tube lens (3) receives the projection light distributed by the beam splitter (2), and focuses the projection light on the image side focal plane of the first tube lens (3); A second tube lens processing step: focusing the reflected light onto a digital camera photosensitive chip (42) through a second tube lens (41), wherein the reflected light includes a plurality of sub-field light beams; Image acquisition step: the sub-viewing field light beams are projected onto the digital camera photosensitive chip (42) through the second tube lens (41), each sub-viewing field light beam forms a corresponding image on the digital camera photosensitive chip (42), the multiple view fields respectively cover different areas of the digital camera photosensitive chip (42), the digital camera photosensitive chip (42) is tilted so that one of the view fields is focused on the digital camera photosensitive chip (42), and the remaining view fields are all out of focus, and the digital camera photosensitive chip (42) transmits the digital image signal to the image processing module (5); Partition processing step: performing regional segmentation on the digital camera photosensitive chip (42) according to different resolutions, and segmenting into coarse equal partitions and fine equal partitions respectively, wherein the coarse equal partition is larger than the fine equal partition, and performing regional segmentation on the digital camera photosensitive chip (42) with a resolution between the coarse equal partition and the fine equal partition to form overlapping partitions, wherein the overlapping partitions cover adjacent coarse equal partitions and fine equal partitions; Image processing step: The image processing module (5) uses a preset algorithm to evaluate the clarity of each field of view on the partitioned image obtained after the partitioning process, and draws a clarity curve diagram. The position of the field of view with the highest clarity is confirmed by the clarity curve diagram, and the position of the partition corresponding to the field of view is determined. If the peak position of the image clarity curve is on the central field of view, the position of the object plane is the focal plane of the objective lens (1). If the peak position of the image clarity curve is not on the central field of view, the image plane defocus amount is obtained by the relationship between the partition position and the tilt angle of the digital camera photosensitive chip (42). The object plane defocus amount is calculated according to the relationship between the image plane defocus amount and the object plane defocus amount with respect to the magnification. The objective lens (1) is moved a corresponding distance according to the solved object plane defocus amount, and the preset algorithm is used again to evaluate the clarity of each field of view. The above process is repeated until the peak position of the image clarity curve is on the central field of view. At this time, the position of the object plane is the focal plane of the objective lens (1).

3. The method for detecting the focal plane of an infinite microscope system according to claim 2, characterized in that: In the image processing step, the image plane defocus amount is obtained by the relationship between the partition position and the tilt angle of the digital camera photosensitive chip (42), which is expressed as d 像 =L s × sinθ, where L s is the field of view position, and θ is the tilt angle of the digital camera photosensitive chip (42).

4. The method for detecting the focal plane of an infinity microscope system according to claim 3, characterized in that: In the image processing step, the object plane defocus amount is calculated based on the relationship between the image plane defocus amount and the object plane defocus amount with respect to the magnification, and is expressed as d 物 =d 像 ÷β 2 , where d 像 is the image plane defocus amount, and β is the magnification of the system.

5. The method for detecting the focal plane of an infinity microscope system according to claim 2, characterized in that: In the image processing step, the image processing module (5) uses a preset algorithm to evaluate the clarity of the partitioned image obtained after the partitioning process. The preset algorithm uses a Laplacian algorithm, wavelet transform, Fourier transform or contrast method to evaluate the clarity of the image in the partition.

6. The method for detecting the focal plane of an infinity microscope system according to claim 2, characterized in that: In the image processing step, the image processing includes using a variational autoencoder model VAE_c to process the image data in the coarse partition, and the variational autoencoder model VAE_c includes an encoder and a decoder The encoder is represented as: Among them, μ c (x c )and Generated by the encoder network to map the input image x c To the latent space z c ; The decoder is represented as: For the latent space z c Reconstruct image x c , optimized to capture a wide range of features of an image; The image processing also includes using a variational autoencoder model VAE_f to process image data in the fine partitions, and the variational autoencoder model VAE_c includes an encoder and a decoder The encoder is represented as: Among them, μ f (x f )and Generated by the encoder network to map the input image x c To the latent space z c ; The decoder is represented as: For the latent space z f Reconstruct image x f , improve the details and texture accuracy of the image; In overlapping partitions, the model outputs of the coarse and fine partitions use the weighted sum method to obtain the information of the edge overlap, expressed as: combined =ω c μ′ c (z c )+ω f μ′ f (z f ), where ω c and ω f are the decoding output weight factors of the coarse and fine partitions respectively.

7. The method for detecting the focal plane of an infinity microscope system according to claim 6, characterized in that: The network of the coarse equally partitioned variational autoencoder includes at least three convolutional layers, the convolution kernel size is 5x5, and the number of channels of the convolutional layers is 32, 64 and 128 respectively; the convolution operation of the lth layer in the variational autoencoder model VAE_f is expressed as: in is the convolution kernel of layer l, is the bias term; The network of the coarse equally partitioned variational autoencoder includes at least five convolutional layers, the convolution kernel size is 3x3, and the number of channels of the convolutional layers is 64, 128, 256 and 512 respectively; the convolution operation of the lth layer in the variational autoencoder model VAE_c is expressed as: in is the convolution kernel of layer l, is the bias term.

8. The method for detecting the focal plane of an infinity microscope system according to claim 6, characterized in that: The weight factor ω c and ω f Use gradient descent to optimize to minimize the reconstruction error after fusion, which includes the following steps: Objective function definition: Define the objective function J(ω) of the image reconstruction error after fusion c ,ω f ), specifically expressed as: J(ω c ,oh f )=||ω c ·m′ c (z c )+ω f ·m′ f (z f )-x true || 2 Among them, μ ′ c (z c ) and μ ′ f (z f ) are the decoded outputs from the coarse and fine partition variational autoencoder models, respectively, x true is the target image, and the goal is to find ω that minimizes the reconstruction error c and ω f . Gradient calculation: Calculate the objective function with respect to ω c and ω f The gradient is: Among them, the gradient and Denotes J with respect to ω c and ω f sensitivity to change; The rule for updating the weight factor using the gradient descent method is expressed as: Among them, η is the learning rate, which is used to control the update step size; Initialize the weight factor ω c and ω f , expressed as ω c =ω f = 0.5; Apply the gradient descent method to update the weight factor rule for multiple iterations, each iteration using a new set of μ obtained from the data set ′ f (z f ), μ ′ c (z c ) and x true To calculate the gradient and update the weights, iterate until the reconstruction error J(ω c ,ω f ) reaches the preset value.

9. The method for detecting the focal plane of an infinity microscope system according to claim 5, characterized in that: In the image processing step, the image plane defocus is determined according to the tilt angle and field of view position of the digital camera photosensitive chip, and then the object plane defocus is calculated, and the focal length is adjusted in real time through the machine learning model to optimize the imaging effect; the establishment of the machine learning model specifically includes the following steps: Training data preparation: Collect images with different focal length settings and their corresponding clarity scores. The clarity scores are obtained using the Laplacian algorithm and are expressed as: Among them, x i represents the i-th image, y i is the corresponding focal length setting, d i is the clarity score obtained by the Laplacian algorithm; Model training: Using the regression model f(y i ; θ) to learn the relationship between focal length y and image clarity d, where θ is a model parameter; the least squares method is used to train the regression model to minimize the difference between the predicted clarity and the actual clarity, expressed as: Real-time clarity assessment: Capture images in real time and assess their clarity, expressed as: d current =Laplacian(capture()) Use the optimization algorithm to make slight adjustments to the focal length to find the best focus, expressed as: Among them, α is the learning rate, Represents the gradient of the objective function with respect to the focal length, which is used to update the current focal length to improve image clarity.

10. The method for detecting the focal plane of an infinity microscope system according to claim 9, characterized in that: The steps to obtain the clarity score through the Laplacian algorithm include: Convolution operation: Apply the Laplacian kernel to the original image I and calculate the Laplacian value of each pixel, expressed as: I Laplacian =L*I, where * represents the convolution operation and I represents the original image; Calculate variance: The sharpness can be evaluated by calculating the variance of the Laplacian image, expressed as: Sharpness=Was(I Laplacian ) The variance is calculated as: where x i isI Laplacian The pixel value in , μ is x i is the average value of , where n is the total number of pixels.

Images