System and method for infinity microscope system focal plane detection

Abstract

The application relates to a system for detecting the focal plane of an infinite microsystem, which comprises an objective lens, a beam splitter, a tube lens, an image acquisition module and an image processing module. The objective lens collects the light reflected by an object and converts the light into parallel light, the beam splitter separates the parallel light into two parts which are transmitted to the tube lens and the image acquisition module respectively. The first tube lens receives part of the light and focuses the light on the focal plane of the image side, the second tube lens in the image acquisition module receives another part of the light and focuses the light on the photosensitive chip of a digital camera, the photosensitive chip of the digital camera is installed obliquely to ensure that the light of a specific field of view is focused. The image processing module processes the digital image by partitioning and analyzes the definition curve diagram to calculate the defocus amount of the object plane and determine the position of the best focal plane.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of microscope technology, more particularly, to a system and method for focus detection of an infinity microscope system. BACKGROUND

[0002] With the development of optical technology, auto-focusing technology is increasingly applied to imaging systems. Although the theory and method of auto-focusing technology develop rapidly, the passive method for auto-focusing requires multiple image acquisition and analysis processing time for image sharpness, which is too long. A new fast and efficient focus detection method needs to be found to reduce the focus detection time and improve the focusing speed. SUMMARY

[0003] The present application provides a system for focus detection of an infinity microscope system, which can improve the financial ability and acceptance rate of customers, reduce the risk coefficient, and inversely improve the financial ability and acceptance rate of customers to find a yield balance point and improve the business profitability.

[0004] The present application provides a system for focus detection of an infinity microscope system, which includes an objective lens, a beam splitter, a first tube lens, an image acquisition module, and an image processing module, wherein,

[0005] The objective lens is used to collect light reflected by an object located on the object-side focal plane and convert it into parallel light to be projected to infinity.

[0006] The beam splitter is arranged behind the objective lens and is used to separate the parallel light into two parts and guide them to the first tube lens and the image acquisition module, respectively.

[0007] The first tube lens is arranged in the projection light direction of the beam splitter and is used to receive part of the projection light separated by the beam splitter and converge it to the first image-side focal plane.

[0008] The image acquisition module includes a second tube lens and a digital camera photosensitive chip arranged in the reflection light direction of the beam splitter in sequence. The second tube lens is used to receive another part of the reflection light separated by the beam splitter and focus it to the second image-side focal plane. The digital camera photosensitive chip is arranged obliquely on the second image-side focal plane. The digital camera photosensitive chip adopts an oblique design, and only light of a specific field of view can be focused on the chip, and other fields of view are out of focus due to the oblique design of the chip.

[0009] The image processing module is used for receiving a digital image signal transmitted by a digital camera photosensitive chip, and performing partition processing on the image, and performing image processing on a field of view image obtained after the partition processing, the image processing including using a preset algorithm to perform image sharpness evaluation and draw a sharpness curve graph, confirming a position corresponding to an imaging partition with the highest sharpness through the sharpness curve graph, calculating an image plane defocus amount through a relationship between the imaging partition position and a tilt angle of the digital camera photosensitive chip, and calculating an object plane defocus amount according to a relationship among the image plane defocus amount, the object plane defocus amount and a magnification to determine a best focus plane position.

[0010] The application further provides a method for detecting a focus plane of an infinity microscopic system, based on the system for detecting the focus plane of the infinity microscopic system, and characterized by comprising the following steps:

[0011] Light collection step: using an objective lens to collect light reflected by an object located on an object focus plane, and converting the light into parallel light to be projected to infinity;

[0012] Light separation step: using a beam splitter to separate the light in the parallel light path into two beams of projection light and reflection light;

[0013] First tube lens processing step: the first tube lens receives the projection light distributed by the beam splitter, and focuses the projection light on an image focus plane of the first tube lens;

[0014] Second tube lens processing step: the reflection light is focused to a digital camera photosensitive chip through the second tube lens, and the reflection light includes a plurality of sub-field of view beams;

[0015] Image collection step: the sub-field of view beams are projected onto the digital camera photosensitive chip through the second tube lens, each sub-field of view beam forms a corresponding image on the digital camera photosensitive chip, a plurality of fields of view cover different areas of the digital camera photosensitive chip, the digital camera photosensitive chip is arranged to be inclined, so that one field of view is in focus on the digital camera photosensitive chip, and the remaining fields of view are out of focus, and the digital camera photosensitive chip transmits a digital image signal to an image processing module;

[0016] Partition processing step: the digital camera photosensitive chip is divided into regions according to different resolutions, and a coarse equal partition and a fine equal partition are divided, the coarse equal partition is larger than the fine equal partition, and a region of the digital camera photosensitive chip is divided at a resolution between the coarse equal partition and the fine equal partition to form an overlapping partition, and the overlapping partition covers adjacent coarse equal partitions and fine equal partitions;

[0017] Image processing steps: The image processing module uses a preset algorithm to evaluate the sharpness of each field of view in the partitioned image after partitioning, and plots a sharpness curve. The sharpness curve is used to determine the location of the field of view with the highest sharpness and the location of the corresponding partition. If the peak position of the image sharpness curve is on the center field of view, the location of the object plane is the focal plane of the objective lens. If the peak position of the image sharpness curve is not on the center field of view, the image plane defocus is calculated based on the relationship between the partition position and the tilt angle of the digital camera's image sensor. Based on the relationship between the image plane defocus, the object plane defocus, and the magnification, the object plane defocus is calculated. The objective lens is moved a corresponding distance according to the calculated object plane defocus, and the sharpness of each field of view is evaluated again using the preset algorithm. The above process is repeated until the peak position of the image sharpness curve is on the center field of view, at which point the location of the object plane is the focal plane of the objective lens.

[0018] Preferably, in the image processing step, the image plane defocusing amount is determined by the relationship between the partition position and the tilt angle of the digital camera's image sensor, expressed as: ,in It is the position of the field of view. It refers to the tilt angle of the digital camera's image sensor.

[0019] Preferably, in the image processing step, the object plane defocus is calculated based on the relationship between the image plane defocus, the object plane defocus, and the magnification, and is expressed as follows: ,in β is the defocusing amount of the image plane, and β is the magnification of the system.

[0020] Preferably, in the image processing step, the image processing module uses a preset algorithm to evaluate the sharpness of the partitioned images obtained after partitioning. The preset algorithm is to use the Laplacian algorithm, wavelet transform, Fourier transform, or contrast method to evaluate the image sharpness in the partition.

[0021] Preferably, in the image processing step, the image processing includes processing the image data in the coarse partitions using a variational autoencoder model VAE_c, wherein the variational autoencoder model VAE_c includes an encoder. and a decoder ,

[0022] The encoder is represented as: ,in, and Generated by an encoding network, used to map the input image. To potential space ;

[0023] The decoder is represented as: , used from potential space Reconstructed image , the optimization is to capture the extensive features of the image, wherein, represents the pixel mean distribution of the reconstructed image generated by the decoder network according to the latent vector , represents the pixel variance distribution output by the decoder network according to the latent vector ;

[0024] The image processing further comprises processing the image data in the fine sub-zones by a variational autoencoder model VAE_f, the variational autoencoder model VAE_f comprising an encoder and a decoder ,

[0025] The encoder is represented as: wherein, and are generated by the encoding network for mapping the input image to the latent space ;

[0026] The decoder is represented as: for reconstructing the image from the latent space , improving the detail and texture accuracy of the image;

[0027] In the overlapping zones, the model outputs of the coarse sub-zones and the fine sub-zones use a weighted sum method to obtain the information of the edge overlapping part, represented as: wherein, and are the decoding output weight factors of the coarse sub-zones and the fine sub-zones, respectively.

[0028] Preferably, the network of the coarse sub-zone variational autoencoder comprises at least three convolutional layers, the convolution kernel size is 5x5, and the number of channels of the convolutional layers is 32, 64 and 128 in turn; the convolution operation of the first layer in the variational autoencoder model VAE_f is represented as: wherein is the convolution kernel of the first layer, is the bias term;

[0029] The network of the coarse sub-zone variational autoencoder comprises at least five convolutional layers, the convolution kernel size is 3x3, and the number of channels of the convolutional layers is 64, 128, 256 and 512 in turn; the convolution operation of the first layer in the variational autoencoder model VAE_c is represented as: wherein is the convolution kernel of the first layer, is a bias term.

[0030] Preferably, the weight factor and is optimized using gradient descent to minimize the reconstruction error of the fused image, including the following steps:

[0031] Objective function definition: define the objective function of the reconstruction error of the fused image , which is specifically expressed as:

[0032]

[0033] wherein, and are the decoding outputs from the coarse and fine partition variational autoencoder models respectively, is the target image, and the goal is to find and that minimize the reconstruction error.

[0034] Gradient calculation: calculate the gradient of the objective function with respect to and :

[0035]

[0036]

[0037] wherein, the gradients and represent the sensitivity of with respect to and when they change;

[0038] The rule for updating the weight factor using gradient descent is expressed as:

[0039]

[0040]

[0041] wherein, is the learning rate, used to control the update step size;

[0042] Initialize the weight factors and , expressed as ; apply the rule for updating the weight factor using gradient descent for multiple iterations, and use a new set of , and from the data set to calculate the gradient and update the weight in each iteration, until the reconstruction error The preset value has been reached.

[0043] Preferably, in the image processing step, the image plane defocus amount is determined based on the tilt angle and field of view position of the digital camera's image sensor, then the object plane defocus amount is calculated, and the focal length is adjusted in real time using a machine learning model to optimize the imaging effect; the establishment of the machine learning model specifically includes the following steps:

[0044] Training data preparation: Collect images and their corresponding sharpness scores at different focal length settings. The sharpness scores are obtained using the Laplacian algorithm and are represented as follows:

[0045]

[0046] in, Indicates the first One image, This corresponds to the focal length setting. The sharpness score is obtained through the Laplacian algorithm;

[0047] Model training: using a regression model Let's learn about focal length and image clarity The relationship between them, These are the model parameters; the least squares method is used to train the regression model, minimizing the difference between predicted sharpness and actual sharpness, expressed as:

[0048]

[0049] Real-time sharpness assessment: This involves capturing images in real time and evaluating their sharpness, expressed as follows:

[0050]

[0051] The optimal focus is found by fine-tuning the focal length using an optimization algorithm, as shown below:

[0052]

[0053] in, It's the learning rate. This represents the gradient of the objective function with respect to the focal length, used to update the current focal length to improve image sharpness.

[0054] Preferably, the steps for obtaining a sharpness score using the Laplacian algorithm include:

[0055] Convolution operation: Applying a Laplacian kernel to the original image Above, calculate the Laplacian value for each pixel, expressed as: ,in, This represents the convolution operation. Represents the original image;

[0056] Calculating the variance: Sharpness can be assessed by calculating the variance of a Laplacian image, expressed as:

[0057]

[0058] The variance is calculated as follows:

[0059]

[0060] in yes Pixel values ​​in yes The average value, It represents the total number of pixels.

[0061] The beneficial effects of this invention are as follows: This invention splits parallel light into two beams—a projected beam and a reflected beam—using a beam splitter, and processes them independently through a first and second lens, increasing the system's imaging flexibility. The independent operation of the first and second lenses allows the system to simultaneously optimize focusing and image quality on different imaging paths, adapting to a wider range of application needs. Each sub-field of view beam emanating from the beam splitter corresponds to a coarse equal partition on the image sensor, with overlapping areas at the edges of the coarse equal partitions serving as fine equal partitions. This design not only enhances the focusing accuracy of the image but also ensures the integrity of image information, especially in edge areas, effectively reducing information loss and improving imaging continuity and detail. Through the tilted design of the digital camera's image sensor, the system can achieve optimal focusing on specific fields of view, while other fields of view are adjusted appropriately as needed. This design allows the system to adjust the focus according to different imaging requirements, making it suitable for various occasions requiring high-precision focusing and wide field-of-view coverage. The system utilizes a combination of image sharpness evaluation techniques, including the Laplacian algorithm, wavelet transform, Fourier transform, and contrast method, through its image processing module to accurately assess the image quality of each region and precisely control the focal plane using a sharpness curve. By calculating and adjusting the relationship between image plane defocus and object plane defocus, the system can dynamically adjust imaging parameters to ensure optimal focal plane position at different magnifications. Integrating image processing algorithms and autofocus technology, the system automatically analyzes and adjusts imaging parameters, significantly reducing operational complexity and improving operational efficiency and imaging speed. This is particularly beneficial for scientific research and industrial applications requiring rapid, efficient, and frequent adjustments to focal length and imaging parameters. Therefore, this invention enhances the performance and application value of microscopy systems, enabling them to play a vital role in multiple fields such as scientific research, medical diagnosis, and industrial inspection, meeting the imaging challenges of higher precision and broader requirements. Attached Figure Description

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the present application will be further described below with reference to the drawings and embodiments; the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings:

[0063] Figure 1 is a focusing embodiment one schematic diagram of a system for focus detection of an infinity microscopic system according to an embodiment of the present application;

[0064] Figure 2 is a clarity curve diagram of a system for focus detection of an infinity microscopic system according to an embodiment of the present application;

[0065] Figure 3 is a focusing embodiment two schematic diagram of a system for focus detection of an infinity microscopic system according to an embodiment of the present application;

[0066] Figure 4 is a clarity curve diagram of a system for focus detection of an infinity microscopic system according to an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely below; obviously, the described embodiments are some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0068] As shown in Figures 1-4 , the present application provides a system for focus detection of an infinity microscopic system, comprising an objective lens 1, a beam splitter 2, a first tube lens 3, an image acquisition module 4 and an image processing module 5, wherein,

[0069] The objective lens 1 is used for collecting light reflected by an object located on an object focal plane and converting the light into parallel light to be projected to infinity;

[0070] The beam splitter 2 is arranged behind the objective lens 1 and is used for separating the parallel light into two parts and guiding the two parts to the first tube lens 3 and the image acquisition module 4 respectively;

[0071] The first tube lens 3 is arranged in the projection light direction of the beam splitter 2 and is used for receiving part of the projection light separated by the beam splitter 2 and converging the projection light onto a first image focal plane;

[0072] The image acquisition module 4 includes a second tube mirror 41 and a digital camera image sensor chip 42, which are sequentially arranged in the direction of the reflected light of the beam splitter 2. The second tube mirror 41 is used to receive another part of the reflected light separated by the beam splitter 2 and focus it onto the second image-side focal plane. The digital camera image sensor chip 42 is tilted and arranged on the second image-side focal plane. The digital camera image sensor chip adopts a tilted design, so that only the light in a specific field of view can be focused on the chip, while other fields of view will be out of focus due to the tilt of the chip.

[0073] The image processing module 5 is used to receive the digital image signal transmitted by the digital camera image sensor chip 42 and perform image partitioning. The resulting field of view image after partitioning is processed by image processing. The image processing includes using a preset algorithm to evaluate the image sharpness and draw a sharpness curve. The sharpness curve is used to determine the position of the imaging partition with the highest sharpness. The image plane defocus is obtained by the relationship between the imaging partition position and the tilt angle of the digital camera image sensor chip 42. Based on the relationship between the image plane defocus and the object plane defocus with respect to magnification, the object plane defocus is calculated to determine the optimal focal plane position.

[0074] like Figures 1-4 As shown, the reflected light formed after the parallel light passes through the beam splitter 2 is divided into several sub-field beams, which enter the second tube mirror 41 and are focused onto the digital camera's image sensor chip 42.

[0075] It also includes a method for detecting the focal plane of an infinity-based microscopy system, based on the aforementioned system for detecting the focal plane of an infinity-based microscopy system, comprising the following steps:

[0076] Light acquisition steps: Use objective lens 1 to collect the light reflected from the object located on the object-side focal plane, and convert it into parallel light that is directed to infinity;

[0077] Light separation step: Use beam splitter 2 to separate the light rays in the parallel light path into two beams: the projected light and the reflected light;

[0078] First tube mirror processing steps: The first tube mirror 3 receives the projected light distributed by the beam splitter 2 and focuses the projected light on the image-side focal plane of the first tube mirror 3;

[0079] Second tube lens processing step: The reflected light is focused onto the digital camera image sensor chip 42 through the second tube lens 41. The reflected light includes several sub-field beams.

[0080] The image acquisition step: the sub-field of view beams are projected onto the digital camera photosensitive chip 42 through the second tube mirror 41, each sub-field of view beam forms a corresponding image on the digital camera photosensitive chip 42, the multiple fields of view respectively cover different areas of the digital camera photosensitive chip 42, the digital camera photosensitive chip 42 is obliquely arranged, so that one field of view is in focus on the digital camera photosensitive chip 42, and the remaining fields of view are all out of focus, and the digital camera photosensitive chip 42 transmits a digital image signal to the image processing module 5.

[0081] The partition processing step: the digital camera photosensitive chip 42 is regionally divided according to different resolutions, and a coarse equal partition and a fine equal partition are respectively divided out, the coarse equal partition is larger than the fine equal partition, and the digital camera photosensitive chip 42 is regionally divided at a resolution between the coarse equal partition and the fine equal partition, to form an overlapping partition, and the overlapping partition covers adjacent coarse equal partitions and fine equal partitions.

[0082] The image processing step: the image processing module 5 uses a preset algorithm to evaluate the sharpness of each field of view of the partitioned image, draws a sharpness curve, confirms the position of the field of view with the maximum sharpness through the sharpness curve, and determines the position of the partition corresponding to the field of view, if the peak position of the image sharpness curve is on the central field of view, the position of the object plane is the focal plane of the objective lens 1, if the peak position of the image sharpness curve is not on the central field of view, the defocus amount of the image plane is obtained through the relationship between the partition position and the tilt angle of the digital camera photosensitive chip 42, the defocus amount of the object plane is calculated according to the relationship among the defocus amount of the image plane, the defocus amount of the object plane and the magnification, the defocus amount of the object plane is moved by a corresponding distance, and the sharpness of each field of view is evaluated again using the preset algorithm, and the above process is repeated until the peak position of the image sharpness curve is on the central field of view, and the position of the object plane is the focal plane of the objective lens 1.

[0083] In the image processing step, the defocus amount of the image plane is obtained through the relationship between the partition position and the tilt angle of the digital camera photosensitive chip 42, and is expressed as .

[0084] In the image processing step, the defocus amount of the object plane is calculated according to the relationship among the defocus amount of the image plane, the defocus amount of the object plane and the magnification, and is expressed as , wherein β is the magnification of the system.

[0085] The present application increases the imaging flexibility of the system by splitting the parallel light into two beams, projection light and reflected light, and processing them independently through the first tube lens and the second tube lens. The independent operation of the first tube lens and the second tube lens allows the system to optimize focus and image quality simultaneously on different imaging paths, adapting to a wider range of application requirements. Each sub-field-of-view beam emitted from the splitting lens corresponds to a coarse division zone on the photosensitive chip, and an overlapping area is designed at the edge of the coarse division zone as a fine division zone. This design not only enhances the focusing accuracy of the image, but also ensures the integrity of the image information, especially in the edge area, effectively reducing information loss and improving the continuity and detail performance of the image. Through the tilt design of the digital camera photosensitive chip, the system can achieve optimal focus on a specific field of view, while other fields of view are adjusted as needed. This design allows the system to adjust the focus point according to different imaging requirements, suitable for a variety of occasions that require high-precision focusing and wide field coverage. The system uses a variety of image sharpness evaluation techniques such as Laplacian algorithm, wavelet transform, Fourier transform, or contrast method through the image processing module to accurately evaluate the image quality of each division zone, and accurately control the focusing surface through the sharpness curve. By calculating and adjusting the relationship between the image plane defocus and the object plane defocus, the system can dynamically adjust the imaging parameters to ensure that the best focus position is obtained at different magnifications. The system integrates image processing algorithms and automatic focusing technology, which can automatically analyze and adjust imaging parameters, greatly reducing the operation difficulty and improving the operation efficiency and imaging speed. This is particularly beneficial for scientific research and industrial applications that require fast, efficient, and frequent adjustment of focal length and imaging parameters. Therefore, it can be said that the present application improves the performance and application value of the microscopic system, enabling it to play an important role in scientific research, medical diagnosis, industrial detection, and other fields, meeting the imaging challenges of higher precision and wider demand.

[0086] According to the above scheme, embodiment 1 of the present application is a successful focusing case, as shown in Figure 1

[0087] System configuration and operation process:

[0088] Objective lens: located at the front end of the system, used to capture the light reflected by the object located on its object plane focal surface, and convert it into a parallel light beam directed to infinity.

[0089] The splitting mirror is configured after the objective lens, responsible for splitting the parallel light into two parts, one part is guided to the first tube lens, and the other part is guided to the image acquisition module. The first tube lens receives the part of the projection light allocated by the splitting mirror and focuses it on the image plane focal surface of the tube lens. The second tube lens receives the reflected light from the splitting mirror and focuses it accurately on the photosensitive chip. Since the digital camera photosensitive chip is tilted, Figure 1 ​In the embodiment 1, only the central field of view light is focused on the center of the digital camera photosensitive chip after the tilt processing, and other fields of view are all out of focus.

[0090] The digital camera photosensitive chip is divided into a plurality of partitions, and the digital camera photosensitive chip is divided into 5*5 small windows as a coarse partition. The digital camera photosensitive chip can be divided into 2*2 small windows as a fine partition. The digital camera photosensitive chip is divided into 3*3 small windows as an overlapping partition. The central field of view is field of view 2.

[0091] Under the standard operation condition, the object is accurately placed on the focal plane of the objective lens, and the light emitted from the objective lens is completely parallel. The parallel light is correctly distributed by the beam splitter, and the second tube lens is accurately adjusted to ensure that the central field of view light is focused on the center of the photosensitive chip. Figure 2 is corresponding to Figure 1 The resolution curves of the images collected under different degrees of resolution are drawn, the abscissa is the field of view position, and the ordinate is the image resolution. Figure 2 The field of view with the maximum image resolution is the position of field of view 2 (i.e. the central field of view), and the position corresponding to the partition is confirmed. The image processing module receives the clear image data, and the object surface defocus amount is calculated to be zero, and no additional adjustment is required.

[0092] The embodiment 2 of the present application is a focusing adjustment process. Figure 3 The object is not on the focal plane of the objective lens, and the light passes through the beam splitter to the image acquisition module. Figure 3 In the embodiment 2, only the light of 1 field of view is focused on the digital camera photosensitive chip after the tilt processing, and other fields of view are all out of focus.

[0093] System adjustment process:

[0094] After the light passes through the beam splitter, part of the light fails to be accurately focused on the predetermined position of the photosensitive chip when being focused by the second tube lens. The image data received by the image processing module shows that the focal point is moved forward or part of the field of view is out of focus. Figure 4 is corresponding to Figure 3 The resolution curves of the images collected under different degrees of resolution are drawn, the abscissa is the field of view position, and the ordinate is the image resolution. Figure 4The position corresponding to the partition is determined by the relationship between the partition position and the tilt angle of the digital camera photosensitive chip, and the image plane defocus amount is calculated, and according to the relationship between the image plane defocus amount and the object plane defocus amount and the magnification, the object plane defocus amount can be calculated. Through real-time image sharpness evaluation, the image processing module calculates the image plane defocus amount, and calculates the object plane defocus amount by using the image plane defocus amount and the magnification of the system. According to the object plane defocus amount, the system automatically adjusts the focal length setting to correct the focusing deviation and realize focusing optimization.

[0095] Through the two embodiments, it can be seen that the system not only can realize accurate focusing under ideal conditions, but also has the ability to correct focusing deviation under non-ideal conditions through algorithm and automatic adjustment mechanism, greatly improving the practicality and flexibility of the system.

[0096] In the image processing step in the embodiment, the image processing includes processing image data in the coarse equal partition using a variational autoencoder model VAE_c, the variational autoencoder model VAE_c includes an encoder and a decoder ,

[0097] The encoder is represented as: wherein, and are generated by the encoding network for mapping the input image to the latent space ;

[0098] The decoder is represented as: for reconstructing the image from the latent space , which is optimized to capture extensive features of the image, wherein, represents the pixel mean distribution generated by the decoder network according to the latent vector for reconstructing the image , represents the pixel variance distribution output by the decoder network according to the latent vector ;

[0099] The image processing further includes processing image data in the fine equal partition using a variational autoencoder model VAE_f, the variational autoencoder model VAE_f includes an encoder and a decoder ,

[0100] The encoder is represented as: wherein, and are generated by the encoding network for mapping the input image to the latent space ;

[0101] The decoder is represented as: , for reconstructing the image from the latent space , to improve the detail and texture accuracy of the image;

[0102] In the overlapping partition, the model outputs of the coarse partition and the fine partition use a weighted sum method to obtain the information of the edge overlapping part, represented as: , where and are the decoding output weight factors of the coarse partition and the fine partition, respectively.

[0103] The VAE model of the coarse partition (VAE_c) can capture the macro features and extensive structures of the image, helping to maintain the overall consistency and context information of the image. The VAE model of the fine partition (VAE_f) focuses on the microscopic details and textures of the image, improving the resolution and detail performance of the image, especially in the edge and detail parts of the image. By fusing the image data from the coarse and fine partitions in the overlapping partition, the representation of the overall structure and local details can be effectively balanced, and the dynamic range of the image can be optimized.

[0104] The present application adjusts the weight factors and in the weighted sum method to achieve optimal image representation in different scenarios, thereby providing the best image quality in different applications.

[0105] In this embodiment, first, the image data to be processed is collected, and according to the position of the image content, it is allocated to the coarse partition or the fine partition.

[0106] For the image of the coarse partition, the encoder of VAE_c is used to map the image to the latent space to capture macro features. For the image of the fine partition, the encoder of VAE_f is used to map the image to the latent space to focus on capturing details and textures. Then the decoder is used to reconstruct the image of the coarse partition from the latent space to optimize the overall structure. The decoder is used to reconstruct the image of the fine partition from the latent space to improve the detail performance. In the overlapping partition, the system uses a weighted sum method to fuse the images of the coarse and fine partitions, adjust and ​to optimize the fusion effect and ensure the natural transition of the edge part. The finally fused image is presented to the user or stored for further analysis. The system can adjust the weight factor and processing parameters according to user feedback or automatic analysis results to adapt to different imaging needs.

[0107] The VAE-based image processing strategy can adapt to different image characteristics, such as different lighting conditions, different imaging device characteristics, and different properties of target objects. By intelligently adjusting the weighting factor and VAE parameters, the system can provide customized image processing services while maintaining high processing speed, meeting the high standards of image quality in scientific research and clinical diagnosis fields.

[0108] In this embodiment, the network of the coarse equal-partition variational autoencoder includes at least three convolutional layers, the convolution kernel size is 5x5, and the number of channels of the convolutional layers is 32, 64, and 128 in turn; the convolution operation of the first layer in the variational autoencoder model VAE_f is represented as: wherein is the convolution kernel of the first layer, is the bias term;

[0109] The network of the coarse equal-partition variational autoencoder includes at least five convolutional layers, the convolution kernel size is 3x3, and the number of channels of the convolutional layers is 64, 128, 256, and 512 in turn; the convolution operation of the first layer in the variational autoencoder model VAE_c is represented as: wherein is the convolution kernel of the first layer, is the bias term.

[0110] In this embodiment, the weight factor and are optimized using gradient descent to minimize the reconstruction error after fusion, specifically including the following steps:

[0111] Objective function definition: define the objective function of the image reconstruction error after fusion , which is specifically represented as:

[0112]

[0113] wherein, and are the decoding outputs from the coarse equal-partition and fine equal-partition variational autoencoder models, is the target image, and the goal is to find and that minimize the reconstruction error.

[0114] Larger convolutional kernels and wider receptive fields in coarse-partitioned models help the model effectively capture overall image features and contextual information while maintaining computational efficiency, making it suitable for rapid analysis of background and structure. Smaller convolutional kernels in fine-partitioned models and multi-layered structures enable the model to deeply analyze the microscopic details of images without adding too much computational burden, especially in applications requiring high precision, such as cell structures and material surfaces in medical and scientific research.

[0115] Different network architectures and parameter settings provide optimized processing paths for different types of image data, enabling the system to flexibly meet a wide range of needs, from broad medical imaging to sophisticated scientific research. The ability to automatically adjust focus and fuse images significantly improves operational efficiency, reduces the need for manual intervention, and enhances overall imaging performance by ensuring optimal processing of images in each partition. Therefore, it provides robust support for real-time image processing, especially in clinical environments requiring rapid and accurate diagnosis, and in scientific research fields with extremely high image quality requirements. In summary, the design of the dual variational autoencoder model not only improves the accuracy and efficiency of image processing but also provides reliable support for demanding imaging applications through intelligent image processing technology.

[0116] Gradient calculation: Calculate the gradient of the objective function with respect to the gradient. and gradient:

[0117]

[0118]

[0119] Where, gradient and express about and Sensitivity to change;

[0120] The rule for updating the weight factors using gradient descent is expressed as follows:

[0121]

[0122]

[0123] in, It is the learning rate, used to control the update step size;

[0124] Initialize weight factors and , as shown The gradient descent method is applied to update the weight factors in multiple iterations, with each iteration using a new set of weights obtained from the dataset. , and to calculate the gradient and update the weights, iterating until the reconstruction error reaches a preset value.

[0125] By fine-tuning the weight factors, the system can effectively balance the image contributions between coarse and fine partitions, enhancing the microscopic details while preserving the macroscopic structure of the image. The model can dynamically adjust the weight factors based on real-time image data, adapting to different imaging conditions and requirements. The automatic optimization process reduces the dependence on manual adjustments by professionals, improving the efficiency and convenience of operation. By using gradient descent and automatic weight adjustment, the system can more effectively use its computational resources. This means that while maintaining or even reducing the computational cost, the speed and quality of image processing can be improved. Combining information from coarse and fine partitions can enhance the image resolution of the system, especially when dealing with images with complex textures and boundaries. This comprehensive use of information from different partitions can better handle noise and inconsistencies in the image, improving the accuracy and usability of the final image.

[0126] In this embodiment, in the image processing step, the amount of defocus of the image plane is determined according to the tilt angle and field position of the digital camera's photosensitive chip, then the amount of defocus of the object plane is calculated, and the focal length is adjusted in real time through a machine learning model to optimize the imaging effect; the establishment of the machine learning model specifically includes the following steps:

[0127] Training data preparation: collect images and their corresponding sharpness scores under different focal length settings, the sharpness scores are obtained through the Laplacian algorithm, represented as:

[0128]

[0129] where, represents the th image, is the corresponding focal length setting, is the sharpness score obtained through the Laplacian algorithm;

[0130] Model training: use a regression model to learn the relationship between the focal length and the image sharpness , where is the model parameter; use the least squares method to train the regression model, minimizing the difference between the predicted sharpness and the actual sharpness, represented as:

[0131]

[0132] Real-time sharpness evaluation: by capturing images in real time and evaluating their sharpness, represented as:

[0133]

[0134] Fine-tuning the focal length using optimization algorithms to find the optimal focus, denoted as:

[0135]

[0136] where, is the learning rate, denotes the gradient of the objective function with respect to the focal length, used to update the current focal length to improve image sharpness.

[0137] By precisely adjusting the focal length to match the real-time calculated sharpness, the system can automatically and accurately find the optimal focus position. Through the automated processing flow, the dependence on operators is significantly reduced, speeding up the imaging process, and reducing imaging quality problems caused by human errors. The system continuously evaluates the sharpness of captured images through the Laplacian algorithm, monitoring image quality in real time. The instant feedback mechanism enables the imaging device to respond quickly to changes, suitable for dynamic or rapidly changing imaging environments. Using the relationship model learned from experimental data, the system can predict and optimize imaging results based on historical data and statistical methods, rather than relying solely on single observations. As more and more imaging data is captured and analyzed, the machine learning model can continuously learn and optimize, improving its prediction accuracy and adaptability. This continuous learning ability makes the system become more intelligent and efficient over time.

[0138] In this embodiment, the steps of obtaining the sharpness score through the Laplacian algorithm include:

[0139] Convolution operation: Apply the Laplacian kernel to the original image (also the image information of the edge overlapping part of the overlapping partition obtained in the foregoing embodiment) to calculate the Laplacian value of each pixel, denoted as: where, denotes the convolution operation, denotes the original image;

[0140] Calculate the variance: The sharpness can be evaluated by calculating the variance of the Laplacian image, denoted as:

[0141]

[0142] The calculation method of the variance is:

[0143]

[0144] where is pixel value in the pixel, is the average value of the pixel, is the average value of the pixel, is the total number of pixels.

[0145] Therefore, by precisely adjusting the focal length and continuously monitoring the image sharpness, this method can ensure that the microscopic imaging system is at the best focal plane position at the fastest speed. The functions of automatically adjusting the focal length and evaluating the sharpness reduce the workload of the operator and human errors in the operation, and improve the degree of automation of the imaging process. Through the sub-area processing and sharpness evaluation, the system can analyze the image quality of each field of view in detail and automatically select the sharpest field of view. This optimization method significantly improves the quality of the final image. The present application can quickly adapt to different environments and sample changes by monitoring and quickly adjusting the focal length in real time, ensuring that the imaging quality is always optimal. The sub-area processing method enables the system not only to provide an overall high-definition image, but also to focus on details when necessary, especially in the overlapping sub-areas by fine-tuning the weighting factor to optimize the processing of image edges and joint areas.

[0146] It should be understood that, for those skilled in the art, improvements or changes can be made according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the present application.

Claims

1. An infinity microscope system focal plane detection system, characterized by, The application relates to a kind of optical imaging systems, comprising objective lens (1), beam splitter (2), first tube lens (3), image acquisition module (4) and image processing module (5), wherein, The objective lens (1) is used for collecting the light reflected by the object located on the object focal plane and converting it into parallel light to shoot at infinity; The beam splitter (2) is arranged behind the objective lens (1) and used for separating the parallel light into two parts and guiding them to the first tube lens (3) and the image acquisition module (4) respectively; The first tube lens (3) is arranged in the direction of the projection light of the beam splitter (2) and used for receiving part of the projection light separated by the beam splitter (2) and converging it onto the first image focal plane; The image acquisition module (4) comprises a second tube lens (41) and a digital camera photosensitive chip (42) arranged in the direction of the reflected light of the beam splitter (2) in sequence, the second tube lens (41) is used for receiving another part of the reflected light separated by the beam splitter (2) and focusing it onto the second image focal plane, and the digital camera photosensitive chip (42) is arranged obliquely on the second image focal plane; the digital camera photosensitive chip adopts an oblique design, only the light of a specific field of view can be focused on the chip, and other fields of view are out of focus due to the oblique arrangement of the chip; The image processing module (5) is used for receiving the digital image signal transmitted by the digital camera photosensitive chip (42) and performing partition processing on the image, and the field of view image obtained after the partition processing is subjected to image processing, the image processing includes using a preset algorithm to evaluate the image definition and draw a definition curve graph, confirming the position corresponding to the imaging partition with the highest definition through the definition curve graph, calculating the image plane defocus amount through the relationship between the imaging partition position and the oblique angle of the digital camera photosensitive chip (42), and calculating the object plane defocus amount according to the relationship among the image plane defocus amount, the object plane defocus amount and the magnification to determine the best focal plane position.

2. A method for infinity microscope system focal plane detection, based on the system for infinity microscope system focal plane detection according to claim 1, characterized in that, The application further discloses a kind of optical imaging methods, comprising the following steps: The light collection step: using the objective lens (1) to collect the light reflected by the object located on the object focal plane and converting it into parallel light to shoot at infinity; The light separation step: using the beam splitter (2) to separate the light in the parallel light path into two beams of projection light and reflected light; The first tube lens processing step: the first tube lens (3) receives the projection light distributed by the beam splitter (2) and focuses the projection light on the image focal plane of the first tube lens (3); The second tube lens processing step: the reflected light is focused to the digital camera photosensitive chip (42) by the second tube lens (41), and the reflected light includes a plurality of sub-field-of-view light beams; The image acquisition step: the sub-field-of-view light beams are projected onto the digital camera photosensitive chip (42) by the second tube lens (41), each sub-field-of-view light beam forms a corresponding image on the digital camera photosensitive chip (42), a plurality of fields of view cover different areas of the digital camera photosensitive chip (42), the digital camera photosensitive chip (42) is arranged obliquely so that one field of view is focused on the digital camera photosensitive chip (42) and the remaining fields of view are out of focus, and the digital camera photosensitive chip (42) transmits the digital image signal to the image processing module (5). The partition processing step: the digital camera photosensitive chip (42) is divided into regions according to different resolutions, and coarse equal partitions and fine equal partitions are divided, wherein the coarse equal partitions are larger than the fine equal partitions, the digital camera photosensitive chip (42) is divided into regions at a resolution between the coarse equal partitions and the fine equal partitions, and overlapping partitions are formed, the overlapping partitions cover adjacent coarse equal partitions and fine equal partitions; The image processing step: the image processing module (5) uses a preset algorithm to evaluate the sharpness of each field of view of the partitioned image, draws a sharpness curve, confirms the position of the field of view with the maximum sharpness, and determines the position of the corresponding partition, if the peak position of the image sharpness curve is on the central field of view, the position of the object plane is the focal plane of the objective lens (1), if the peak position of the image sharpness curve is not on the central field of view, the image plane defocus amount is calculated through the relationship between the partition position and the tilt angle of the digital camera photosensitive chip (42), the object plane defocus amount is calculated according to the relationship between the image plane defocus amount and the object plane defocus amount about the magnification, the objective lens (1) is moved by a corresponding distance according to the calculated object plane defocus amount, and the sharpness of each field of view is evaluated again using the preset algorithm, and the above process is repeated until the peak position of the image sharpness curve is on the central field of view, and the position of the object plane is the focal plane of the objective lens (1).

3. The method of claim 2, wherein the method further comprises: In the image processing step, the image plane defocus is calculated by the relationship between the zone position and the tilt angle of the digital camera image sensor (42), denoted as where is the field position, is the tilt angle of the digital camera image sensor (42).

4. The method of claim 3, wherein the step of detecting the focal plane of the infinity microscope system is performed by a focal plane detector. In the image processing step, the object plane defocus amount is calculated based on the relationship of the image plane defocus amount and the object plane defocus amount with respect to the magnification, and is expressed as wherein is the image plane defocus amount, and β is the magnification of the system.

5. The method of claim 2, wherein the step of detecting the focal plane of the infinity microscope system is performed by a focal plane detector. In the image processing step, the image processing module (5) uses a preset algorithm to evaluate the sharpness of the partitioned image, and the preset algorithm is to evaluate the image sharpness in the partition using Laplacian algorithm, wavelet transform, Fourier transform or contrast method.

6. The method of claim 2, wherein the method further comprises: In the image processing step, the image processing comprises processing the image data in the coarse equalization zone with a variational autoencoder model VAE_c, the variational autoencoder model VAE_c comprising an encoder and a decoder , The encoder is represented as: wherein, and is generated by an encoding network for mapping an input image to the latent space ; The decoder is represented as: , for reconstructing an image from a latent space . , optimized to capture extensive characteristics of the image, wherein represents a pixel mean distribution for reconstructing the image generated by the decoder network from the latent vector . represents a pixel variance distribution output by the decoder network from the latent vector . The image processing further comprises processing the image data in the fine sub-zones with a variational autoencoder model VAE_f, the variational autoencoder model VAE_f comprising an encoder and a decoder , The encoder is represented as: wherein, and is generated by an encoding network for mapping an input image to a latent space ; The decoder is represented as: , used from potential space Reconstructed Image This enhances the detail and texture accuracy of the image; In the overlapping partition, the model outputs of the coarse and fine partitions use a weighted sum approach to obtain information for the edge overlapping part, denoted as: where, and are the decoding output weight factors of the coarse and fine partitions, respectively.

7. The method of claim 6, wherein the method further comprises: The coarse, equal-partition variational autoencoder network comprises at least three convolutional layers with a kernel size of 5x5, and the number of channels in the convolutional layers is 32, 64, and 128 respectively; the variational autoencoder model VAE_f contains the following layers: The convolution operation of a layer is represented as: ,in It is the first The convolution kernel of the layer, It is a bias term; The network of the coarse equal-partition variational autoencoder comprises at least five convolutional layers, the convolution kernel size is 3x3, and the number of channels of the convolutional layers is 64, 128, 256 and 512 in turn; the convolution operation of the first layer in the variational autoencoder model VAE_c is represented as: wherein is the convolution kernel of the first layer, is a bias term.

8. The method of claim 6, wherein the method further comprises: The weight factor And Optimization is performed using gradient descent to minimize the reconstruction error after fusion, specifically including the following steps: Objective function definition: define the objective function of the image reconstruction error after fusion , specifically represented as: ; wherein, and are the decoded outputs from the coarse and fine zonotope variational autoencoder models, respectively, is the target image, and the goal is to find the and that minimize the reconstruction error. Gradient computation: Compute the gradient of the objective function with respect to and the parameters. ; ; wherein the gradient and represents with respect to and sensitivity to changes in The rule for updating the weight factor using gradient descent method is represented as: ; ; wherein, is a learning rate for controlling the update step size; Initialization of the weight factors and are expressed as ; the rule of applying gradient descent to update the weight factors is iterated for several times, and in each iteration a new set of , and are used to calculate the gradient and update the weights, and the iteration is performed until the reconstruction error reaches a preset value.

9. The method of claim 5, wherein the method further comprises: In the image processing step, the image plane defocus amount is determined according to the tilt angle of the digital camera photosensitive chip and the field of view position, then the object plane defocus amount is calculated, and the focal length is adjusted in real time through the machine learning model to optimize the imaging effect; the establishment of the machine learning model specifically includes the following steps: Training data preparation: collect images and their corresponding sharpness scores under different focal length settings, the sharpness scores are obtained by Laplacian algorithm, represented as: ; wherein, represents the first image, is the corresponding focal length setting, is the sharpness score obtained by the Laplacian algorithm; Model training: using a regression model Let's learn about focal length and image clarity The relationship between them, These are the model parameters; the least squares method is used to train the regression model, minimizing the difference between predicted sharpness and actual sharpness, expressed as: ; Real-time evaluation of sharpness: real-time capture of images and evaluation of their sharpness, represented as: ; Fine adjustment of focal length using optimization algorithm to find the best focus, represented as: ; wherein, is a learning rate, denotes a gradient of the objective function with respect to the focal length, used to update the current focal length to improve image sharpness.

10. The method of claim 9, wherein the method further comprises: The steps of obtaining sharpness scores by Laplacian algorithm include: Convolution operation: Apply Laplacian kernel to the original image The Laplacian value of each pixel is calculated and denoted as: wherein, denotes the convolution operation, denotes the original image; Variance calculation: the sharpness can be evaluated by calculating the variance of the Laplacian image, represented as: ; The calculation method of variance is: ; wherein is the pixel value in is the average value of is the total number of pixels.

Images