Image focusing method and device based on fusion frequency domain information, and storage medium

Abstract

The invention relates to the technical field of image processing, in particular to an image focusing method and device based on fusion frequency domain information and a storage medium, and the method comprises the steps: obtaining multiple frames of images of a microscope at different Z-axis positions to form an image sequence; extracting a frequency domain definition index and a spatial domain definition index of each frame of image in the image sequence; the frequency domain definition index and the spatial domain definition index are combined, and a multi-modal definition time sequence feature sequence is constructed; inputting the multi-modal definition time sequence feature sequence into a preset improved long-short-term memory network for prediction so as to obtain a definition change trend of a plurality of frames in the future; a frame corresponding to the maximum definition value is located and predicted according to the definition change trend, and the corresponding Z-axis position is determined as the optimal focus point position; and controlling the Z axis of the microscope to move to the optimal focusing point position to complete focusing. According to the scheme of the invention, the robustness and accuracy of focusing are effectively improved.

Description

Technical Field

[0001] This invention generally relates to the field of image processing technology. More specifically, this invention relates to an image focusing method, apparatus, and storage medium based on fused frequency domain information. Background Technology

[0002] With the widespread application of digital microscopy imaging technology, the sharpness and focus of microscopic images have become a crucial factor restricting imaging quality and detection efficiency. In traditional autofocusing methods, a hill-climbing search strategy is typically employed. This involves acquiring images frame by frame at different focal lengths by sliding the lens or adjusting the Z-axis drive platform, and calculating the sharpness evaluation index (such as Tenengrad gradient, variance, high-frequency energy, etc.) for each frame. Finally, the position corresponding to the maximum sharpness evaluation index is selected as the optimal focusing point.

[0003] However, this traditional focusing method based on iterative search has significant drawbacks in practical applications. First, it requires frame-by-frame calculation of all images in the acquisition sequence, resulting in high computational redundancy and high latency in focusing processing, failing to meet the demands of high-speed automated detection. Second, the traditional method is essentially a passive search, lacking the ability to predict sharpness change trends and estimate the optimal focus position in advance, often requiring repeated oscillations around the peak during the search process. Furthermore, traditional sharpness evaluation functions are susceptible to imaging noise or differences in image structure, easily getting trapped in local extrema under low contrast or complex backgrounds, leading to unstable focusing results.

[0004] In recent years, to overcome the limitations of traditional methods, deep learning techniques, especially time series models such as Long Short-Term Memory (LSTM) networks, have been introduced into the field of autofocus. Existing research shows that the change in image sharpness during the focusing process exhibits a typical "blur-sharp-blur" trend, possessing modelable time series characteristics; at the same time, the frequency domain information of the image (such as high-frequency energy) is strongly correlated with sharpness, making it a more robust evaluation criterion than simple spatial gradients.

[0005] Despite the introduction of deep learning models, existing intelligent focusing techniques still have shortcomings. On one hand, current methods rely on a relatively coarse fusion of frequency domain information and LSTM prediction models, failing to effectively capture the dynamic evolution of image sharpness with focal length, resulting in insufficient accuracy in predicting future sharpness trends. On the other hand, in biomedical microscopy, background areas often contain a large amount of redundant information and noise interference, while target cell regions have complex structures and varied morphologies. Existing deep learning focusing models often lack spatially aware attention mechanisms, failing to automatically distinguish key cell regions from unstructured backgrounds. This makes it difficult for the models to accurately model the focusing trend of the target region, thus limiting the stability and reliability of focusing judgments.

[0006] In summary, the main problems with existing technologies are that traditional autofocusing methods have high computational cost, high latency, and poor anti-interference ability, while existing deep learning-based focusing methods lack frequency domain feature fusion and target area spatial perception, making it impossible to achieve both high efficiency and high precision intelligent focusing in complex microscopic cell imaging scenarios. Summary of the Invention

[0007] To address the technical problems of high computational load, high processing delay, and weak prediction ability in the aforementioned microscopic image focusing methods, this invention provides solutions in the following aspects.

[0008] In a first aspect, the present invention provides an image focusing method based on fused frequency domain information, comprising: acquiring multiple frames of images from a microscope at different Z-axis positions to form an image sequence; extracting frequency domain sharpness indices and spatial domain sharpness indices from each frame of the image sequence; combining the frequency domain sharpness indices and spatial domain sharpness indices to construct a multimodal sharpness temporal feature sequence; inputting the multimodal sharpness temporal feature sequence into a preset improved long short-term memory network for prediction to obtain the sharpness change trend of several future frames, wherein the improved long short-term memory network includes a residual connection structure and an attention mechanism; locating the frame corresponding to the predicted maximum sharpness value according to the sharpness change trend, and determining its corresponding Z-axis position as the optimal focusing point position; controlling the Z-axis of the microscope to move to the optimal focusing point position to complete focusing.

[0009] In one embodiment, the method for extracting the frequency domain sharpness index includes: performing a Fourier transform on each frame of the image to obtain a spectrogram, inputting a bandpass filter into the spectrogram to extract high-frequency regions, and calculating the spectral energy value in the high-frequency regions as the frequency domain sharpness index of the frame of the image.

[0010] In one embodiment, the spatial domain sharpness metric is selected from at least one of the following: the Tenengrad gradient magnitude of the image or the Laplacian variance.

[0011] In one embodiment, the frequency domain sharpness index and the spatial domain sharpness index are combined to construct a multimodal sharpness temporal feature sequence, including: normalizing and splicing the frequency domain sharpness index and the spatial domain sharpness index to construct a multimodal sharpness temporal feature sequence reflecting changes in image sharpness.

[0012] In one embodiment, the method further includes: performing salient region detection on each frame of the image to obtain a cell region mask; and weighting and fusing the cell region mask with the sharpness index of the corresponding frame to enhance the weight of the cell region in the feature sequence.

[0013] In one embodiment, the attention mechanism includes a channel attention module, which is used to weight the importance of different feature channels in a multimodal clarity temporal feature sequence.

[0014] In one embodiment, the improved long short-term memory network is a two-layer stacked structure, and the residual connection structure adds the output of the first LSTM layer to the output of the second LSTM layer; the output of the improved long short-term memory network is connected to a prediction head composed of a one-dimensional convolutional layer and a fully connected layer, which is used to perform regression prediction of the sharpness values ​​at multiple future time steps.

[0015] In one embodiment, the method further includes: after completing the Z-axis movement, performing a sharpness assessment on the acquired focused image; if the deviation between the assessment result and the predicted value exceeds a preset threshold, triggering secondary focusing or an alarm.

[0016] In a second aspect, the present invention also provides an image focusing device based on fused frequency domain information, including a processor, a memory, and a control interface connected to a microscope; the memory stores a computer program, and the processor executes the computer program to implement the image focusing method based on fused frequency domain information described in one or more of the preceding embodiments.

[0017] In a third aspect, the present invention also provides a computer-readable storage medium storing computer-readable instructions for image focusing based on fused frequency domain information, which, when executed by one or more processors, implement the image focusing method based on fused frequency domain information as described in one or more embodiments above.

[0018] The beneficial effects of this invention are as follows: According to the solution of this invention, by fusing frequency domain information and long short-term memory network, and using sharpness trend prediction to replace traditional frame-by-frame search, the computational load and focusing latency are significantly reduced; at the same time, by introducing attention mechanism and residual connection structure, the noise interference problem in complex background is effectively solved, and the robustness of focusing and the response accuracy to key cell structures are greatly improved; in addition, the lightweight architecture of the system supports deployment on edge devices, realizing low-cost, high-efficiency and easy-to-integrate automated intelligent control of microscopy. Attached Figure Description

[0019] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 This is a flowchart illustrating an image focusing method based on fused frequency domain information according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the image focusing execution process according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating a long short-term memory network including a residual connection structure and an attention mechanism according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the trend of image sharpness as a function of focal length according to an embodiment of the present invention; Figure 5 This is a schematic block diagram illustrating the structure of an image focusing device based on fused frequency domain information according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] Figure 1 This is a flowchart illustrating an image focusing method 100 based on fused frequency domain information according to an embodiment of the present invention.

[0023] like Figure 1 As shown, in step S101, multiple frames of images from the microscope at different Z-axis positions are acquired to form an image sequence.

[0024] In step S102, the frequency domain sharpness index and spatial domain sharpness index of each frame in the image sequence are extracted. In some embodiments, the method for extracting the frequency domain sharpness index includes performing a Fourier transform on each frame to obtain a spectrogram, inputting a bandpass filter into the spectrogram to extract high-frequency regions, and calculating the spectral energy value in the high-frequency regions as the frequency domain sharpness index of that frame.

[0025] The spatial domain sharpness metric is selected from at least one of the following: the Tenengrad gradient magnitude or the Laplacian variance of the image.

[0026] In step S103, the frequency domain sharpness index and the spatial domain sharpness index are combined to construct a multimodal sharpness temporal feature sequence. In some embodiments, when combining the two, the frequency domain sharpness index and the spatial domain sharpness index can be normalized and concatenated to construct a multimodal sharpness temporal feature sequence reflecting changes in image sharpness.

[0027] Furthermore, the present invention also includes performing salient region detection on each frame of image to obtain a cell region mask, and weighting and fusing the cell region mask with the sharpness index of the corresponding frame to enhance the weight of the cell region in the feature sequence.

[0028] In step S104, the multimodal sharpness temporal feature sequence is input into a preset improved long short-term memory network for prediction to obtain the sharpness change trend of several future frames. The improved long short-term memory network includes a residual connection structure and an attention mechanism. The attention mechanism includes a channel attention module, which is used to weight the importance of different feature channels in the multimodal sharpness temporal feature sequence.

[0029] The improved Long Short-Term Memory (LSTM) network has a two-layer stacked structure. The residual connection structure adds the output of the first LSTM layer to the output of the second LSTM layer. The output of the improved LSTM network is connected to a prediction head consisting of a one-dimensional convolutional layer and a fully connected layer, which is used to perform regression prediction of the sharpness values ​​for multiple future time steps.

[0030] In step S105, the frame corresponding to the maximum sharpness value is located and predicted based on the sharpness change trend, and its corresponding Z-axis position is determined as the optimal focus point position.

[0031] In step S106, the microscope's Z-axis is moved to the optimal focal point to complete focusing. Further, after the Z-axis movement is complete, the sharpness of the acquired focused image is evaluated. If the deviation between the evaluation result and the predicted value exceeds a preset threshold, secondary focusing or an alarm is triggered.

[0032] The above scheme achieves fast, accurate, and deployable microscopic focusing control through a fusion strategy of image sequence feature extraction and time series trend prediction. The following will provide a detailed description of the scheme using specific embodiments.

[0033] Figure 2 This is a schematic diagram illustrating the image focusing execution process according to an embodiment of the present invention.

[0034] like Figure 2 As shown, the execution process in this invention mainly includes the following steps: Step 1, System Initialization and Parameter Setting. As one possible implementation of this embodiment, Step 1 specifically includes the following steps: Step 1.1: Complete the connection and testing of the microscope hardware equipment to ensure that the image acquisition unit and Z-axis drive platform (stepper motor or servo slide) are working properly.

[0035] Step 1.2: Configure parameters such as acquisition interval (e.g., move one step at a time, 10μm per step), number of acquisition frames, and image size.

[0036] Step 1.3: Load the trained sharpness prediction model and initialize the current Z-axis position and the allowed focusing window range.

[0037] Step 2, image sequence acquisition and preprocessing. As one possible implementation of this embodiment, step 2 specifically includes the following steps: Step 2.1: First, gradually adjust the focal length using a set step size, and acquire images at different Z-axis positions. The image acquisition formula is as follows:

[0038] in, Let t be the image frame acquired at step t, where t is the acquisition sequence number, corresponding to a certain Z-axis position or time step. These represent the height and width of the image, respectively.

[0039] Step 2.2: Perform uniform size cropping, brightness normalization, and noise filtering on the acquired images.

[0040] Step 2.3: Construct an index mapping table between image frames and their corresponding Z-axis step sizes to facilitate subsequent prediction alignment.

[0041] Step 3, frequency domain feature extraction and spatial sharpness calculation. As one possible implementation of this embodiment, step 3 specifically includes the following steps: Step 3.1: Perform a two-dimensional Fourier transform (FFT) on each frame of the image to obtain the spectrogram. The specific formula is shown below:

[0042] in, This represents the gray value of the image at coordinates (x, y) in the spatial domain. Represents frequency coordinates in the frequency domain The corresponding complex spectrum value, The imaginary unit, , Represents frequency domain coordinates, representing frequencies in the horizontal (u) and vertical (v) directions.

[0043]

[0044] in, This represents a subset of spectral values ​​in the high-frequency region (extracted by a bandpass filter). Let be the real part of the complex spectrum. It is the imaginary part of the complex spectrum, used to measure the intensity of different frequencies appearing in an image.

[0045] Step 3.2: Extract the high-frequency region using a bandpass filter and calculate the frequency domain sharpness index. The specific formula is shown below:

[0046] Among them, the frequency range is set. Reserve this frequency band, and set all other frequencies to zero. It is the square of the frequency radius (proportional to the square of the distance from the center frequency).

[0047] The high-frequency region is typically defined as:

[0048] in, For a high-frequency region mask (a region in the spectrum considered to "represent sharpness"), a high-frequency mask region can be determined using Fourier frequency distance. This is a high-frequency threshold, usually set based on experience or analysis.

[0049] Calculate the energy sum (square of L2 norm) of the high-frequency part of the spectrum.

[0050] This is the energy index of the t-th frame image in the high-frequency region, reflecting the image sharpness (the stronger the high frequency, the sharper the image). This value is the core feature quantity for judging sharpness in the frequency domain.

[0051] Min-Max normalization:

[0052] in, This is the normalized sharpness value. , These are the minimum and maximum energy values ​​in the current sampling sequence.

[0053] Z-Score standardization:

[0054] in, These are the mean and standard deviation of the energy sequence, respectively. Output the standardized values. As a frequency domain sharpness index for the t-th image, it can be used for focus determination or as input to subsequent models, ensuring scale consistency and facilitating training.

[0055] Step 3.3: Synchronously calculate the image gradient magnitude , variance As an indicator of spatial domain clarity.

[0056] Step 3.4: Normalize the above indicators to form the time series input features.

[0057] Step 4, Multimodal clarity temporal sequence construction. As one possible implementation of this embodiment, step 4 specifically includes the following steps: Step 4.1: The multiple sharpness evaluation metrics obtained from Step 3 are used to construct a multimodal feature representation of image sharpness. Specifically, for each frame of the image, at time step... Construct a feature input in the form of a three-dimensional vector. It includes the following three normalized feature channels, frequency domain indicators Gradient index , index splicing together to form a multimodal sequence Input matrix.

[0058] Step 4.2: Construct a time-series window (e.g., 5 frames) for the model input sliding window. The system then uses a sliding window mechanism to construct a time-series input matrix of fixed length w, in the following form:

[0059] in, Indicates the length of the time series window. This represents the feature sequence input to the deep prediction model, containing the most recent... Multimodal sharpness features of frame images.

[0060] Step 4.3: Label the focus position of each window in the training data to complete the temporal alignment. During the model training phase, it is necessary to label each sliding time window. The corresponding target focus position is labeled. Specifically, the system analyzes the distribution of sharpness indicators in the image frames covered by the current window, determines the frame number corresponding to the maximum sharpness value, and then infers its corresponding... The axis physical location is used as the prediction label. This label is used for supervised training. The network predicts future clarity trends, enabling the model to accurately identify the optimal focal point.

[0061] Step 5, Improve the LSTM prediction model construction and training. As one possible implementation of this embodiment, step 5 specifically includes the following steps: Step 5.1: Constructing a two-layer LSTM model with residual connections to enhance temporal memory capabilities. To improve the ability to model temporal changes in image sharpness, the system designs a two-layer Long Short-Term Memory (LSTM) network with a residual structure. This network consists of two stacked LSTM layers, used to extract local and global temporal dependencies, respectively. A residual connection mechanism is introduced between the layers, where the output of the first LSTM layer is added to the output of the second layer through an identity mapping to prevent gradient vanishing and improve training stability. The number of hidden units in each LSTM layer can be set according to the model capacity requirements; for example, the first layer has 128 units and the second layer has 64 units.

[0062] like Figure 3 The diagram illustrates the Attention-LSTM network structure proposed in this invention, based on multimodal input and residual connections. The model integrates three sharpness features—frequency domain (high-frequency energy), spatial domain (Laplacian variance and gradient magnitude)—as multi-channel inputs. These features are then processed through a two-layer LSTM with residual connections for temporal modeling, and an attention mechanism is introduced to enhance keyframe modeling capabilities. The output combines 1D convolution with a fully connected prediction head to achieve high-precision fitting of sharpness trends for several future frames.

[0063] Step 5.2: Introduce an attention mechanism module at the input of the LSTM ( To further enhance the model's responsiveness to key image frames, a channel attention mechanism module is introduced at the input of the LSTM network. This module calculates the relative importance of each input channel to the prediction task in the current sequence based on the global average pooling statistics of each channel (such as frequency domain energy, gradient magnitude, and Laplacian variance). or The activation function generates weight coefficients, which are then dynamically weighted across the input channels. This attention mechanism effectively suppresses irrelevant feature noise, highlights key regions, and improves prediction accuracy.

[0064] Building upon the existing input feature channels, a spatial attention module based on saliency detection is introduced to generate a region-weighted map for each frame of the image. This module uses global contextual information to determine the importance of cell regions in the current image and applies the attention map to the input frequency domain and spatial sharpness features, achieving dynamic weighting of key cell regions.

[0065] Meanwhile, a multi-modal attention fusion module was designed to assign weight coefficients to different types of features (frequency domain energy, image gradient, cell saliency) before LSTM encoding, further enhancing the model's ability to perceive trends in image sharpness changes.

[0066] Step 5.3: The output terminal adopts... A fully connected layer serves as the prediction head, used to fit the sharpness trend. A set of regression prediction structures is connected to the LSTM output to fit the image sharpness trend over several future time steps (frames). The prediction head typically has an output length of... , indicating the future Frame sharpness prediction sequence The final model output is a trend curve of image sharpness over time, which can be used to determine the optimal focus point in the future.

[0067] Step 5.4: Model training uses or As a loss function, supervised learning using existing focused data can be performed, and the loss function can be selected from the following two or a combination thereof, depending on the fitting target: Mean square error loss :

[0068] smooth loss :

[0069] in, This is the error tolerance threshold.

[0070] By backpropagating the loss function described above, the network parameters are optimized, enabling the model to stably fit the trend of sharpness changes and accurately locate the optimal focus frame.

[0071] Step 6, Sharpness Trend Prediction and Focus Determination. As one possible implementation of this embodiment, Step 6 specifically includes the following steps: Step 6.1: Input the feature matrix of the currently acquired image sequence into the trained LSTM network.

[0072] Step 6.2: Output the future sharpness change trend sequence .

[0073] Step 6.3: Search for global or local maximum values ​​in the prediction results to find the point with the maximum prediction sharpness and obtain the optimal focus point. .

[0074] Step 6.4: Convert the focal point position into Z-axis physical coordinates. This is to control the motor to move to that position.

[0075] Step 7, Automatic Focus Execution and Control. As one possible implementation of this embodiment, step 7 specifically includes the following steps: Step 7.1: To The axis platform issues a movement command to fine-tune to the optimal focus position. .

[0076] Step 7.2: After stabilization, acquire a focused image (i.e., reacquire a high-quality image) for verification by human-machine interaction or downstream algorithms.

[0077] Step 7.3: Record the execution parameters such as focusing process time and focusing quality index, and recalculate the energy index (sharpness index) in the high-frequency region of the image.

[0078] like Figure 4 The figure shows a schematic diagram illustrating the trend of image sharpness as a function of focal length in the intelligent focusing system of this invention. The horizontal axis represents the image number or Z-axis displacement step size, and the vertical axis represents sharpness indicators (such as high-frequency energy). The overall curve exhibits a typical "blurred-sharp-blurred" bell-shaped distribution. This system uses this trend curve to determine the physical focus position corresponding to the sharpest image, providing a basis for subsequent autofocus.

[0079] Step 8, closed-loop feedback and exception handling. As one possible implementation of this embodiment, step 8 specifically includes the following steps: Step 8.1: Analyze the difference between the true sharpness index and the predicted value of the focused image to determine whether secondary focusing compensation is needed.

[0080] Step 8.2: If the focusing deviation exceeds the threshold, automatic small-range resampling and fine-tuning will be performed, for example, in... Several frames were resampled within the range, and the focus was predicted and located again.

[0081] Step 8.3: If an abnormality is triggered 3 times consecutively, an alarm will prompt the user to manually check the optical path or sample status, and at the same time, "optical path abnormality" or "sample foreign object" will be automatically marked.

[0082] Step 9: Log recording, data management, and model update. As one possible implementation of this embodiment, step 9 specifically includes the following steps: Step 9.1: Completely record the sampling frames, prediction output, final positioning, focused image, and result evaluation for each focusing process.

[0083] Step 9.2: Store the recorded data in the database using the sample number index for easy quality traceability.

[0084] Step 9.3: If the online training module is enabled, the model parameters can be fine-tuned periodically based on high-quality historical data to continuously improve focusing performance.

[0085] Step 10, System Deployment and Edge Operation and Maintenance Mechanism. As one possible implementation of this embodiment, step 10 specifically includes the following steps: Step 10.1: Deploy the compressed model to edge devices, such as... Raspberry Pi, domestic industrial control boards, etc.

[0086] Step 10.2: Support local model loading and remote upgrades, and support offline operation and exception caching.

[0087] Step 10.3: The system has remote monitoring and fault log uploading functions, which facilitates remote maintenance and model evaluation.

[0088] Figure 5 This is a schematic block diagram illustrating the structure of an image focusing device based on fused frequency domain information according to an embodiment of the present invention.

[0089] This invention also provides an image focusing device based on fused frequency domain information. For example... Figure 5 As shown, the device includes a processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement the image focusing method based on fused frequency domain information as described above.

[0090] The device also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0091] Furthermore, the present invention also provides a computer-readable storage medium storing computer-readable instructions for image focusing based on fused frequency domain information. When these computer-readable instructions are executed by one or more processors, they implement the image focusing method based on fused frequency domain information as described above.

[0092] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions that can be stored or otherwise maintained by such a computer-readable medium.

[0093] In the description of this specification, "multiple" or "several" means at least two, such as two, three or more, unless otherwise explicitly specified.

[0094] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. An image focusing method based on fused frequency domain information, characterized in that, include: Multiple frames of images obtained from the microscope at different Z-axis positions are used to form an image sequence; Extract the frequency domain sharpness index and spatial domain sharpness index of each frame in the image sequence; The frequency domain sharpness index and the spatial domain sharpness index are combined to construct a multimodal sharpness temporal feature sequence; The multimodal sharpness temporal feature sequence is input into a preset improved long short-term memory network for prediction to obtain the sharpness change trend of several future frames. The improved long short-term memory network includes a residual connection structure and an attention mechanism. Based on the sharpness change trend, locate and predict the frame corresponding to the maximum sharpness value, and determine its corresponding Z-axis position as the optimal focus point position; The microscope's Z-axis is moved to the optimal focal point to complete focusing.

2. The image focusing method based on fused frequency domain information according to claim 1, characterized in that, The method for extracting the frequency domain sharpness index includes: A Fourier transform is performed on each frame of the image to obtain a spectrogram. The spectrogram is then input into a bandpass filter to extract high-frequency regions, and the spectral energy value within the high-frequency regions is calculated as the frequency domain sharpness index of that frame of the image.

3. The image focusing method based on fused frequency domain information according to claim 1, characterized in that, The spatial domain sharpness index is selected from at least one of the following: the Tenengrad gradient magnitude or the Laplacian variance of the image.

4. The image focusing method based on fused frequency domain information according to claim 1, characterized in that, The frequency domain sharpness index and the spatial domain sharpness index are combined to construct a multimodal sharpness temporal feature sequence, including: The frequency domain sharpness index and the spatial domain sharpness index are normalized and spliced ​​to construct a multimodal sharpness time feature sequence that reflects changes in image sharpness.

5. The image focusing method based on fused frequency domain information according to claim 4, characterized in that, Also includes: Perform salient region detection on each frame of the image to obtain cell region masks; The cell region mask is weighted and fused with the sharpness index of the corresponding frame to enhance the weight of the cell region in the feature sequence.

6. The image focusing method based on fused frequency domain information according to claim 1, characterized in that, The attention mechanism includes a channel attention module, which is used to weight the importance of different feature channels in the multimodal clarity temporal feature sequence.

7. The image focusing method based on fused frequency domain information according to claim 1, characterized in that, The improved long short-term memory network is a two-layer stacked structure. The residual connection structure adds the output of the first LSTM layer to the output of the second LSTM layer. The output of the improved long short-term memory network is connected to a prediction head composed of a one-dimensional convolutional layer and a fully connected layer, which is used to perform regression prediction of the sharpness values ​​at multiple future time steps.

8. The image focusing method based on fused frequency domain information according to claim 1, characterized in that, Also includes: After completing the Z-axis movement, the sharpness of the acquired focused image is evaluated. If the deviation between the evaluation result and the predicted value exceeds a preset threshold, a second focusing or alarm is triggered.

9. An image focusing device based on fused frequency domain information, characterized in that, It includes a processor, a memory, and a control interface connected to a microscope; the memory stores a computer program, and when the processor executes the computer program, it implements the image focusing method based on fused frequency domain information according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It stores computer-readable instructions for image focusing based on fused frequency domain information, which, when executed by one or more processors, implement the image focusing method based on fused frequency domain information as described in any one of claims 1-8.

Images