Microscope focus sequence image definition ranking method, device, medium and system

Abstract

The application provides a microscope focus sequence image sharpness sorting method, device, medium and system, the method is executed by a computing device, and comprises the following steps: S1: obtaining a focus sequence image to be sorted; S2: using an interleaved grouping and iterative reduction sorting screening strategy, the focus sequence image is batched into a pre-trained sorting model for processing to generate the sharpness score of each image frame; S3: according to the sharpness score, the highest score image frame is selected as a global sharpest image frame; wherein the sorting model comprises: a feature extraction module: extracting the depth visual features of a single frame image in the focus sequence image; a multi-scale time sequence processing module: based on the depth visual features, the time sequence dependence between the focus sequence images is captured to generate the sharpness score.

Description

Technical Field

[0001] This invention relates to the fields of image processing and artificial intelligence technology, and more specifically, to a method, apparatus, medium, and system for ranking the sharpness of microscope focusing sequence images. Background Technology

[0002] In microscopy applications, such as gynecological clinical examinations, obtaining clear images is crucial for accurate subsequent diagnosis. During manual or automatic focusing of a microscope, a focusing sequence containing multiple frames is generated, of which typically only a few frames are clear or nearly clear. Therefore, automatically, quickly, and accurately selecting the clearest frame from this sequence is a key technological step in improving diagnostic efficiency and automation.

[0003] In existing technologies, one type of method constructs a sharpness evaluation function based on traditional image processing theory. These methods typically employ a single evaluation metric, such as a gradient-based function or a variance-based function, to score each frame in a sequence of images. To optimize the specificity of these methods, some research has proposed (e.g., Chinese patent CN202111119572.8) first identifying target objects in the image through object detection, and then calculating the sharpness of that specific target using a preset matrix and gradient values, thereby automatically adjusting the microscope's focusing parameters. However, these gradient-based or specific mathematical model-based evaluation metrics remain sensitive to interference factors such as noise and changes in illumination, and the focusing process often relies on a search strategy to find the scoring peak, which can easily lead to the search process getting trapped in local extrema. Furthermore, finding the scoring peak usually still requires acquiring and processing a large number of images, which to some extent limits the timeliness of focusing.

[0004] With the development of deep learning technology, methods for sharpness determination using neural networks have emerged. Some solutions train neural network models to simulate the human eye's judgment of image sharpness. However, these methods heavily rely on manual sharpness annotation of massive amounts of images during the model training phase. This annotation process is not only time-consuming and labor-intensive, but the annotation results are also greatly affected by the subjective factors of the annotators, resulting in insufficient accuracy and stability of the trained models. Other solutions attempt to use multi-scale fusion network structures to process time-series signals, but they are mainly applied to tasks such as signal classification, using general classification loss functions, and are not optimized for the ranking task of "finding the unique best item in a sequence". At the same time, when processing long sequences with a large number of frames, existing methods also lack efficient inference strategies, making it difficult to meet the real-time processing requirements of clinical applications while ensuring ranking accuracy.

[0005] Therefore, existing technologies are insufficient in terms of the accuracy of sharpness evaluation, the degree of automation of model training, and the efficiency of long sequence processing, making it difficult to meet the needs of efficient and accurate automatic focusing in scenarios such as clinical diagnosis. Summary of the Invention

[0006] In view of the deficiencies in the prior art, the purpose of this invention is to provide a method, apparatus, medium and system for sorting the sharpness of microscope focusing sequence images.

[0007] According to a first aspect of the present invention, a method for ranking the sharpness of microscope focusing sequence images includes:

[0008] Step S1: Obtain the focus sequence images to be sorted;

[0009] Step S2: Using an alternating grouping and iterative reduction sorting and filtering strategy, the focused sequence images are fed into a pre-trained sorting model in batches for processing to generate a sharpness score for each image frame.

[0010] Step S3: Based on the sharpness score, select the image frame with the highest score as the globally sharpest image frame;

[0011] The sorting model includes:

[0012] Feature extraction module: Extracts the depth visual features of a single frame image from the focused image sequence;

[0013] Multi-scale temporal processing module: Based on the depth visual features, capture the temporal dependencies between the focused sequence images to generate a sharpness score.

[0014] Preferably, in step S2, the sorting and filtering strategy of staggered grouping and iterative reduction includes:

[0015] Step S2.1: Calculate the number of groups M based on the total number of frames in the focused sequence images and the upper limit of the number of frames that the sorting model can process in a single run;

[0016] Step S2.2: Perform interleaved sampling on the focused sequence images with a step size of M groups to divide them into M subgroups;

[0017] Step S2.3: Take all the image frames in the M subgroups as the initial candidate image frame set;

[0018] Step S2.4: Narrowing the candidate image frame set by iterative filtering, wherein the iterative filtering includes: in each iteration, feeding the current candidate image frame set into the ranking model in batches to generate their respective sharpness scores, and selecting one or more image frames with the highest scores from each batch to form a smaller set of candidate image frames for the next iteration, until the number of image frames in the candidate image frame set can be processed by the ranking model in a single step.

[0019] Preferably, the multi-scale temporal processing module includes multiple parallel one-dimensional convolutional layers with different temporal convolutional scales, which are used to extract temporal dependencies of different ranges in the temporal dimension.

[0020] Preferably, the feature extraction module is a module based on a pre-trained convolutional neural network, wherein the pre-trained convolutional neural network is selected from the ResNet series network or the EfficientNet series network.

[0021] Preferably, in step S2, the ranking model is obtained through a training method including the following steps:

[0022] Obtain training sequences, where each training sequence contains a pre-labeled ground truth most sharp frame;

[0023] For each image frame in the training sequence, perform automatic sharpness annotation processing using multi-source index fusion to generate a sharpness annotation score;

[0024] A Top-1 ranking loss strategy is adopted, and the ranking model is trained based on the sharpness label score so that the ranking model predicts a higher score for the sharpest frame in the training sequence than for other frames.

[0025] Preferably, the automatic clarity annotation process for multi-source indicator fusion includes:

[0026] Calculate at least two of the following image quality metrics: target quantity, target sharpness, image brightness, and image contrast.

[0027] A weighted inverse ranking fusion strategy is adopted to generate a sharpness label score based on the ranking results of the at least two image quality indicators.

[0028] Preferably, the Top-1 ranking loss strategy includes:

[0029] The difference between the predicted score of the ranking model for the clearest frame and the predicted scores for other frames is calculated, and the difference is input into the Logistic function to calculate the Top-1 ranking loss.

[0030] According to a second aspect of the present invention, a microscope system includes:

[0031] Module M1: Acquires the focus sequence images to be sorted;

[0032] Module M2: Employs a sorting and filtering strategy of staggered grouping and iterative reduction to feed the focused sequence images into a pre-trained sorting model in batches for processing, in order to generate a sharpness score for each image frame.

[0033] Module M3: Based on the sharpness score, select the image frame with the highest score as the globally sharpest image frame;

[0034] The sorting model includes:

[0035] Feature extraction module: Extracts the depth visual features of a single frame image from the focused image sequence;

[0036] Multi-scale temporal processing module: Based on the depth visual features, capture the temporal dependencies between the focused sequence images to generate a sharpness score.

[0037] According to a third aspect of the present invention, a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a method for sorting the sharpness of microscope focusing sequence images.

[0038] According to a fourth aspect of the present invention, a sharpness ranking apparatus includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of a sharpness ranking method for microscope focusing sequence images.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. Highly accurate automatic labeling was achieved. Through an automatic labeling strategy that integrates multiple sources of metrics, high-quality, clear labels can be generated for training data without human intervention. This avoids the bias of single metrics and the subjectivity of manual labeling, significantly improving the accuracy and reliability of labeling and reducing model training costs.

[0041] 2. Improved accuracy of the ranking model. By adopting a convolutional neural network and a multi-scale temporal fusion network architecture, the model can simultaneously extract the depth visual features of images and the temporal dynamic information of sequences, enhancing its adaptability to different focusing behaviors. Combined with a Top-1 ranking loss function specifically designed for ranking tasks, the model can more accurately identify the clearest image frames globally.

[0042] 3. Efficient processing of long sequences is achieved. By employing an inference strategy of staggered grouping and iterative reduction, and through a mechanism similar to an "elimination tournament," the best frames can be quickly and accurately selected from sequences containing a large number of image frames. This effectively solves the efficiency bottleneck of long sequence processing and meets the needs of application scenarios such as real-time diagnosis.

[0043] 4. A fully automated solution is provided. This application realizes a fully automated process from image acquisition, model training data generation, model training to final sharpness ranking, overcoming many shortcomings of traditional methods and providing reliable and efficient technical support for microscopic applications such as gynecological clinical diagnosis. Attached Figure Description

[0044] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 A schematic diagram of the architecture of a sharpness ranking system for microscope focusing sequence images provided in an embodiment of this application;

[0046] Figure 2 A flowchart illustrating a method for ranking the sharpness of microscope focusing sequence images provided in this application embodiment;

[0047] Figure 3 This is a schematic diagram of the structure of a multi-scale temporal fusion network model provided in an embodiment of this application;

[0048] Figure 4 A schematic diagram illustrating an interleaved grouping and iterative reduction strategy provided in an embodiment of this application;

[0049] Figure 5 This is a signaling interaction timing diagram for a resolution ranking method provided in an embodiment of this application.

[0050] In the figure, 10 is the microscope image acquisition device; 20 is the computer processing equipment; 21 is the image preprocessing module; 22 is the automatic sharpness annotation module; 23 is the ranking model; 24 is the ranking and filtering module; 30 is the CNN backbone network; 31 is the temporal feature tensor; 32 is the parallel temporal processing branch; 33 is the one-dimensional convolutional layer; 34 is the pooling layer; 35 is the stitching layer; 36 is the output layer; 40 is the original sequence; 41 is the staggered sampling; 42 is the subgroup 1; 43 is the subgroup 2; 44 is the intra-group ranking; 45 is the candidate set; 46 is the final selection; 47 is the global sharpest frame; S100 is the acquisition of sequence images; S200 is the model training stage; S210 is the automatic annotation; S220 is the model training; S300 is the model inference stage; S310 is the staggered grouping; S320 is the iterative reduction and filtering; S330 is the final selection; and S340 is the output result. Detailed Implementation

[0051] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0052] Example 1:

[0053] This embodiment provides a method, apparatus, and system for ranking the sharpness of microscope focusing sequence images. The solution aims to automate the entire process, from the automated generation of model training data to efficient and accurate model training, and finally to rapid inference and ranking of long image sequences.

[0054] Please see Figure 1 This illustration shows a schematic diagram of the architecture of a sharpness ranking system for a microscope focusing sequence image according to an embodiment of this application. The system includes a microscope image acquisition device 10 and a computer processing device 20. The microscope image acquisition device 10, such as a digital microscope with a motorized focusing module, is responsible for continuously capturing a series of image frames during focusing to form a focusing sequence image and transmitting them to the computer processing device 20. The computer processing device 20 can be a personal computer, workstation, server, or embedded system, internally configured with software modules that execute the methods of this application.

[0055] Specifically, these modules may include: an image preprocessing module 21, which standardizes the received raw images; an automatic sharpness labeling module 22, which automatically generates high-quality sharpness labels during the model training phase; a core ranking model 23, which scores the sharpness of the input image sequence; and a ranking filtering module 24, which efficiently filters long sequences during the model inference phase.

[0056] Figure 2 The overall flow of a method for ranking the sharpness of a microscope focusing sequence of images provided in this application embodiment is illustrated. The method is mainly divided into a model training stage S200 and a model inference stage S300, both of which begin with the acquisition of the sequence of images S100.

[0057] In the model training phase S200, the goal is to build and optimize a ranking model 23 that can accurately predict image sharpness scores. The detailed steps of this phase are as follows:

[0058] First, image acquisition and preprocessing are performed. A large number of focused sequence images are acquired using the microscope image acquisition device 10 as a training dataset. For example, in a gynecological wet mount examination scenario, sequences generated from hundreds of different samples under different focusing processes can be acquired. Each sequence contains a complete process from blurry to clear and back to blurry, with the number of frames ranging from tens to hundreds. The acquired raw image sequences are processed by the image preprocessing module 21. The preprocessing steps may include: 1) Image normalization, scaling the pixel values ​​of all images to a uniform range (e.g., [0,1]) to eliminate the influence of different lighting conditions; 2) Polarization difference processing, utilizing the characteristics of polarized light to enhance the contours of specific targets such as cells and suppress background noise; 3) Scattering correction, using algorithms to reduce the scattering effects caused by media such as mucus and blood, thereby improving image transparency.

[0059] Subsequently, the sharpness automatic annotation module 22 performs multi-source index fusion-based sharpness automatic annotation processing S210. The core purpose of this step is to automatically and objectively generate a "ground truth" sharpness score sequence for each focus sequence in the training dataset, thereby determining the generally accepted sharpest frame in the sequence. The entire process requires no manual intervention. Specifically, for a sequence containing N frames, the processing flow is as follows:

[0060] 1. Calculate multi-dimensional image quality metrics. For each frame in the sequence, calculate at least two, or four different image quality metrics in this embodiment. These metrics evaluate image quality from different dimensions, including but not limited to:

[0061] Target Count: The number of targets of interest (such as epithelial cells) in an image is estimated using cell detection algorithms (e.g., circular detection based on Hough transform or simple thresholding and connected component analysis). Typically, cell outlines are clearest and most easily detected near the focal point, thus the target count peaks.

[0062] Target sharpness: Using one or more classic sharpness evaluation functions, such as the Laplacian operator, Sobel operator, or Tenengrad gradient function, the sum or average of the edge gradients of all identified targets in the image is calculated. The higher the gradient value, the sharper the target edges and the clearer the image.

[0063] Image brightness: Calculates the average pixel grayscale value of the entire image. In some focusing scenarios, the sharpest image may correspond to the brightest field of view.

[0064] Image contrast: Calculates the standard deviation or variance of an image's gray levels. Higher contrast generally means a richer level of detail in the image.

[0065] 2. Independent Sorting. Based on each calculated metric (target quantity, target sharpness, brightness, contrast), the N frames in the sequence are independently sorted in descending order. Thus, each frame will receive four different rankings in four different sorting lists. For example, image frame i might rank [number] in the target quantity sorting. The name ranks first in target clarity. The names, and so on.

[0066] 3. Weighted Fusion Scoring. A weighted inverse ranking fusion strategy is used to merge the four independent ranking lists, generating a comprehensive sharpness label score for each image frame. For image frame i, its final sharpness label score is... Calculated using the following formula:

[0067]

[0068] in, The preset weights for each indicator are given, and k is a small smoothing constant (e.g., 60) to avoid the denominator being too small when the ranking is 1. As the target quantity weight, Weighted by target clarity. Image brightness weighting, Image contrast weights, Rank image frame i in terms of the number of targets; The ranking of image frame i in the target sharpness sorting; The ranking of image frame i in the target brightness sorting; The ranking of image frame i in the target contrast sort.

[0069] In this embodiment, based on experience, the weights of each indicator are set as follows: target quantity weight. Target clarity weight Image brightness weight and image contrast weights These weights reflect the contribution of different indicators to the final clarity judgment.

[0070] 4. Position Weighting. Considering that the sharpest frame usually appears in the early stages of the focusing process during actual focusing, this embodiment applies a weighted bonus to the final score of the image frames that appear earlier in the sequence to further optimize the annotation results. Specifically, the final fusion score of the first 5 image frames in the sequence is weighted. Then multiply by a weighting factor of 1.5. Through the above steps, each frame in each training sequence obtains a sharpness label score, and the frame with the highest score is considered the "ground truth sharpest frame" of the sequence.

[0071] Next, model building and training (S220) are carried out to train the ranking model (23).

[0072] Please see Figure 3 This illustrates the detailed structure of the sorting model 23 in this embodiment. This model is a multi-scale temporal fusion network.

[0073] Model architecture:

[0074] Feature extraction module: The model uses a 2D convolutional neural network pre-trained on the ImageNet dataset as the backbone network 30 to extract the depth visual features of a single frame image. In this embodiment, the ResNet50 network is selected. For each frame image in the input sequence, the shared-weights ResNet50 network is used for forward propagation to extract the feature map before its last convolutional layer or global average pooling layer, and flatten it into a high-dimensional feature vector.

[0075] Temporal feature reshaping module: The T feature vectors extracted from all image frames in a sequence (assuming there are T frames) are stacked in chronological order to form a temporal feature tensor of shape (T, C), where C is the dimension of the feature vector of a single frame.

[0076] Multi-scale temporal processing module: This is a key part of the model, used to capture temporal dependencies between image sequences. It consists of multiple parallel temporal processing branches (such as...). Figure 3 The parallel temporal processing branches 32a, 32b, and 32c constitute the structure. In this embodiment, five parallel branches are constructed. The core of each branch is a one-dimensional convolutional layer 33. These one-dimensional convolutional layers use temporal convolutional kernels of different sizes to extract temporal features of different ranges (scales) in the temporal dimension. Specifically, the one-dimensional convolutional kernel sizes of these five branches are set to 3, 5, 7, 9, and 11, respectively. The branch with a convolutional kernel size of 3 can capture short-term, sharpness change trends between adjacent frames; while the branch with a convolutional kernel size of 11 can capture longer-term, larger-span change patterns. Each one-dimensional convolutional layer can be followed by a batch normalization layer and an activation function such as ReLU.

[0077] Feature Fusion and Output: The output of each parallel branch passes through an adaptive average pooling layer 34, compressing it into a fixed-length vector along the time dimension. In this embodiment, the features output by each branch are compressed into a 50-dimensional vector. These five 50-dimensional vectors are then concatenated along the channel dimension in a concatenation layer 35 to form a 500-dimensional composite feature vector. This composite feature vector incorporates sequence dynamics information from different time scales. Finally, this composite feature vector is fed into an output layer 36, which is typically one or more fully connected layers, ultimately mapping the 500-dimensional features into T scalar values, corresponding to the sharpness prediction score for each frame of the input sequence. Model Training:

[0078] The model is trained using a Top-1 ranking loss strategy. It is understood that traditional classification losses (such as cross-entropy) or regression losses (such as mean squared error) are not suitable for ranking tasks that require selecting a single best value. Therefore, this embodiment employs a specially designed Top-1 ranking loss, the goal of which is to ensure that the model's predicted score for the "frame with the clearest truth value" is significantly higher than its predicted score for any other frame in the sequence.

[0079] The specific calculation process is as follows: For a training sequence, firstly, the clearest frame of the ground truth is determined using the aforementioned automatic annotation method. The sequence is then input into the currently training ranking model 23 to obtain the model's predicted score sequence for all frames. Extract the prediction score from the clearest frame that holds the true value. and the set of prediction scores for all other non-sharpest frames. Then, calculate. With each The difference between These differences are input into a penalty function. In this embodiment, the Logistic function is used. The total Top-1 ranking loss is calculated as follows:

[0080]

[0081] The loss function has the following properties: when Much larger hour, Large positive numbers When the value approaches zero, the loss contribution is very small; conversely, when... Less than or close to hour, Negative or close to 0 The loss is relatively large, resulting in a significant loss. By backpropagating this loss to update the model parameters, the model will be driven to "amplify" the score difference between the sharpest frame and all other frames.

[0082] After training is completed, the resulting ranking model 23 can be used in the model inference stage S300.

[0083] In the model inference stage S300, the goal is to quickly and accurately identify the globally sharpest frame from a new, unsorted sequence of focused images. Considering that sequences can be very long in real-world applications (e.g., exceeding 1000 frames), feeding all frames into the model at once might exceed the memory limitations of the hardware (e.g., GPU) and be inefficient. Therefore, this embodiment employs a sorting and filtering strategy of staggered grouping and iterative reduction, executed by the sorting and filtering module 24.

[0084] Please see Figure 4 The diagram illustrates this strategy. The detailed steps are as follows:

[0085] 1. Interleaved Grouping S310: Assume the total number of frames in the original sequence 40 to be sorted is N, and the upper limit of the number of frames that the sorting model 23 can process in a single run is P. In this embodiment, P=16. First, calculate the required number of groups M, which is calculated as follows: (That is, N divided by P and rounded up).

[0086] Then, using an interleaved sampling method with a step size of M, the entire sequence is divided into M subgroups (subgroup 1, subgroup 42, subgroup 2, subgroup 43, etc.). The specific sampling method is as follows: the 1st frame, the 1+Mth frame, the 1+2Mth frame, etc. constitute subgroup 1; the 2nd frame, the 2+Mth frame, the 2+2Mth frame, etc. constitute subgroup 2; and so on, until the Mth frame, the M+Mth frame, the M+2Mth frame, etc. constitute subgroup M.

[0087] It should be noted that this staggered grouping method ensures that the temporal distribution of image frames within each subgroup in the original sequence is broad and uniform, avoiding the grouping of temporally consecutive frames into the same group, thus making the sorting results of each subgroup have a certain degree of global representativeness.

[0088] 2. Iterative Reduction and Selection S320: This step gradually narrows down the set of candidate image frames through a "elimination tournament" mechanism. Specifically, it includes:

[0089] The M subgroups are divided and fed into the trained ranking model 23 one by one or in batches for intra-group ranking 44 to obtain the sharpness score and ranking result within each subgroup.

[0090] The selection strategy is determined based on the size of the number of groups M. Case 1: If the number of groups M is greater than the model's processing limit P (e.g., N=300, P=16, then M=19>16), it indicates that the size of the winner set still exceeds the model's single-processing capacity, requiring multiple rounds of elimination. In this case, the top-1 frames with the highest scores are selected from each subgroup, and these M winners form a new, shorter candidate set 45. Subsequently, the steps of staggered grouping and iterative reduction selection are repeated for this new candidate set.

[0091] Scenario 2: If the number of groups M is not greater than the model's processing limit P (e.g., N=100, P=16, then M=7<=16), it indicates that all winners can be included in a single processing run. In this case, calculate the number of winners K to be retained in each group, as follows: (That is, P divided by M and rounded down). Then, the top-K frames with the highest scores are selected from each subgroup. All of these M×K frames together form the final candidate set 45.

[0092] 3. Final selection S330: Take the final candidate set 45 obtained in the previous step (with a total number of frames not exceeding P) as a complete sequence and send it back to the sorting model 23 for the final sorting.

[0093] 4. Output result S340: In the final ranking results, select the image frame with the highest score as the global clearest frame 47 of the entire original sequence, and output or display it.

[0094] Please see Figure 5 This demonstrates a typical signaling interaction timing of the method in practical application.

[0095] User 50 initiates an acquisition command to microscope acquisition device 51 through the user interface. Microscope acquisition device 51 then activates the focusing motor and continuously captures images, sending the resulting focused sequence of images to computer processing device 52 in real time or in batches. After receiving the sequence, computer processing device 52's internal sorting and filtering module 24 and sorting model 23 work together to execute the aforementioned staggered grouping, iterative reduction filtering, and final selection process. This process is transparent to the user. After the calculation is completed, computer processing device 52 displays the result of the finally selected globally clearest frame on the user interface for user 50 (e.g., a doctor) to observe and diagnose.

[0096] In summary, this embodiment provides an end-to-end solution by combining automated annotation methods, a powerful temporal network model, and an efficient inference strategy. This solution can automatically, accurately, and quickly select the clearest frame from a gynecological wet mount focusing sequence containing hundreds of images. Compared to traditional single-index evaluation methods, its accuracy is significantly improved, and it eliminates the need for manual annotation, thus offering high processing efficiency. It is particularly suitable for scenarios with high real-time and accuracy requirements, such as clinical diagnosis.

[0097] Example 2

[0098] As an optional implementation, this embodiment is a variation of Embodiment 1. Its overall method flow is consistent with Embodiment 1, also including a model training phase and a model inference phase, and employing the same Top-1 ranking loss and staggered grouping and iterative reduction strategy. The main difference in this embodiment lies in the weight parameters of the automatic clarity annotation phase and the specific network architecture and parameter settings of the ranking model, aiming to demonstrate the versatility and flexibility of the proposed solution.

[0099] In step S210 of the automatic clarity annotation process, this embodiment adjusts the fusion weights of multi-source indicators. Specifically, the weights used in the weighted inverse ranking fusion strategy are set as follows: target quantity weight. Target clarity weight Image brightness weight and image contrast weights Compared to Example 1, this example slightly increases the weights of the two metrics most strongly correlated with core sharpness: target quantity and target sharpness. Furthermore, regarding the positional weighting, this example multiplies the final score of the first 8 image frames in the sequence by a weighting factor of 1.2, which is a more moderate early preference strategy.

[0100] In the model building and training step S220, this embodiment uses different configurations of the backbone network 30 and the multi-scale temporal processing module. Specifically:

[0101] Feature extraction module: In this embodiment, EfficientNetV2 is selected as the pre-trained convolutional neural network backbone network 30. Compared with ResNet50, EfficientNetV2 generally has a better balance between computational efficiency and feature extraction capability.

[0102] Multi-scale temporal processing module: This embodiment constructs 6 parallel temporal processing branches, which is one scale larger than that in Embodiment 1. The kernel sizes of the one-dimensional convolutional layers 33 in these 6 branches are set to 3, 5, 7, 9, 11 and 13, respectively. Adding a larger-scale convolutional kernel (13) enables the model to capture sequence dependencies over a longer time span, which may be more advantageous for some sequences with slower focusing speeds or longer processes.

[0103] Feature fusion and output: Accordingly, the output of each parallel branch is compressed into a 64-dimensional vector after passing through an adaptive average pooling layer 34. These six 64-dimensional vectors are concatenated in a concatenation layer 35 to form a 640-dimensional composite feature vector. This vector is then fed into the output layer 36 to generate the final sharpness prediction score.

[0104] The strategy of the model inference stage S300 is consistent with that of Implementation Example 1, that is, setting the upper limit of single model processing P=16 frames, and using the same staggered grouping, iterative reduction and final selection logic.

[0105] It is understandable that by employing the EfficientNetV2 backbone network and more temporal scales, the ranking model constructed in this embodiment theoretically possesses stronger feature representation capabilities and a wider temporal awareness range. When processing certain specific types or more complex focus sequences, such as sequences with severe background interference or indistinct sharpness peaks, the scheme in this embodiment may achieve a higher ranking accuracy than that in Embodiment 1, thus verifying the potential for performance improvement within the framework of this application's method by replacing and adjusting core components.

[0106] Example 3

[0107] This embodiment is another variation of Embodiment 1, designed to verify that the proposed method framework still maintains effectiveness and robustness even with simplified automatic annotation metrics. In practical applications, simplifying the annotation process can reduce the computational complexity of preprocessing training data, thereby accelerating model iteration.

[0108] The core process, model architecture (based on ResNet50), training strategy (Top-1 ranking loss), and inference strategy (alternating grouping and iterative reduction) of this embodiment are consistent with those of Embodiment 1. The main difference lies in the automatic resolution annotation process S210 in the model training phase S200.

[0109] Specifically, in the automatic annotation process of this embodiment, the indicators have been simplified, and only two core indicators most directly related to image sharpness are calculated and used:

[0110] 1. Target sharpness: Similar to Example 1, the edge sharpness of the target in the image is calculated using a gradient function.

[0111] 2. Image contrast: Same as in Example 1, calculate the standard deviation of the image's grayscale.

[0112] This embodiment omits the calculation of the auxiliary indicators "target quantity" and "image brightness". During the weighted inverse ranking fusion, it is based solely on the independent ranking results of the target sharpness and contrast indicators. Accordingly, the weights are reset to: target sharpness weight. Contrast weight Furthermore, this embodiment does not use any positional weighting.

[0113] Although the sources of "ground truth" information used to generate training labels are reduced, the model can still learn the dynamic change pattern from blurry to clear from the data because the most core clarity-related metrics are retained, and the subsequent model training still employs a powerful multi-scale temporal fusion network architecture and an efficient Top-1 ranking loss strategy. Experimental results show that the model trained using this simplified labeling strategy, while its ranking accuracy may be slightly lower than that of Example 1 using four metrics, still achieves a high level, far superior to traditional single-metric methods, and significantly reduces the computational cost of the automatic labeling process.

[0114] This embodiment demonstrates the robustness of the method framework of this application. Even if the user selectively reduces or adjusts the combination of indicators used for automatic annotation according to the needs of computing resources or specific application scenarios, the entire technical solution can still work effectively and achieve satisfactory results.

[0115] Example 4

[0116] This embodiment is another variation of Embodiment 1, focusing on demonstrating a different implementation of the sorting and filtering strategy in the model inference stage S300. All steps in the model training stage S200, including data acquisition, preprocessing, automatic annotation, model building, and training, are exactly the same as in Embodiment 1. The main difference lies in the specific logic of the iterative reduction filtering S320 used during inference.

[0117] In Example 1, when the number of groups M is not greater than the model's processing limit P, Top-K (K=floor(P / M)) winners are selected from each subgroup to enter the final round. In this example, to further improve the speed of long sequence processing, especially in scenarios with extremely high real-time requirements, a simplified single-elimination iterative reduction mode is adopted.

[0118] Specifically, in the iterative reduction screening step S320, regardless of the size of the number of groups M, in each round of screening, only the top-1 (i.e., K=1) winners with the highest scores from each subgroup are selected to advance to the next round. This process iterates until the total number of remaining candidates is less than or equal to the upper limit of the model's single processing time P (i.e., 16 frames). These final candidates together constitute the final group.

[0119] To illustrate this more clearly, let's take an original sequence 40 containing N=100 frames as an example, with a model processing limit of P=16.

[0120] 1. Interleaved Grouping S310: Calculate the number of groups M = ceil(100 / 16) = 7. Divide the sequence into 7 subgroups by interleaved sampling with a step size of 7.

[0121] 2. Iterative reduction screening of S320:

[0122] First round of selection: The 7 subgroups are fed into the model for in-group ranking 44. According to the strategy of this embodiment, only the Top-1 winner is selected from each subgroup. Thus, after the first round of selection, a new candidate set consisting of 7 winners is obtained 45.

[0123] Determining whether a next round is needed: Since the new candidate set size is 7, which is less than P=16, no further iteration is required. These 7 frames directly constitute the final candidate set.

[0124] 3. Final selection S330: The final candidate set consisting of these 7 frames is fed into the model for a final sorting, and the clearest frame in the world is selected as 47.

[0125] Compared to the strategy in Example 1 (where K = floor(16 ÷ 7) = 2 when N = 100, resulting in 7 × 2 = 14 frames being selected for the final round), this example's strategy retains only 7 frames after the first round of selection, resulting in fewer images being sent to the final selection. Therefore, the overall computational cost is lower and the inference speed is faster. This advantage is even more pronounced when dealing with extremely long sequences (e.g., N > 1000) because it can reduce the size of the candidate set much faster.

[0126] It should be noted that this strategy theoretically carries the risk of accuracy loss, as it may prematurely eliminate some frames that, while not first in their group, are potentially globally "suboptimal." However, in numerous experiments, this risk has been kept to a very low level due to the staggered grouping ensuring the representativeness of each group and the robust model's inherent high in-group ranking accuracy. Therefore, this embodiment provides a preferred implementation scheme for scenarios prioritizing extreme inference speed, demonstrating the flexibility of the proposed method in balancing efficiency and accuracy.

[0127] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0128] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for ranking the sharpness of microscope focusing sequence images, characterized in that, include: Step S1: Obtain the focus sequence images to be sorted; Step S2: Using an alternating grouping and iterative reduction sorting and filtering strategy, the focused sequence images are fed into a pre-trained sorting model in batches for processing to generate a sharpness score for each image frame; wherein, the alternating grouping and iterative reduction sorting and filtering strategy includes: The number of groups M is calculated based on the total number of frames in the focused sequence images and the upper limit of the number of frames that the sorting model can process in a single run. The focused sequence images are interleaved and sampled in steps of M groups to divide them into M subgroups; Use all image frames in the M subgroups as the initial candidate image frame set; The candidate image frame set is narrowed down by iterative filtering, wherein the iterative filtering includes: in each iteration, feeding the current candidate image frame set into the ranking model in batches to generate their respective sharpness scores, and selecting one or more image frames with the highest scores from each batch to form a smaller set of candidate image frames for the next iteration, until the number of image frames in the candidate image frame set can be processed by the ranking model in a single step. Step S3: Based on the sharpness score, select the image frame with the highest score as the globally sharpest image frame; The sorting model includes: Feature extraction module: Extracts the depth visual features of a single frame image from the focused image sequence; Multi-scale temporal processing module: Based on the depth visual features, capture the temporal dependencies between the focused sequence images to generate a sharpness score; The ranking model is obtained through a training method that includes the following steps: Obtain training sequences, where each training sequence contains a pre-labeled ground truth most sharp frame; For each image frame in the training sequence, perform automatic sharpness annotation processing using multi-source index fusion to generate a sharpness annotation score; A Top-1 ranking loss strategy is adopted, and the ranking model is trained based on the sharpness label score so that the ranking model predicts a higher score for the sharpest frame in the training sequence than for other frames.

2. The method according to claim 1, characterized in that, The multi-scale temporal processing module includes multiple parallel one-dimensional convolutional layers with different temporal convolutional scales. These one-dimensional convolutional layers are used to extract temporal dependencies of different ranges in the temporal dimension.

3. The method according to claim 1, characterized in that, The feature extraction module is based on a pre-trained convolutional neural network, which is selected from the ResNet series or the EfficientNet series.

4. The method according to claim 1, characterized in that, The automatic clarity annotation process for multi-source indicator fusion includes: Calculate at least two of the following image quality metrics: target quantity, target sharpness, image brightness, and image contrast. A weighted inverse ranking fusion strategy is adopted to generate a sharpness label score based on the ranking results of the at least two image quality indicators.

5. The method according to claim 1, characterized in that, The Top-1 ranking loss strategy includes: The difference between the predicted score of the ranking model for the clearest frame and the predicted scores for other frames is calculated, and the difference is input into the Logistic function to calculate the Top-1 ranking loss.

6. A sharpness ranking system for microscope focusing sequence images, characterized in that, include: Module M1: Acquires the focus sequence images to be sorted; Module M2: Employs a staggered grouping and iterative reduction ranking and filtering strategy to feed the focused sequence images into a pre-trained ranking model in batches for processing, thereby generating a sharpness score for each image frame; wherein, the staggered grouping and iterative reduction ranking and filtering strategy includes: The number of groups M is calculated based on the total number of frames in the focused sequence images and the upper limit of the number of frames that the sorting model can process in a single run. The focused sequence images are interleaved and sampled in steps of M groups to divide them into M subgroups; Use all image frames in the M subgroups as the initial candidate image frame set; The candidate image frame set is narrowed down by iterative filtering, wherein the iterative filtering includes: in each iteration, feeding the current candidate image frame set into the ranking model in batches to generate their respective sharpness scores, and selecting one or more image frames with the highest scores from each batch to form a smaller set of candidate image frames for the next iteration, until the number of image frames in the candidate image frame set can be processed by the ranking model in a single step. Module M3: Based on the sharpness score, select the image frame with the highest score as the globally sharpest image frame; The sorting model includes: Feature extraction module: Extracts the depth visual features of a single frame image from the focused image sequence; Multi-scale temporal processing module: Based on the depth visual features, capture the temporal dependencies between the focused sequence images to generate a sharpness score; The ranking model is obtained through a training method that includes the following steps: Obtain training sequences, where each training sequence contains a pre-labeled ground truth most sharp frame; For each image frame in the training sequence, perform automatic sharpness annotation processing using multi-source index fusion to generate a sharpness annotation score; A Top-1 ranking loss strategy is adopted, and the ranking model is trained based on the sharpness label score so that the ranking model predicts a higher score for the sharpest frame in the training sequence than for other frames.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the sharpness ranking method for microscope focusing sequence images according to any one of claims 1 to 5.

8. A device for sorting the sharpness of microscope focusing sequence images, characterized in that, The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when executed by the processor, the computer program implements the steps of the sharpness ranking method for microscope focusing sequence images according to any one of claims 1 to 5.

Images