A target tracking method and device for microscope cross-magnification images

Abstract

The application relates to the technical field of electron microscopic imaging and motion control technology, and provides a microscope cross-magnification image target tracking method and device, which comprises the following steps: acquiring a first image of a to-be-detected slide under a first magnification of a microscope; in response to an observation instruction, determining initial pixel coordinates of a target observation object in the first image; inputting the initial pixel coordinates into a second-order polynomial model to obtain estimated pixel coordinates; acquiring a second image of the to-be-detected slide under a second magnification and real-time center coordinates of the second image; wherein the first magnification is smaller than the second magnification; calculating an estimated pushing distance according to the real-time center coordinates and the estimated pixel coordinates; and generating real-time indication information according to the real-time center coordinates, the estimated pixel coordinates and the estimated pushing distance and displaying the real-time indication information on the second image. The application can improve the accuracy of the estimated pixel coordinates and the finding efficiency of the target observation object after the magnification of the microscope is improved.

Description

Technical Field

[0001] This application relates to the field of electron microscopy imaging and motion control technology, and in particular to a target tracking method and apparatus for microscope images across magnification. Background Technology

[0002] In parasite egg detection, a multi-magnification strategy of "low-magnification coarse screening + high-magnification fine inspection" is a common practice to balance efficiency and accuracy. Because the field of view shrinks significantly when switching from low to high magnification, targets that were centered in the field of view at low magnification may be outside the field of view at high magnification. Current methods typically involve overlaying crosshairs on the eyepiece or monitor, manually moving the target to the center of the reticle when the field of view decreases at low magnification, remembering the target's position relative to the crosshairs, and then manually moving the slide back to the vicinity of the crosshairs upon switching to high magnification.

[0003] However, the precision of the above operation depends entirely on the operator's spatial memory ability and is affected by mechanical assembly tolerances. The optical axes of the low-magnification and high-magnification objectives of the microscope may not be completely aligned, and there may be a fixed offset of several micrometers to more than ten micrometers. This causes the displacement direction of the current observation target in the field of view after switching to high magnification to be inconsistent with the theory. The existing cross-shaped reticle-assisted scheme cannot handle the nonlinear offset caused by optical aberrations, which further reduces the efficiency of finding the current observation target after switching magnification. Summary of the Invention

[0004] This application provides a target tracking method and apparatus for microscope images across magnification, which can improve the accuracy of estimated pixel coordinates and the efficiency of finding the target object after the microscope magnification is increased.

[0005] In a first aspect, embodiments of this application provide a target tracking method for microscope images across magnification, comprising:

[0006] Acquire the first image of the glass slide to be tested under the microscope at the first magnification;

[0007] In response to the observation command, determine the initial pixel coordinates of the target object in the first image;

[0008] Input the initial pixel coordinates into the second-order polynomial model to obtain the estimated pixel coordinates;

[0009] Acquire the real-time center coordinates of the second image and the second image of the slide under the second magnification; wherein the first magnification is less than the second magnification;

[0010] The estimated pushing distance is calculated based on the real-time center coordinates and the estimated pixel coordinates;

[0011] Real-time indication information is generated based on the real-time center coordinates, estimated pixel coordinates, and estimated pushing distance, and then displayed on the second image.

[0012] In response to the target being found command, stop displaying real-time indicator information.

[0013] Optionally, the method further includes:

[0014] In response to a magnification switching command, focus indication information is generated and displayed on the second image.

[0015] Optionally, the method further includes:

[0016] Multiple sets of historical pixel pairs are obtained, and a second-order polynomial model is trained using the random sampling consensus algorithm and each set of historical pixel pairs; wherein, the historical pixel pairs include the first pixel coordinates at the first magnification and the second pixel coordinates corresponding to the first pixel coordinates at the second magnification.

[0017] Optionally, the real-time indication information includes directional arrows, distance values, crosshairs, multiple concentric circles with different radii centered on the estimated pixel coordinates, and operation prompts.

[0018] Optionally, the method further includes:

[0019] After determining the initial pixel coordinates, the target feature signature of the target observed object in the first image is extracted based on shape features, texture features, relative brightness, and size ratio.

[0020] When the estimated pixel coordinates are located in the second image, all observed objects in the second image are obtained and used as candidate observed objects; candidate feature signatures of each candidate observed object in the second image are extracted based on shape features, texture features, relative brightness, and size ratio.

[0021] Calculate the similarity between the target feature signature and each candidate feature signature, and display the similarity above the candidate observation objects corresponding to the second image.

[0022] Optionally, the method further includes:

[0023] In response to a successful match instruction, the confirmed position coordinates and initial pixel coordinates corresponding to the target found instruction are used as a successful match coordinate pair and added to the successful match set;

[0024] Once the number of successfully matched coordinate pairs in the successful matching set reaches the preset update threshold, the second-order polynomial model is updated based on the successful matching set and multiple sets of historical pixel pairs.

[0025] Optionally, the method further includes: verifying the model error after updating the second-order polynomial model to obtain a model credibility score; if the updated model credibility score is greater than the unupdated model credibility score, then the updated second-order polynomial model is applied, otherwise the unupdated second-order polynomial model is applied.

[0026] Secondly, another embodiment of this application provides a target tracking device for microscope images across magnification, comprising:

[0027] The first acquisition module is used to acquire a first image of the glass slide to be tested under a microscope at a first magnification.

[0028] The low-magnification positioning module is used to determine the initial pixel coordinates of the target object in the first image in response to the observation command;

[0029] The mapping module is used to input the initial pixel coordinates into the second-order polynomial model to obtain the estimated pixel coordinates.

[0030] The second acquisition module is used to acquire the second image of the slide under test at a second magnification and the real-time center coordinates of the second image; wherein the first magnification is less than the second magnification;

[0031] The indicator module is used to calculate the estimated pushing distance based on the real-time center coordinates and the estimated pixel coordinates; generate real-time indicator information based on the real-time center coordinates, the estimated pixel coordinates and the estimated pushing distance, and display the real-time indicator information on the second image;

[0032] The response module is used to stop displaying real-time indicator information in response to the target being found command.

[0033] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the steps of a target tracking method for microscope cross-magnification images as described in any of the above embodiments.

[0034] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a target tracking method for microscope cross-magnification images as described in any of the above embodiments.

[0035] Compared with existing technologies, the target tracking method for microscope images across magnification provided in this application has the following beneficial effects: First, this application establishes a second-order polynomial model to realize nonlinear coordinate mapping after magnification switching, which greatly avoids positional shifts outside the theoretical range caused by lens physical tolerances and improves the accuracy of estimated pixel coordinates; Second, this application generates real-time indication information on the second image by using real-time center coordinates, estimated pixel coordinates, and the estimated push distance between the two, which helps users push the film according to the real-time indication information, realizes assisted guided search of the target object, reduces the difficulty of manual film pushing, and improves the tracking and finding efficiency of the target object after magnification increase. Attached Figure Description

[0036] Figure 1 This is a flowchart of a target tracking method for microscope images across magnification, as described in this application.

[0037] Figure 2 This is a structural diagram of a target tracking device for microscope images across magnification, as described in this application. Detailed Implementation

[0038] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0039] Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not necessarily limited to those explicitly listed, but may include other steps or modules not explicitly listed or inherent to these processes, methods, products, or devices. The module divisions appearing in the embodiments of this application are merely logical divisions; in actual applications, there may be other division methods. For example, multiple modules may be combined into or integrated into another system, or some features may be ignored or not performed. In addition, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces, and the indirect couplings or communication connections between modules may be electrical or other similar forms, none of which are limited in the embodiments of this application. Moreover, the modules or sub-modules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed among multiple circuit modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of this application.

[0040] In parasite egg detection, a multi-magnification strategy of "low-magnification coarse screening + high-magnification fine inspection" is a common practice to balance efficiency and accuracy. Because the field of view shrinks significantly when switching from low to high magnification, targets that were centered in the field of view at low magnification may be outside the field of view at high magnification. Current methods typically involve overlaying crosshairs on the eyepiece or monitor, manually moving the target to the center of the reticle when the field of view decreases at low magnification, remembering the target's position relative to the crosshairs, and then manually moving the slide back to the vicinity of the crosshairs upon switching to high magnification.

[0041] However, the precision of the above operation depends entirely on the operator's spatial memory ability and is affected by mechanical assembly tolerances. The optical axes of the low-magnification and high-magnification objectives of the microscope may not be completely aligned, and there may be a fixed offset of several micrometers to more than ten micrometers. This causes the displacement direction of the current observation target in the field of view after switching to high magnification to be inconsistent with the theory. The existing cross-shaped reticle-assisted scheme cannot handle the nonlinear offset caused by optical aberrations, which further reduces the efficiency of finding the current observation target after switching magnification.

[0042] To resolve the above issues, please refer to [link / reference]. Figure 1 This application provides a target tracking method for microscope images across magnification, applied to a terminal with a display device. The terminal is connected to a microscope camera, which is connected to a multi-magnification microscope. Preferably, the microscope camera has a resolution of at least 2 megapixels. The method mainly includes:

[0043] Step S1: Obtain the first image of the slide to be tested under the microscope at the first magnification.

[0044] Step S2: In response to the observation command, determine the initial pixel coordinates of the target object in the first image.

[0045] The initial pixel coordinates are the coordinates of the center pixel of the target object in the first image.

[0046] Step S3: Input the initial pixel coordinates into the second-order polynomial model to obtain the estimated pixel coordinates.

[0047] This embodiment uses a first magnification of 10x and a second magnification of 40x as examples. It is assumed that the initial pixel coordinates of the target object at 10x magnification are... This application employs a second-order polynomial model with radial distortion to establish a nonlinear mapping relationship between the two scaling factors:

[0048]

[0049]

[0050] in, This refers to the estimated pixel coordinates of the target object in a coordinate system 40 times larger than the image coordinate system. The mapping coefficients to be estimated are: This is the squared term of the radial distance.

[0051] In the specific implementation process, in addition to second-order polynomial model mapping, neural network end-to-end mapping method or lookup table interpolation method can also be used. Among them, neural network end-to-end mapping can use lightweight MLP to directly learn coordinate mapping. Although it has strong fitting ability, it requires a large amount of labeled data. When the sample size is small, it is unstable and cannot be interpreted. Lookup table interpolation method establishes a lookup table through uniform sampling. If it is a new position, interpolation calculation is performed. Its advantage is that it is intuitive and interpretable, but it requires pre-calibration of sampling. When the coverage is incomplete, the error is large.

[0052] Step S4: Obtain the real-time center coordinates of the second image and the second image of the slide under the second magnification; wherein, the first magnification is less than the second magnification.

[0053] It should be noted that the first image and the second image are images of the glass slide to be tested at the first magnification and the second magnification of the microscope, respectively. They are acquired by the microscope camera and transmitted to the terminal for display and processing.

[0054] Step S5: Calculate the estimated pushing distance based on the real-time center coordinates and the estimated pixel coordinates.

[0055] Assume the coordinates of the center of the field of view of the second image, i.e., the real-time center coordinates, are... The coordinates can be obtained directly from the camera resolution; the estimated pushing distance, i.e., the offset required due to mechanical tolerances, is:

[0056]

[0057] The estimated propulsion distance directly informs the user of the pixel deviation of the observed target object relative to the real-time center coordinates in the X and Y directions. To further enhance intuitiveness, the estimated propulsion distance is converted into physical distance (micrometers):

[0058]

[0059] in, and The pixel size at the second magnification can be obtained via a ruler or factory settings.

[0060] Step S6: Generate real-time indication information based on the real-time center coordinates, estimated pixel coordinates, and estimated pushing distance, and display the real-time indication information on the second image.

[0061] The real-time indication information includes directional arrows, distance values, crosshair reference lines, multiple concentric rings with different radii centered on the estimated pixel coordinates, and operation prompts.

[0062] In practice, when the microscope is switched from low to high magnification, the field of view is greatly reduced. This means that the target object, which is centered in the field of view at low magnification, may be outside the field of view at high magnification. In addition, the field of view center may be offset due to mechanical assembly tolerances, making it difficult for inexperienced users to find the target object at high magnification by manually pushing the slide. This results in a significant increase in the time required for a single inspection. To address this, this application calculates the estimated pixel coordinates at high magnification using a second-order polynomial model after locking the target object at low magnification, rather than simply scaling the magnification. At the same time, in the absence of a motorized stage, real-time instructions inform the user which direction and how far the stage should move, guiding the user to manually push the slide until the target object is found in the displayed second image.

[0063] In practice, after switching to high magnification, a directional arrow, distance value, and crosshair reference line are overlaid on the real-time updated second image. The directional arrow points from the center of the field of view to the estimated pixel coordinates, with its length normalized to indicate direction, and uses a bright color (such as red or bright green) for easy viewing. The distance value is displayed next to the arrow. and The system provides specific numerical and directional descriptions, such as "Please move approximately 120µm to the upper right as indicated by the arrow." A crosshair reference line is superimposed on the center of the second image's field of view to help the user determine the current center position. Furthermore, multiple concentric rings with different radii can be drawn on the second image, centered on the estimated pixel coordinates (radii can be selected as 0.5, 1.0, 1.5, or 2.0 times the estimated pushing distance), guiding the user to search ring by ring from the inside out, avoiding blindly pushing the image too much. Operation prompts can also be dynamically displayed in the corners of the display screen, such as: "The target is outside the field of view, please move in the direction of the arrow," "The target may have entered the edge of the field of view, please carefully examine the ring area" (when concentric rings are displayed), or "Please search for the target within the field of view" (when the estimated pixel coordinates enter the second image).

[0064] Furthermore, if the terminal used in this application has an audio player or other related functional modules, real-time indication information can be broadcast via voice, but the effect is poor in noisy environments and the sense of direction is not as intuitive as visual.

[0065] Step S7: In response to the target found command, stop displaying real-time indication information.

[0066] After the user finds the target, they click the "Target found" button. At this time, the terminal responds to the target found instruction and records the actual confirmed position coordinates of the target object at the second magnification (obtained by clicking or selecting with the mouse).

[0067] Compared with existing technologies, the target tracking method for microscope images across magnification provided in this application has the following beneficial effects: First, this application establishes a second-order polynomial model to realize nonlinear coordinate mapping after magnification switching, which greatly avoids positional shifts outside the theoretical range caused by lens physical tolerances and improves the accuracy of estimated pixel coordinates; Second, this application generates real-time indication information on the second image by using real-time center coordinates, estimated pixel coordinates, and the estimated push distance between the two, which helps users push the film according to the real-time indication information, realizes assisted guided search of the target object, reduces the difficulty of manual film pushing, and improves the tracking and finding efficiency of the target object after magnification increase.

[0068] In some embodiments, the method further includes:

[0069] In response to a magnification switching command, focus indication information is generated and displayed on the second image.

[0070] Specifically, a target object that is clear at the first magnification may be out of focus at the second magnification. Therefore, a sharpness score is calculated for the target object using Laplace variance during the first magnification test, and then a sharpness score is calculated for the target object after magnification during the second magnification test. The amount of defocus X can be estimated based on the difference between the two scores, and the direction is determined based on the change of the difference. Then, a focusing prompt message is displayed on the screen: "It is recommended to fine-tune the Z-axis by about ±X micrometers" as a focusing reference.

[0071] The Z-axis adjustment here corresponds to the fine-tuning knob on a manual microscope. Therefore, X, in practical operation, tells the user how many turns to make on the fine-tuning knob. In practice, the sensitivity of the fine-tuning knob is fixed for each microscope. Dividing X (in micrometers) by this sensitivity gives the number of turns, indicated by ± (positive up, negative down). Because X is on the order of micrometers, it can only be used for fine-tuning. Coarse adjustments would be on the order of millimeters and would directly overshoot the focal plane, therefore coarse adjustments are not possible.

[0072] The above embodiment generates focusing prompt information after the microscope switches from low magnification to high magnification, avoiding defocusing that may occur after magnification switching and further improving the tracking efficiency of the target object.

[0073] In some embodiments, the method further includes: acquiring multiple sets of historical pixel pairs, and training a second-order polynomial model using a random sampling consensus algorithm and each set of historical pixel pairs; wherein, the historical pixel pair includes the coordinates of a first pixel at a first magnification and the coordinates of a second pixel corresponding to the first pixel at a second magnification.

[0074] Random Sample Consensus (RANSAC) is a robust parameter estimation method used to estimate the correct mathematical model from a dataset containing outliers. Its basic principle is as follows: Candidate models are constructed by randomly sampling a small number of data points (historical pixel pairs in this application); the number of data points (inliers) that conform to the model is counted; after multiple iterations, the model with the most inliers is selected as the optimal model; finally, the model parameters are re-estimated using all inliers.

[0075] In this application, RANSAC is used to remove erroneous training samples caused by human error such as mispointing or misidentifying targets during manual labeling, thereby ensuring the accuracy of coordinate mapping coefficient estimation.

[0076] Specifically, existing successful tracking cases (i.e., records where users manually find the target and confirm a match) are used as training samples, and the mapping coefficients of the second-order polynomial model are solved using the least squares method with RANSAC (Random Sample Consensus Algorithm). After each successful tracking by the user, the first pixel coordinate at the first magnification and the second pixel coordinate at the second magnification are recorded, gradually accumulating historical pixel pairs.

[0077] N sets of successfully tracked historical pixel pairs were collected:

[0078]

[0079] Construct a design matrix, where the i-th row corresponds to the i-th group of historical pixel pairs, containing the following 7 elements:

[0080]

[0081] Then solve for the coefficient vector:

[0082]

[0083] Obtain the mapping coefficients to be estimated .

[0084] Finally, the random sampling consensus algorithm is used to iteratively remove outliers (erroneous records caused by user misjudgment) until the coefficients converge, resulting in a trained second-order polynomial model that is then deployed.

[0085] The above embodiment trains a second-order polynomial model using historical pixel pairs obtained through manual operation, realizing nonlinear mapping of coordinates between low and high magnification. This solves the problem that the target observation object cannot be quickly found by simply scaling according to the magnification after the magnification is switched due to the nonlinear offset of the optical axis introduced by mechanical assembly tolerance. At the same time, the random sampling consistency algorithm used in the model iterative training process avoids the reduction in model mapping accuracy caused by possible user error records in the training data.

[0086] Furthermore, in the specific implementation process, there may be multiple observation objects with similar shapes near the estimated pixel coordinates. At this time, it is difficult for the user to determine whether the target observation object found is the same as the target observation object in the first image. It is easy to mistakenly identify other similar objects as the target, resulting in the error of the subsequent analysis object.

[0087] To address this issue, in some embodiments, the method further includes:

[0088] Step S01: After determining the initial pixel coordinates, extract the target feature signature of the target observed object in the first image based on shape features, texture features, relative brightness, and size ratio.

[0089] Specifically, at the first magnification, the following features are extracted and concatenated to form the target feature signature. :

[0090] 1. Shape feature signature: Fourier descriptor of normalized contour (first 16 coefficients), which has rotation, scaling and translation invariance.

[0091] 2. Texture feature signature: Local binary pattern uniform pattern histogram (59 dimensions).

[0092] 3. Relative brightness signature: The ratio of the grayscale mean of a 3×3 grid to the global mean.

[0093] 4. Size and proportion signature: aspect ratio of the detection frame, and area-to-perimeter ratio.

[0094] The above features are concatenated into an approximately 80-dimensional vector. .

[0095] Step S02: When the estimated pixel coordinates are located in the second image, all observation objects in the second image are obtained and used as candidate observation objects; candidate feature signatures of each candidate observation object in the second image are extracted based on shape features, texture features, relative brightness and size ratio.

[0096] Specifically, after the user pushes the image to the vicinity of the estimated pixel coordinates, all observed objects are detected in the field of view at the second magnification, and the same feature vector is extracted for each candidate observed object. , as candidate feature signatures.

[0097] Step S03: Calculate the similarity between the target feature signature and each candidate feature signature, and display the similarity on the candidate observation object corresponding to the second image.

[0098] For each candidate feature signature, calculate its similarity to the target feature signature:

[0099]

[0100] Here, cos() is the cosine similarity, and the weights w are learned through the validation set. 10 The size ratio signature value of the target observation object at the first magnification is obtained by combining the aspect ratio of the detection box and the perimeter ratio of the area of ​​the target observation object in the first image into a scalar value (e.g., by taking a weighted average or reducing the dimensionality after stitching); ratio 40 (j) The size ratio signature value of the j-th candidate observation object at the second magnification is obtained by combining the aspect ratio of the detection box and the perimeter ratio of the area of ​​the candidate observation object in the second image.

[0101] After obtaining the similarity score for each candidate observation, each candidate observation in the second image is outlined with a different color and its corresponding similarity score is labeled:

[0102] 1. >75%: Green box + "Highly matched, recommended to confirm first".

[0103] 2.50%~75%: Yellow box + "Possible match, please make a manual judgment".

[0104] 3. <50%: Gray box + "Mismatch".

[0105] In addition to the methods mentioned above for calculating similarity, feature matching can also be performed using deep learning feature matching or traditional classifiers. Deep learning feature matching can use the Siamese network to directly output cross-sizing similarity, which has higher accuracy, but requires a large amount of paired data for training and is difficult to deploy at the edge. Traditional classifiers extract features and then use SVM for classification, which is simple to implement, but requires labeled data and has general cross-sizing generalization ability.

[0106] Users can quickly locate a target object by referring to similarity data when searching for it. After the user enters the target location command, the system simultaneously records the focus prompt information and / or the similarity information of the identified target object.

[0107] The above embodiments achieve cross-magnification feature matching verification by extracting morphological feature vectors as the "identity fingerprint" of the same observation object, helping users to quickly identify the target observation object when there are many observation objects.

[0108] In some embodiments, the method further includes:

[0109] Step S81: In response to the successful matching instruction, the confirmed position coordinates and initial pixel coordinates corresponding to the target found instruction are taken as a successful matching coordinate pair and added to the successful matching set.

[0110] Specifically, the user determines whether the second-order polynomial model is successfully matched by actively judging the difference between the confirmed position coordinates of the target observed object in the second image and the estimated pixel coordinates mapped by the computer. If the user considers it a successful match, a successfully matched coordinate pair is recorded. Confirmed location coordinates .here The coordinates are obtained by the user clicking on the second image (the pixel center coordinates are automatically calculated as the confirmation position coordinates after clicking or selecting with the mouse). If the user thinks that there is no match, the corresponding coordinate pair can be marked as a mismatch coordinate pair and stored in the difficult example set for subsequent analysis.

[0111] Step S82: When the number of successfully matched coordinate pairs in the successful matching set reaches the preset update threshold, update the second-order polynomial model based on the successful matching set and multiple sets of historical pixel pairs.

[0112] Specifically, once the number of successfully matched coordinate pairs in the successful matching set accumulates to M (M can be 30~50), the model is updated. The coefficients are re-estimated using the weighted least squares method, and outliers are removed using RANSAC. In this process, the weight of the successful matching set is higher than the weight of the historical pixel pairs.

[0113] The above embodiments enable the second-order polynomial model to be gradually optimized using accumulated data in practical applications, so that the model's mapping accuracy becomes higher and higher as it is used, and the estimated pixel coordinates become more and more accurate.

[0114] In some embodiments, the method further includes: verifying the model error after updating the second-order polynomial model to obtain a model credibility score; if the updated model credibility score is greater than the unupdated model credibility score, then the updated second-order polynomial model is applied, otherwise the unupdated second-order polynomial model is applied.

[0115] Specifically, after updating the model, the error of the new model can be verified using a validation set that includes multiple sets of pixel pairs. If the new model is better than the old model, it is deployed; otherwise, the old model is retained, and relevant prompts are given on the terminal based on the error.

[0116] 1. Green: Error <100μm, usually found in one push-up.

[0117] 2. Yellow: Error 100~200μm, may require searching 2~3 fields of view.

[0118] 3. Red: Error > 200μm, displaying the prompt message: "Model accuracy is low, it is recommended to accumulate more tracking data" or "Please check if the objective lens is securely installed".

[0119] Please refer to Figure 2Another embodiment of this application provides a target tracking device for microscope images across magnification, which mainly includes the following modules:

[0120] The first acquisition module 101 is used to acquire a first image of the glass slide to be tested under a microscope at a first magnification.

[0121] The low-magnification positioning module 102 is used to determine the initial pixel coordinates of the target object in the first image in response to an observation command.

[0122] The mapping module 103 is used to input the initial pixel coordinates into the second-order polynomial model to obtain the estimated pixel coordinates.

[0123] The second acquisition module 104 is used to acquire the second image of the glass slide under the second magnification and the real-time center coordinates of the second image; wherein the first magnification is less than the second magnification.

[0124] The indicator module 105 is used to calculate the estimated pushing distance based on the real-time center coordinates and the estimated pixel coordinates; generate real-time indicator information based on the real-time center coordinates, the estimated pixel coordinates and the estimated pushing distance, and display the real-time indicator information on the second image.

[0125] The response module 106 is used to stop displaying real-time indication information in response to the target found command.

[0126] In some embodiments, the device further includes a focus prompting module for:

[0127] In response to a magnification switching command, focus indication information is generated and displayed on the second image.

[0128] In some embodiments, the apparatus further includes a model training module for:

[0129] Multiple sets of historical pixel pairs are obtained, and a second-order polynomial model is trained using the random sampling consensus algorithm and each set of historical pixel pairs; wherein, the historical pixel pairs include the first pixel coordinates at the first magnification and the second pixel coordinates corresponding to the first pixel coordinates at the second magnification.

[0130] In some embodiments, the device further includes a feature verification module for:

[0131] After determining the initial pixel coordinates, the target feature signature of the target observed object in the first image is extracted based on shape features, texture features, relative brightness, and size ratio.

[0132] When the estimated pixel coordinates are located in the second image, all observed objects in the second image are obtained and used as candidate observed objects; candidate feature signatures of each candidate observed object in the second image are extracted based on shape features, texture features, relative brightness, and size ratio.

[0133] Calculate the similarity between the target feature signature and each candidate feature signature, and display the similarity above the candidate observation objects corresponding to the second image.

[0134] In some embodiments, the device further includes a feedback update module for:

[0135] In response to a successful match instruction, the confirmed position coordinates and initial pixel coordinates corresponding to the target found instruction are used as a successful match coordinate pair and added to the successful match set;

[0136] Once the number of successfully matched coordinate pairs in the successful matching set reaches the preset update threshold, the second-order polynomial model is updated based on the successful matching set and multiple sets of historical pixel pairs.

[0137] In some embodiments, the device further includes an error verification module, configured to:

[0138] After updating the second-order polynomial model, the model error is verified to obtain the model confidence score. If the updated model confidence score is greater than the unupdated model confidence score, the updated second-order polynomial model is applied; otherwise, the unupdated second-order polynomial model is applied.

[0139] The specific limitations of the target tracking device for microscope images across magnification provided in this embodiment can be found in the embodiment of the target tracking method for microscope images across magnification described above, and will not be repeated here. Each module in the aforementioned target tracking device for microscope images across magnification can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0140] This application provides a computer device that may include a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it causes the processor to perform the steps of a target tracking method for microscope cross-magnification images as described in any of the above embodiments.

[0141] The working process, working details, and technical effects of the computer device provided in this embodiment can be found in the embodiment of a target tracking method for microscope images across magnification described above, and will not be repeated here.

[0142] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the steps of a target tracking method for microscope images across magnification as described in any of the above embodiments. The computer-readable storage medium refers to a data storage carrier, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or memory sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The working process, details, and technical effects of the computer-readable storage medium provided in this embodiment can be found in the embodiments of a target tracking method for microscope images across magnification described above, and will not be repeated here.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0144] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A target tracking method for microscope images across magnification, characterized in that, include: Acquire the first image of the glass slide to be tested under the microscope at the first magnification; In response to an observation command, the initial pixel coordinates of the target object in the first image are determined. The initial pixel coordinates are input into a second-order polynomial model to obtain the estimated pixel coordinates; Acquire a second image of the slide to be tested at a second magnification and the real-time center coordinates of the second image; wherein the first magnification is less than the second magnification; The estimated pushing distance is calculated based on the real-time center coordinates and the estimated pixel coordinates; Real-time indication information is generated based on the real-time center coordinates, the estimated pixel coordinates, and the estimated pushing distance, and the real-time indication information is displayed on the second image. In response to the target being found command, the display of the real-time indication information is stopped.

2. The target tracking method for microscope images across magnification according to claim 1, characterized in that, Also includes: In response to a magnification switching command, focus indication information is generated and displayed on the second image.

3. The target tracking method for microscope images across magnification according to claim 1, characterized in that, Also includes: Multiple sets of historical pixel pairs are obtained, and a second-order polynomial model is trained using a random sampling consensus algorithm and each set of historical pixel pairs; wherein, the historical pixel pair includes the first pixel coordinates at a first magnification and the second pixel coordinates corresponding to the first pixel coordinates at a second magnification.

4. The target tracking method for microscope images across magnification according to claim 1, characterized in that, The real-time indication information includes directional arrows, distance values, crosshairs, multiple concentric rings with different radii centered on the estimated pixel coordinates, and operation prompts.

5. The target tracking method for microscope images across magnification according to claim 1, characterized in that, Also includes: After determining the initial pixel coordinates, the target feature signature of the target observed object in the first image is extracted based on shape features, texture features, relative brightness, and size ratio. When the estimated pixel coordinates are located in the second image, all observed objects in the second image are obtained and used as candidate observed objects; candidate feature signatures of each of the candidate observed objects in the second image are extracted based on shape features, texture features, relative brightness, and size ratio. Calculate the similarity between the target feature signature and each of the candidate feature signatures, and display the similarity on the candidate observation object corresponding to the second image.

6. The target tracking method for microscope images across magnification according to claim 3, characterized in that, Also includes: In response to a successful match instruction, the confirmed location coordinates corresponding to the target found instruction and the initial pixel coordinates are used as a successful match coordinate pair and added to the successful match set; When the number of successfully matched coordinate pairs in the successful matching set reaches a preset update threshold, the second-order polynomial model is updated based on the successful matching set and multiple sets of historical pixel pairs.

7. The target tracking method for microscope images across magnification according to claim 6, characterized in that, Also includes: After updating the second-order polynomial model, the model error is verified to obtain a model credibility score. If the updated model credibility score is greater than the unupdated model credibility score, the updated second-order polynomial model is applied; otherwise, the unupdated second-order polynomial model is applied.

8. A target tracking device for microscope images across magnification, characterized in that, include: The first acquisition module is used to acquire a first image of the glass slide to be tested under a microscope at a first magnification. The low-magnification positioning module is used to determine the initial pixel coordinates of the target object in the first image in response to the observation command; The mapping module is used to input the initial pixel coordinates into the second-order polynomial model to obtain the estimated pixel coordinates. The second acquisition module is used to acquire a second image of the slide to be detected at a second magnification and the real-time center coordinates of the second image; wherein the first magnification is less than the second magnification; The indicator module is used to calculate the estimated pushing distance based on the real-time center coordinates and the estimated pixel coordinates; generate real-time indicator information based on the real-time center coordinates, the estimated pixel coordinates and the estimated pushing distance, and display the real-time indicator information on the second image; The response module is used to stop displaying the real-time indication information in response to the target finding command.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the target tracking method for microscope cross-magnification images as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the target tracking method for microscope cross-magnification images as described in any one of claims 1 to 7.

Images