Medical instrument tracking method and device, electronic equipment and storage medium
By dividing and labeling video frames in medical device tracking scenarios, the error detection or missed detection problems caused by device deformation, occlusion or low resolution in traditional methods are solved, and higher tracking accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202510333155.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-20
- Publication Date
- 2025-07-18
AI Technical Summary
Traditional medical device tracking methods have shortcomings in real-time or small-object detection, and are often caused by mis-detection or missed detection due to device deformation, occlusion or low resolution.
By dividing the video frames in the medical device tracking scene, multiple image blocks are formed, and each image block is marked using the medical device annotation model, and the tracking area is expanded with adjacent image blocks to improve detection accuracy.
It significantly improves tracking accuracy and robustness in low resolution and noise environments, and avoids missed detection or missed detection due to device deformation or occlusion.
Smart Images
Figure CN120339577A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image processing, and in particular, to a tracking method, device, electronic device and storage medium for medical devices. Background Art
[0002] The tracking of medical devices has been widely applied in the medical field. Traditional medical field tracking methods include SIFT (Scale-Invariant Feature Transform), SURF (Speeded-Up Robust Features), and object detection methods based on deep learning (such as YOLO, Faster R-CNN), etc. However, traditional medical device tracking methods have deficiencies in real-time performance or small target detection, and often result in false detections or missed detections due to instrument deformation, occlusion, or low resolution. Summary of the Invention
[0003] In view of this, the purpose of the present application is to provide a tracking method, device, electronic device and storage medium for medical devices, which can track each image block in the current video frame in real time, and expand the tracking area through adjacent image blocks, avoiding false detections or missed detections caused by instrument deformation, occlusion, or low resolution.
[0004] In a first aspect, an embodiment of the present application provides a tracking method for medical devices, the method includes:
[0005] Divide the current video frame in the medical device tracking scene according to a preset division size to obtain a plurality of first image blocks;
[0006] Determine the first image blocks at the latest tracking position in the current video frame and the first image blocks adjacent to the first image blocks at the latest tracking position as second image blocks; wherein, the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is latest tracked;
[0007] Input each second image block into a medical device annotation model to annotate the medical device area in each second image block, and obtain a medical device annotation image corresponding to each second image block; the medical device annotation model is trained through medical device sample images and corresponding actual medical device annotation images;
[0008] Determine the medical device area in the current video frame according to the medical device annotation images corresponding to all second image blocks and all first image blocks.
[0009] In a possible implementation, determining, as the second image block, the first image block at the latest tracking position in the current video frame and the first image block adjacent to the first image block at the latest tracking position includes:
[0010] Matching each first image block with the medical device region image in the video frame where the medical device region is latest tracked, to obtain a matching result corresponding to each first image block;
[0011] If the matching results corresponding to all the first image blocks are all matching failures, then determining, as the second image block, the first image block at the latest tracking position in the current video frame and the first image block adjacent to the first image block at the latest tracking position.
[0012] In a possible implementation, the method further includes:
[0013] If there is a first image block with a matching result of successful matching, then determining, as the third image block, the first image block with a successful matching and the first image block adjacent to the first image block with a successful matching in the current video frame;
[0014] Inputting each third image block into a medical device annotation model to annotate the medical device region in each third image block, to obtain a medical device annotation image corresponding to each third image block;
[0015] Determining the medical device region in the current video frame according to the medical device annotation images corresponding to all the third image blocks and all the first image blocks.
[0016] In a possible implementation, determining the medical device region in the current video frame includes:
[0017] Determining all the medical device annotation images as the latest candidate medical device annotation images;
[0018] Judging whether there is a latest candidate medical device annotation image annotated with a medical device region;
[0019] If there is, then determining the latest medical device annotation image annotated with a medical device region as the latest target medical device annotation image;
[0020] Inputting the first image block adjacent to the latest target medical device annotation image in the current video frame into the medical device annotation model to annotate the medical device region, to obtain the latest candidate medical device annotation image; and jumping to the step of judging whether there is a latest candidate medical device annotation image annotated with a medical device region to continue execution;
[0021] If not, all the medical device regions marked by the medical device marking model are determined as the medical device regions in the current video frame.
[0022] In a possible implementation, the medical device marking model is trained through the following steps;
[0023] Obtain medical device sample images and corresponding actual medical device marking images;
[0024] Input the medical device sample images into the medical device marking model to mark the medical device regions in the medical device sample images, obtaining predicted medical device marking images;
[0025] Train the medical device marking model according to the medical device sample images, the corresponding actual medical device marking images and the predicted medical device marking images.
[0026] In a possible implementation, the inputting the medical device sample images into the medical device marking model to mark the medical device regions in the medical device sample images, obtaining predicted medical device marking images, includes:
[0027] Preprocess the medical device sample images to obtain a first medical device sample feature map;
[0028] Adopt multiple convolution blocks with the same area but different sizes to perform convolution on the medical device sample feature map after max pooling respectively, obtaining multiple second medical device sample feature maps corresponding to different sizes;
[0029] Calculate the target medical device sample features corresponding to each size according to the second medical device sample feature maps corresponding to each size;
[0030] Fuse the target medical device sample features corresponding to all sizes and the first medical device sample feature map after standard convolution to obtain a predicted medical device marking image.
[0031] In a possible implementation, the calculating the target medical device sample features corresponding to each size according to the second medical device sample feature maps corresponding to each size, includes:
[0032] Calculate the offset of each second medical device sample feature map of the size relative to the medical device sample feature map after max pooling;
[0033] Perform bilinear interpolation calculation based on the offset of the medical device sample feature map after maximum pooling and the offsets of all second medical device sample feature maps of the size relative to the medical device sample feature map after maximum pooling to obtain the initial medical device sample feature corresponding to the size;
[0034] Perform standard convolution operations and upsampling operations respectively on the initial medical device sample feature corresponding to the size to obtain the target medical device sample feature corresponding to the size.
[0035] In a second aspect, an embodiment of the present application further provides a tracking device for a medical device. The device includes:
[0036] A partitioning module, configured to partition a current video frame in a medical device tracking scenario according to a preset partitioning size to obtain a plurality of first image blocks;
[0037] A determining module, configured to determine, as second image blocks, the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position; wherein, the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is latest tracked;
[0038] A labeling module, configured to input each second image block into a medical device labeling model to label the medical device area in each second image block to obtain a medical device labeling image corresponding to each second image block; the medical device labeling model is trained by using medical device sample images and corresponding actual medical device labeling images;
[0039] The determining module is further configured to determine the medical device area in the current video frame according to the medical device labeling images corresponding to all second image blocks and all first image blocks.
[0040] In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the tracking method for a medical device according to any one of the first aspects.
[0041] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it performs the steps of the tracking method for a medical device according to any one of the first aspects.
[0042] An embodiment of the present application provides a method, apparatus, electronic device, and storage medium for tracking medical devices. The method includes: dividing a current video frame in a medical device tracking scenario according to a preset division size to obtain a plurality of first image blocks; determining, as second image blocks, the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position, where the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is latest tracked; inputting each second image block into a medical device annotation model to annotate the medical device area in each second image block to obtain a medical device annotation image corresponding to each second image block; the medical device annotation model is trained by using medical device sample images and corresponding actual medical device annotation images; determining the medical device area in the current video frame according to the medical device annotation images corresponding to all second image blocks and all first image blocks. The present application can improve the tracking effect of medical devices by tracking each image block in the current video frame separately. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.
[0044] Figure 1 The flowchart of a method for tracking a medical device provided by an embodiment of the present application is shown;
[0045] Figure 2 The flowchart of another method for tracking a medical device provided by an embodiment of the present application is shown;
[0046] Figure 3 The structural schematic diagram of a device for tracking a medical device provided by an embodiment of the present application is shown;
[0047] Figure 4 The structural schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application are only for the purposes of illustration and description, and are not used to limit the protection scope of this application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.
[0049] In addition, the described embodiments are only some embodiments of this application, rather than all embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application that is required to be protected, but only represents the selected embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.
[0050] To enable those skilled in the art to use the content of this application, the following implementation manners are given in combination with a specific application scenario, the "autonomous driving field". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around the "autonomous driving field", it should be understood that this is only an exemplary embodiment.
[0051] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the features stated later, but does not exclude adding other features.
[0052] The following provides a detailed description of a tracking method for a medical device provided in the embodiments of this application.
[0053] Referring to Figure 1 As shown, it is a schematic flowchart of a tracking method for a medical device provided in the embodiments of this application. The following explains each exemplary step in the embodiments of this application:
[0054] S101: Divide the current video frame in the medical device tracking scenario according to a preset division size to obtain a plurality of first image blocks.
[0055] In the embodiment of the present application, the current video frame is obtained in real time from the video stream in the medical device tracking scenario; the current video frame is divided according to a preset division size (such as 256×256) to obtain a plurality of non-overlapping first image blocks.
[0056] Among them, the medical device tracking scenario can be any scenario that requires medical device tracking, such as an ophthalmic surgery; the size of the first image block is the same as the preset division size.
[0057] It should be noted that during the process of dividing the current video frame according to the preset division size, if the situation where the current video frame cannot be completely divided occurs, the method of supplementing pixel points with pixel value 0 can be adopted to complete the division operation of all the content of the current video frame.
[0058] Here, dividing the current video frame into a plurality of first image blocks facilitates subsequent independent processing of each first image block to adapt to different resolutions and local changes, improving the tracking efficiency and anti-noise performance.
[0059] S102. Determine the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position as the second image blocks; among them, the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is latest tracked.
[0060] In the embodiment of the present application, the video frame where the medical device area is latest tracked can be understood as the latest video frame in which the medical device area is determined.
[0061] Example: The video frame where the medical device area is latest tracked is A, and the position of the medical device area in video frame A is X. Position X refers to the image position of the medical device in the video frame, rather than the actual position in the medical device tracking scenario. Assume that the first image block at position X in the current video frame is M, and the first image blocks adjacent to the first image block M in the current video frame include N, P, and Q. Therefore, the first image block M, the first image block N, the first image block P, and the first image block Q are all determined as the second image blocks.
[0062] In addition, in order to prevent the medical device from re-entering the medical device tracking scenario after exiting the medical device tracking scenario, it is necessary to ensure that the duration from the acquisition time of the video frame where the medical device area is latest tracked to the current time is less than or equal to the preset duration.
[0063] S103. Input each second image block into the medical device annotation model to annotate the medical device area in each second image block, and obtain the medical device annotation image corresponding to each second image block.
[0064] In the embodiment of the present application, the medical device annotation model is trained through medical device sample images and corresponding actual medical device annotation images. The medical device annotation model can annotate the area and type (which can be the name) of the medical device in the image.
[0065] Optionally, determining the second image block from the first image block at the latest tracking position in the current video frame and the first image block adjacent to the first image block at the latest tracking position includes:
[0066] Step 1: Match each first image block with the medical device area image in the video frame where the medical device area is latest tracked to obtain the matching result corresponding to each first image block.
[0067] In the embodiment of the present application, for any first image block, calculate the similarity score between the first image block and the video frame where the medical device area is latest tracked; if the similarity score between the first image block and the video frame where the medical device area is latest tracked is greater than or equal to the preset score, the matching result corresponding to the first image block is a successful match; if the similarity score between the first image block and the video frame where the medical device area is latest tracked is less than the preset score, the matching result corresponding to the first image block is a failed match.
[0068] Among them, the similarity score can be calculated through the Euclidean distance or correlation measure.
[0069] Here, the matching result includes a successful match and a failed match. If the first image block successfully matches the medical device area image, it means that the first image block includes the medical device area image. If the first image block fails to match the medical device area image, it means that the first image block does not include the medical device area image.
[0070] Step 2: If the matching results corresponding to all the first image blocks are failed matches, determine the first image block at the latest tracking position in the current video frame and the first image block adjacent to the first image block at the latest tracking position as the second image block.
[0071] In the embodiment of the present application, if the matching results corresponding to the first image blocks are all failed matches, it may be due to partial occlusion of the medical device. Therefore, it is necessary to spread out from the latest tracking position in the current video frame to obtain the second image block and continue to track the medical device to improve the tracking accuracy.
[0072] Furthermore, if there are first image blocks with a matching result of successful match, the method further includes:
[0073] Step 1: Determine the third image blocks by taking the successfully matched first image blocks and the first image blocks adjacent to the successfully matched first image blocks in the current video frame.
[0074] In the embodiments of the present application, since the surgical instruments are moving in the video frame, the medical device areas in some of the first image blocks are small, resulting in failed matching. Therefore, it is necessary to obtain the third image blocks by spreading outwards based on the successfully matched first image blocks, and dynamically update the medical device areas to ensure the accuracy and real-time nature of the tracking.
[0075] Step 2: Input each third image block into the medical device annotation model to annotate the medical device area in each third image block, obtaining the medical device annotation image corresponding to each third image block.
[0076] Step 3: Determine the medical device area in the current video frame based on the medical device annotation images corresponding to all the third image blocks and all the first image blocks.
[0077] S104: Determine the medical device area in the current video frame based on the medical device annotation images corresponding to all the second image blocks and all the first image blocks.
[0078] Further, determining the medical device area in the current video frame includes:
[0079] Step 1: Determine all the medical device annotation images as the latest candidate medical device annotation images.
[0080] In the embodiments of the present application, this refers to determining the medical device annotation images corresponding to all the second image blocks or all the third image blocks as the latest candidate medical device annotation images.
[0081] Step 2: Determine whether there is a latest candidate medical device annotation image marked with a medical device area.
[0082] Step 3: If there is, determine the latest medical device annotation image marked with a medical device area as the latest target medical device annotation image.
[0083] In the embodiments of the present application, if there is a latest candidate medical device annotation image marked with a medical device area, it is necessary to continue spreading outwards to obtain the boundary of the medical device area.
[0084] Step 4: Input the first image blocks adjacent to the latest target medical device annotation image in the current video frame into the medical device annotation model to annotate the medical device area, obtaining the latest candidate medical device annotation image; and jump to determine whether there is a latest candidate medical device annotation image marked with a medical device area to continue the execution.
[0085] Step 5: If not, determine all the medical device regions marked by the medical device annotation model as the medical device regions in the current video frame.
[0086] In the embodiment of the present application, if not, it indicates that there is no medical device region in the current video frame or the boundary of the medical device region has been found.
[0087] Further, the tracking process of S101 to S104 is for the video frames after the current video frame is the first video frame. When the current video frame is the first video frame, the specific steps for tracking medical devices in the current video frame include: inputting the current video frame into the medical device annotation model to obtain the medical device annotation image corresponding to the current video frame; determining the medical device regions in the medical device annotation image corresponding to the current video frame as the medical device regions in the current video frame.
[0088] Among them, the first video frame refers to the first video frame obtained after adding medical devices in the medical device tracking scenario.
[0089] Further, the medical device annotation model is trained through the following steps:
[0090] Step 1: Obtain medical device sample images and corresponding actual medical device annotation images.
[0091] Step 2: Input the medical device sample images into the medical device annotation model to annotate the medical device regions in the medical device sample images, and obtain predicted medical device annotation images.
[0092] Refer to Figure 2 As shown, it is a flowchart of another medical device tracking method provided by the embodiment of the present application. The following explains each step of the embodiment of the present application exemplarily:
[0093] S201. Preprocess the medical device sample images to obtain the first medical device sample feature map.
[0094] In the embodiment of the present application, the medical device sample images are adjusted to a fixed resolution through bilinear interpolation for data alignment and unified data format; local features in the data-aligned medical device sample images are extracted by standard convolution (a 3*3 convolution block) according to a preset step size (such as 1 pixel); the local features in the medical device sample images are activated by using the activation function ReLU; and batch normalization is performed on the activated local features to obtain the first medical device sample feature map.
[0095] Among them, the data-aligned medical device sample image I is extracted by standard convolution Conv2D (a 3*3 convolution kernel) alignedThe local feature F in conv has the following formula:
[0096] F conv = Conv2D(I aligned , W conv , b conv );
[0097] Among them, W conv is the weight of the convolutional kernel, and b conv is the bias of the convolutional kernel.
[0098] The activated local feature is batch-normalized to obtain the first medical device sample feature map F norm has the following formula:
[0099]
[0100] Among them, μ is the average value of all feature values in the activated local feature, and σ 2 is the variance of all feature values in the activated local feature, and γ and β are learnable parameters.
[0101] S202. Use multiple convolutional blocks with the same area but different sizes to perform convolution on the medical device sample feature map after max pooling respectively to obtain second medical device sample feature maps corresponding to multiple sizes.
[0102] In the embodiment of the present application, the sizes may include 32*32 and 64*64.
[0103] S203. Calculate the target medical device sample features corresponding to each size according to the second medical device sample feature maps corresponding to each size.
[0104] i. Calculate the offset of each second medical device sample feature map of this size relative to the medical device sample feature map after max pooling.
[0105] In the embodiment of the present application, the offset Δp of each second medical device sample feature map of this size relative to the medical device sample feature map F pool after max pooling is calculated through the following formula:
[0106] Δp = Conv2D offset (F pool );
[0107] Among them, offset is the position of the second medical device sample feature map in the medical device sample feature map F pool after max pooling.
[0108] ii. Perform bilinear interpolation calculation based on the offset Δp of the medical device sample feature map after max pooling and the offsets of all second medical device sample feature maps q of this size relative to the medical device sample feature map after max pooling to obtain the initial medical device sample feature corresponding to this size.
[0109] In the embodiment of the present application, the initial medical device sample feature corresponding to this size is calculated by the following formula:
[0110] F sample = ∑ q G(q, Δp) · F pool (q);
[0111] where G(q, Δp) is the interpolation weight, and F pool (q) is the feature at the corresponding position of the second medical device sample feature map q in the medical device sample feature map after max pooling.
[0112] iii. Perform standard convolution operation and upsampling operation respectively on the initial medical device sample feature corresponding to this size to obtain the target medical device sample feature corresponding to this size.
[0113] S204. Fuse the target medical device sample features corresponding to all sizes and the first medical device sample feature map after standard convolution to obtain the predicted medical device annotation image.
[0114] In the embodiment of the present application, add the target medical device sample features corresponding to all sizes and the first medical device sample feature map after standard convolution to obtain the predicted medical device annotation image.
[0115] Step three. Train the medical device annotation model according to the medical device sample image and the corresponding actual medical device annotation image and predicted medical device annotation image.
[0116] In addition, the content annotated by the medical device annotation model for the medical device sample image includes the region box and identification of the medical device. During the tracking process, each medical device in the medical device tracking scenario is tracked separately.
[0117] Optionally, the embodiment of the present application further provides a self-supervised optimization mechanism for a medical device annotation model: before each tracking of a medical device, receive the medical device image uploaded by the user for this tracking and the corresponding medical device type; input the medical device image for this tracking into the medical device annotation model to obtain a medical device annotation image corresponding to the tracked medical device image; match the medical device annotation image corresponding to the tracked medical device image with each historical medical device image tracked by the medical device annotation model; if the medical device annotation image corresponding to the tracked medical device image fails to match all historical medical device images, update the medical device annotation model based on the medical device image for this tracking and the corresponding medical device type, so that the medical device annotation model can be automatically updated to identify new medical devices.
[0118] The embodiment of the present application provides a method for tracking a medical device, and the method includes: dividing a current video frame in a medical device tracking scenario according to a preset division size to obtain a plurality of first image blocks; determining, as second image blocks, the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position, where the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is latest tracked; inputting each second image block into a medical device annotation model to annotate the medical device area in each second image block to obtain a medical device annotation image corresponding to each second image block; the medical device annotation model is trained by using medical device sample images and corresponding actual medical device annotation images; determining the medical device area in the current video frame according to the medical device annotation images corresponding to all second image blocks and all first image blocks. The present application can improve the tracking effect of the medical device by tracking each image block in the current video frame separately.
[0119] Based on the same inventive concept, the embodiment of the present application further provides a tracking device for a medical device corresponding to the method for tracking a medical device. Since the principle of solving problems by the device in the embodiment of the present application is similar to that of the method for tracking a medical device in the above embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.
[0120] Refer to Figure 3 As shown, it is a schematic diagram of a tracking device for a medical device provided by the embodiment of the present application, and the tracking device for the medical device includes:
[0121] A division module 301, configured to divide a current video frame in a medical device tracking scenario according to a preset division size to obtain a plurality of first image blocks;
[0122] Determination module 302, configured to determine, as a second image block, a first image block at the latest tracking position in the current video frame and a first image block adjacent to the first image block at the latest tracking position; wherein, the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is tracked latest.
[0123] Labeling module 303, configured to input each second image block into a medical device labeling model to label the medical device area in each second image block, so as to obtain a medical device labeling image corresponding to each second image block; the medical device labeling model is trained by using medical device sample images and corresponding actual medical device labeling images.
[0124] The determination module 302 is further configured to determine the medical device area in the current video frame according to the medical device labeling images corresponding to all second image blocks and all first image blocks.
[0125] An embodiment of the present application provides a tracking device for a medical device. The device includes: a division module 301, configured to divide a current video frame in a medical device tracking scenario according to a preset division size to obtain a plurality of first image blocks; a determination module 302, configured to determine, as a second image block, a first image block at the latest tracking position in the current video frame and a first image block adjacent to the first image block at the latest tracking position; wherein, the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is tracked latest; a labeling module 303, configured to input each second image block into a medical device labeling model to label the medical device area in each second image block, so as to obtain a medical device labeling image corresponding to each second image block; the medical device labeling model is trained by using medical device sample images and corresponding actual medical device labeling images; the determination module 302 is further configured to determine the medical device area in the current video frame according to the medical device labeling images corresponding to all second image blocks and all first image blocks. The present application can improve the tracking effect of the medical device by tracking each image block in the current video frame separately.
[0126] As Figure 4 shown, an electronic device 400 provided by an embodiment of the present application includes: a processor 401, a memory 402, and a bus. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs, the processor 401 communicates with the memory 402 through the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the medical device tracking method as described above.
[0127] Specifically, the above-mentioned memory 402 and processor 401 can be general-purpose memory and processor, which are not specifically defined here. When the processor 401 runs the computer program stored in the memory 402, it can execute the above-mentioned tracking method for medical devices.
[0128] Corresponding to the above-mentioned tracking method for medical devices, an embodiment of the present application also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it executes the steps of the above-mentioned tracking method for medical devices.
[0129] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the method embodiments, which will not be elaborated in this application. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or modules can be in electrical, mechanical, or other forms.
[0130] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0131] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.
[0132] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the information processing method described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.
[0133] The above are only specific implementation manners of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.
Claims
1. A tracking method for a medical device, characterized in that, The method includes: Dividing a current video frame in a medical device tracking scenario according to a preset division size to obtain a plurality of first image blocks; Determining, as second image blocks, the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position; wherein, the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is tracked latest; Inputting each second image block into a medical device annotation model to annotate the medical device area in each second image block, obtaining a medical device annotation image corresponding to each second image block; the medical device annotation model is trained by using medical device sample images and corresponding actual medical device annotation images; Determining the medical device area in the current video frame according to the medical device annotation images corresponding to all second image blocks and all first image blocks.
2. The tracking method of the medical device according to claim 1, characterized in that, The determining, as second image blocks, the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position includes: Matching each first image block with the medical device area image in the video frame where the medical device area is tracked latest, obtaining a matching result corresponding to each first image block; If the matching results corresponding to all first image blocks are all matching failures, then determining, as second image blocks, the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position.
3. The tracking method of the medical device according to claim 2, characterized in that The method further includes: If there are first image blocks with matching results being successful, then determining, as third image blocks, the first image blocks with successful matching and the first image blocks adjacent to the first image blocks with successful matching in the current video frame; Inputting each third image block into a medical device annotation model to annotate the medical device area in each third image block, obtaining a medical device annotation image corresponding to each third image block; Determining the medical device area in the current video frame according to the medical device annotation images corresponding to all third image blocks and all first image blocks.
4. The tracking method of the medical device according to claim 1 or 3, characterized in that, The determining the medical device area in the current video frame includes: Determining all medical device annotation images as the latest candidate medical device annotation images; Judging whether there is a latest candidate medical device annotation image annotated with a medical device area; If there is, then determining the latest medical device annotation image annotated with a medical device area as the latest target medical device annotation image; Inputting the first image blocks adjacent to the latest target medical device annotation image in the current video frame into a medical device annotation model to annotate the medical device area, obtaining a latest candidate medical device annotation image; and jumping to the step of judging whether there is a latest candidate medical device annotation image annotated with a medical device area to continue execution; If there is not, then determining all the medical device areas annotated by the medical device annotation model as the medical device area in the current video frame.
5. The tracking method of the medical device according to claim 1 or 3, characterized in that Training the medical device annotation model through the following steps; Obtain a medical device sample image and the corresponding actual medical device annotation image; Input the medical device sample image into the medical device annotation model to annotate the medical device area in the medical device sample image, and obtain a predicted medical device annotation image; Train the medical device annotation model according to the medical device sample image, the corresponding actual medical device annotation image, and the predicted medical device annotation image.
6. The tracking method of the medical device according to claim 5, characterized in that, The step of inputting the medical device sample image into the medical device annotation model to annotate the medical device area in the medical device sample image and obtain a predicted medical device annotation image includes: Preprocess the medical device sample image to obtain a first medical device sample feature map; Use multiple convolution blocks with the same area but different sizes to perform convolution on the max-pooled medical device sample feature map respectively, and obtain multiple second medical device sample feature maps corresponding to different sizes; Calculate the target medical device sample features corresponding to each size according to the second medical device sample feature maps corresponding to each size; Fuse the target medical device sample features corresponding to all sizes and the first medical device sample feature map after standard convolution to obtain a predicted medical device annotation image.
7. The tracking method of the medical device according to claim 6, wherein The step of calculating the target medical device sample features corresponding to each size according to the second medical device sample feature maps corresponding to each size includes: Calculate the offset of each second medical device sample feature map of the size relative to the max-pooled medical device sample feature map; Perform bilinear interpolation calculation according to the offset of the max-pooled medical device sample feature map and the offsets of all second medical device sample feature maps of the size relative to the max-pooled medical device sample feature map, and obtain the initial medical device sample features corresponding to the size; Perform standard convolution operations and upsampling operations on the initial medical device sample features corresponding to the size respectively to obtain the target medical device sample features corresponding to the size.
8. A tracking device for a medical device, characterized in that, The device includes: A division module for dividing the current video frame in the medical device tracking scenario according to a preset division size to obtain a plurality of first image blocks; A determination module for determining the first image block at the latest tracking position in the current video frame and the first image blocks adjacent to the first image block at the latest tracking position as second image blocks; wherein, the latest tracking position refers to the position of the medical device area in the video frame where the medical device area is tracked latest; A labeling module for inputting each second image block into the medical device annotation model to annotate the medical device area in each second image block, and obtaining a medical device annotation image corresponding to each second image block; the medical device annotation model is trained through a medical device sample image and the corresponding actual medical device annotation image; The determination module is further configured to determine the medical device area in the current video frame according to the medical device annotation images corresponding to all second image blocks and all first image blocks.
9. An electronic device, characterized in that, It includes: A processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the tracking method of the medical device according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, it performs the steps of the tracking method of the medical device according to any one of claims 1 to 7.