Target object tracking method and device, computer device, readable storage medium and program product

Abstract

The application relates to a target object tracking method and device, equipment, readable storage medium and computer program product, relates to the technical field of medical image processing, and can balance the real-time performance and accuracy of medical image processing. The method comprises the following steps: acquiring slice layers constituting a three-dimensional medical image frame; a plurality of slice layers are stacked into a three-dimensional medical image frame containing a target object to be tracked according to an in-frame stacking order in a slice layer stacking direction; the following processing is performed in parallel in the process of acquiring each slice layer: determining slice layer image features of a current slice layer currently acquired; determining adjacent slice layers adjacent to the current slice layer in the in-frame stacking order from the acquired slice layers; complementing the slice layer image features of the current slice layer according to the slice layer image features of the adjacent slice layers in the slice layer stacking direction to obtain spatial image features; and obtaining a slice layer processing result of the current slice layer according to the spatial image features; and obtaining a target object tracking result of the three-dimensional medical image frame according to the slice layer processing result.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a target object tracking method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] Real-time scanning, as a continuous imaging technique, can acquire high-resolution three-dimensional medical images during dynamic processes and is often used in navigation surgery and guided interventional treatment procedures. In continuous imaging technology, each imaging session yields three-dimensional image data composed of multiple layers. Each frame includes multiple two-dimensional (2D) layers, and when all the multiple 2D layers in a complete frame are received, a complete three-dimensional image frame is obtained.

[0003] In related technologies, target object tracking based on continuous imaging mainly begins image processing after receiving a complete 3D image frame. For example, surgical instrument segmentation begins identifying the position of surgical instruments in 3D space after receiving a complete 3D image frame. However, the above processing method results in a significant delay in target object tracking, making it difficult to meet the real-time requirements for target object tracking during operation. Summary of the Invention

[0004] Therefore, it is necessary to provide a target object tracking method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the real-time performance of tracking in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a target object tracking method, including:

[0006] Obtain slices for composing a three-dimensional medical image frame; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked;

[0007] During the acquisition of each of the aforementioned layers, layer processing steps are performed in parallel on the currently acquired layer to obtain the layer processing result of the current layer. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the intra-frame stacking order adjacent to the current layer from the previously acquired layers; and completing the layer image features of the current layer in the layer stacking direction according to the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer according to the spatial image features.

[0008] Based on the slice processing results of multiple slices belonging to the same 3D medical image frame, the target object tracking result of the 3D medical image frame is obtained.

[0009] In one embodiment, determining the neighboring layers whose intra-frame stacking order is adjacent to the current layer from previously acquired layers includes:

[0010] If the current layer is the first in the intra-frame stacking order, then the next layer in the intra-frame stacking order after the current layer is determined from the previously acquired layers and is taken as the neighboring layer of the current layer.

[0011] If the intra-frame stacking order of the current slice is between the first and last slices, then the preceding slice with an intra-frame stacking order before the current slice and / or the following slice with an intra-frame stacking order after the current slice are determined from the previously acquired slices and are taken as the neighboring slices of the current slice.

[0012] If the current tile is the last in the intra-frame stacking order, then the previous tile whose intra-frame stacking order precedes the current tile is determined from the previously acquired tiles and is taken as the neighboring tile of the current tile.

[0013] In one embodiment, the front sheet layer is determined by the following steps:

[0014] Identify the preceding three-dimensional medical image frames that have been acquired; the preceding three-dimensional medical image frames are the three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current slice belongs;

[0015] From the multiple layers corresponding to the previous three-dimensional medical image frame, at least one layer whose intra-frame stacking order precedes the current layer is determined as the previous layer preceding the current layer.

[0016] And / or,

[0017] If a frame is found to be in the preceding layer, the preceding layer is defined as the layer preceding the current layer; the preceding layer is a layer that belongs to the same three-dimensional medical image frame as the current layer and whose intra-frame stacking order precedes the current layer.

[0018] In one embodiment, the back layer is determined by the following steps:

[0019] Identify the preceding three-dimensional medical image frames that have been acquired; the preceding three-dimensional medical image frames are the three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current slice belongs;

[0020] From the multiple layers corresponding to the preceding three-dimensional medical image frame, at least one layer whose intra-frame stacking order is later than the current layer is determined as the subsequent layer after the current layer.

[0021] In one embodiment, the adjacent layers include the preceding layers and / or the following layers determined from the layers corresponding to the preceding three-dimensional medical image frames, wherein the preceding three-dimensional medical image frames are three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current layer belongs.

[0022] The step of completing the layer image features of the current layer according to the layer image features of the neighboring layers in the layer stacking direction to obtain spatial image features containing the neighboring spatial image information of the current layer includes:

[0023] Based on the layer image features in the preceding layer, predict the layer image features of the image content in the preceding layer at the current layer acquisition time, and / or, based on the layer image features in the following layer, predict the layer image features of the image content in the following layer at the current layer acquisition time.

[0024] In the stacking direction of the layers, based on the layer image features of the image content in the preceding layer and / or the following layer at the time of acquisition of the current layer, the layer image features of the current layer are completed to obtain spatial image features containing the neighboring spatial image information of the current layer.

[0025] In one embodiment, each of the layered image features is a three-dimensional feature;

[0026] The step of completing the layer image features of the current layer based on the layer image features of the image content in the preceding layer and / or the following layer at the time of acquisition of the current layer, to obtain spatial image features containing the neighboring spatial image information of the current layer, includes:

[0027] Along the stacking direction of the layers, at least two of the following are stacked sequentially: the layer image features of the image content in the previous layer at the current layer acquisition time, the layer image features of the current layer, and the layer image features of the image content in the subsequent layer at the current layer acquisition time, to obtain the feature stitching result.

[0028] Based on the feature stitching result, spatial image features containing neighboring spatial image information of the current slice are obtained.

[0029] In one embodiment, the layer processing result includes a local segmentation result obtained by segmenting the target object of the current layer based on the spatial image features of the current layer;

[0030] The step of obtaining the target object tracking result of the three-dimensional medical image frame based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame includes:

[0031] The local segmentation results of multiple layers belonging to the same three-dimensional medical image frame are stitched together, and the target object tracking result of the three-dimensional medical image frame is obtained based on the stitched local segmentation results.

[0032] In one embodiment, the layer processing result includes the spatial image features of the current layer;

[0033] The step of obtaining the target object tracking result of the three-dimensional medical image frame based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame includes:

[0034] The spatial image features of multiple layers belonging to the same three-dimensional medical image frame are fused to obtain fused spatial image features.

[0035] Based on the fused spatial image features, the target object tracking results of the three-dimensional medical image frame are obtained.

[0036] In one embodiment, the method further includes:

[0037] During the process of acquiring the current slice, the target preprocessing task corresponding to the current slice is executed in parallel to obtain the preprocessing task result; the target preprocessing task corresponding to the current slice is a preprocessing task that is related to the current slice being acquired and whose task execution process does not depend on the current slice being acquired.

[0038] The process of fusing the spatial image features of multiple layers belonging to the same three-dimensional medical image frame to obtain fused spatial image features includes:

[0039] Based on the spatial image features of multiple layers belonging to the same three-dimensional medical image frame and the obtained preprocessing task results, feature fusion is performed to obtain fused spatial image features.

[0040] Secondly, this application also provides a target object tracking device, comprising:

[0041] Obtain slices for composing a three-dimensional medical image frame; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked;

[0042] During the acquisition of each of the aforementioned layers, layer processing steps are performed in parallel on the currently acquired layer to obtain the layer processing result of the current layer. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the intra-frame stacking order adjacent to the current layer from the previously acquired layers; and completing the layer image features of the current layer in the layer stacking direction according to the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer according to the spatial image features.

[0043] Based on the slice processing results of multiple slices belonging to the same 3D medical image frame, the target object tracking result of the 3D medical image frame is obtained.

[0044] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0045] Obtain slices for composing a three-dimensional medical image frame; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked;

[0046] During the acquisition of each of the aforementioned layers, layer processing steps are performed in parallel on the currently acquired layer to obtain the layer processing result of the current layer. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the intra-frame stacking order adjacent to the current layer from the previously acquired layers; and completing the layer image features of the current layer in the layer stacking direction according to the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer according to the spatial image features.

[0047] Based on the slice processing results of multiple slices belonging to the same 3D medical image frame, the target object tracking result of the 3D medical image frame is obtained.

[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0049] Obtain slices for composing a three-dimensional medical image frame; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked;

[0050] During the acquisition of each of the aforementioned layers, layer processing steps are performed in parallel on the currently acquired layer to obtain the layer processing result of the current layer. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the intra-frame stacking order adjacent to the current layer from the previously acquired layers; and completing the layer image features of the current layer in the layer stacking direction according to the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer according to the spatial image features.

[0051] Based on the slice processing results of multiple slices belonging to the same 3D medical image frame, the target object tracking result of the 3D medical image frame is obtained.

[0052] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0053] Obtain slices for composing a three-dimensional medical image frame; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked;

[0054] During the acquisition of each of the aforementioned layers, layer processing steps are performed in parallel on the currently acquired layer to obtain the layer processing result of the current layer. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the intra-frame stacking order adjacent to the current layer from the previously acquired layers; and completing the layer image features of the current layer in the layer stacking direction according to the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer according to the spatial image features.

[0055] Based on the slice processing results of multiple slices belonging to the same 3D medical image frame, the target object tracking result of the 3D medical image frame is obtained.

[0056] The aforementioned target object tracking method, apparatus, computer device, computer-readable storage medium, and computer program product can acquire layers used to compose a three-dimensional medical image frame. Multiple layers can be stacked in a stacking direction according to an intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked. During the acquisition of each layer, layer processing steps are performed in parallel on the currently acquired layer to obtain the layer processing result. The layer processing steps include determining the layer image features of the currently acquired layer, determining neighboring layers with an intra-frame stacking order adjacent to the current layer from previously acquired layers, and completing the layer image features of the current layer in the stacking direction based on the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer. The layer processing result of the current layer is obtained based on the spatial image features. Furthermore, the target object tracking result of the three-dimensional medical image frame is obtained based on the respective layer processing results of multiple layers belonging to the same three-dimensional medical image frame. In this embodiment, on the one hand, by performing two-dimensional image processing on the currently acquired slice in parallel during the acquisition of each slice, the process of independently processing based on two-dimensional slices can be distributed in advance in a reasonable and orderly manner throughout the entire process. This makes full use of the original idle computing resources during the transmission of each slice image and avoids the computing demand being too concentrated after the reception of the complete three-dimensional medical image frame, effectively improving the real-time performance of image processing. On the other hand, by using the slice image features of neighboring slices to complete the slice image features of the current slice, spatial image features containing the neighboring spatial image information of the current slice are obtained. This can complete the neighboring spatial information of incomplete slices. When performing image processing based solely on the current slice, it can effectively improve the accuracy and reliability of the slice processing results. Thus, when tracking target objects, it can effectively balance the real-time performance and accuracy of image processing. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating a target object tracking method in one embodiment;

[0059] Figure 2 This is a diagram illustrating the application environment of a target object tracking method in one embodiment.

[0060] Figure 3a This is a schematic diagram of a single sheet in one embodiment;

[0061] Figure 3b This is a schematic diagram of a three-dimensional medical image frame composed of multiple layers in one embodiment;

[0062] Figure 4 This is a schematic diagram of a three-dimensional image transmission and processing in one embodiment;

[0063] Figure 5a This is a schematic diagram of another three-dimensional image transmission and processing method in one embodiment;

[0064] Figure 5b This is a schematic diagram of another three-dimensional image transmission and processing method in one embodiment;

[0065] Figure 6a This is a schematic diagram of the structure of a feature extraction network in one embodiment;

[0066] Figure 6b This is a schematic diagram of the feature extraction process of a spliced ​​convolutional layer in one embodiment;

[0067] Figure 7 This is a schematic diagram of a three-dimensional image processing flow in one embodiment;

[0068] Figure 8 This is a schematic diagram of another three-dimensional image processing procedure in one embodiment;

[0069] Figure 9 This is a structural block diagram of a target object tracking device in one embodiment;

[0070] Figure 10 This is an internal structural diagram of a computer device in one embodiment;

[0071] Figure 11 This is an internal structural diagram of another computer device in one embodiment. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0073] To enable those skilled in the art to better understand this application, the relevant technologies are first introduced below.

[0074] Real-time scanning, as a continuous imaging technology, can acquire high-resolution three-dimensional medical images in a dynamic process. Taking computed tomography (CT) images as an example, CT-guided percutaneous puncture surgery is a common clinical method. Traditional CT-guided percutaneous puncture surgery involves multiple pauses during the puncture process, during which CT scans are used to determine the relative position of the lesion and the puncture needle. By having a doctor control a robot to perform the puncture, the efficiency and accuracy of the puncture are greatly improved, and the radiation exposure to the patient is reduced. To address this, real-time CT fluoroscopy can be introduced into the above operation. Real-time CT fluoroscopy is a continuous X-ray imaging technology that can acquire high-resolution medical images in a dynamic process and is often used in navigation surgery and guided interventional treatments. Real-time CT fluoroscopy is a continuous imaging technology; each image captures three-dimensional image data composed of multiple layers. Each frame includes multiple two-dimensional layers, and when all the two-dimensional layers in a complete frame are received, a complete three-dimensional image frame is obtained.

[0075] In related technologies, target object tracking based on real-time perspective scanning mainly involves performing image processing on a complete 3D image frame after it is received. For example, surgical instrument segmentation begins by identifying the position of the surgical instrument in 3D space after a complete 3D image frame is received. However, the above processing method results in a significant delay in the target object tracking results, making it difficult to meet the real-time requirements for target object tracking during operation.

[0076] Based on this, this application provides a target object tracking method, apparatus, computer device, computer-readable storage medium, and computer program product to at least solve the above-mentioned technical problems.

[0077] In one embodiment, such as Figure 1 As shown, a target object tracking method is provided. This embodiment illustrates the application of this method to a terminal. In some examples, the terminal can be a medical imaging device, such as a CT scanner or a robot image processing host. It is understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be implemented through the interaction between the terminal and the server.

[0078] In some embodiments, such as Figure 2The diagram shows the overall functional flowchart of the real-time CT mode. After scanning, the CT scanner reconstructs the image and transmits the image data to the interventional robot's image processing host via an Ethernet switch. In real-time mode, the CT scanner can perform a single scan of the target object and reconstruct multiple two-dimensional slices. These slices can form a three-dimensional medical image frame. When the CT scanner continuously scans and reconstructs the target object, multiple three-dimensional medical image frames can be obtained. The multiple two-dimensional slices reconstructed by the CT scanner can be transmitted in parallel to the interventional robot's image processing host at intervals. After receiving the image data of the two-dimensional slices, the robot's image processing host stores the image data in memory. Subsequently, the robot's image processing host copies the image data from the CPU's memory to the GPU's video memory. The entire real-time image processing flow can then be implemented by the GPU, including image data preprocessing, image processing algorithms, post-processing of image processing algorithms, and finally, result display.

[0079] Image data preprocessing can include image normalization, and post-processing of image processing algorithms can include adding connected component analysis after image segmentation to remove noise.

[0080] Image processing algorithms can include one or more of the following: image segmentation, point tracking, target object detection, and target object prediction. These image processing functions constitute the real-time mode image processing function module. Taking the target object as an example, which includes surgical instruments and tissues and organs, the CT real-time mode surgical instrument segmentation and 3D reconstruction can prompt the user on the 3D view to indicate the relative position of the current surgical instruments and tissues and organs, making the surgical operation safer. For example, when the target object includes tumor targets, the CT real-time mode target prediction function can predict the position of the tumor targets as the body moves with breathing, which can guide the real-time mode surgical process.

[0081] In this embodiment, the method includes the following steps:

[0082] S101, Obtain the slices used to compose the three-dimensional medical image frame; Multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked.

[0083] The target object can be any object being tracked. In some exemplary embodiments, the target object may include one or more of the following: organ tissue, target area, surgical instrument, or a specific target point on the surgical instrument (such as the tip of a puncture needle). Tracking the target object may include one or more of the following processes: segmenting, detecting, and identifying the target object in the image, and predicting or tracking the movement trajectory of the target object.

[0084] Three-dimensional medical image frames can be one or more of the following medical images: computed tomography images, X-ray images, magnetic resonance imaging (MRI), positron emission tomography (PET), and single-photon emission computed tomography (SPECT).

[0085] In practical applications, medical imaging equipment can scan and reconstruct the target object multiple times to obtain multiple slices and acquire a slice stream composed of multiple slices generated in real time. In some exemplary embodiments, each slice can be sent to the terminal sequentially according to the spatial order or reconstruction order of the slices.

[0086] Multiple layers can be stacked in the stacking direction according to the intra-frame stacking order to form a 3D medical image frame composed of these multiple layers. For example, if each 3D medical image frame is composed of 4 consecutive stacked layers, the kth to k+3rd layers in the layer stream can form 3D medical image frame n, and the k+4th to k+7th layers can form 3D medical image frame n+1. In some examples, the total number of layers to be reconstructed in this scan can be predetermined before scanning. If a complete 3D medical image frame obtained in this scan contains 4 layers, then when receiving each layer, the relevant layer information "n,k,imagedata" can be carried, which indicates that the currently received imagedata is the kth layer of the nth frame. When all 4 layers of the nth frame are received, it is determined that a complete 3D frame has been received. It can be understood that since the actual target object is a physical object occupying a certain space in 3D space, by scanning and reconstructing the region containing the target object, the 3D medical image frame composed of multiple stacked layers obtained from the reconstruction also contains the target object. For example, Figure 3a The diagram illustrates layers containing partial image information of the target object. By stacking multiple layers containing partial image information of the target object in an intra-frame stacking order along the head-corner direction, a three-dimensional medical image frame containing a complete (or most) view of the target object from a certain perspective can be obtained, such as... Figure 3b As shown.

[0087] For the receiving / transmission process of the slice layer, Figure 4 A schematic diagram of lamellar flow transport during real-time fluoroscopic scanning is also shown, such as... Figure 4As shown, each three-dimensional medical image frame contains multiple slices. Multiple slices in the same three-dimensional medical image frame can be transmitted in parallel. The start time of transmission for each slice can have a certain time interval, for example, slices can be transmitted to the terminal at intervals of 10 to 30 milliseconds. In some exemplary embodiments, the real-time performance of CT real-time fluoroscopy can be improved by limiting the scanning range. Specifically, the field of view (FOV) of each scan is small (e.g., less than the field of view threshold). Taking the head-to-toe direction as an example, only a scanning range of 10mm, 20mm, or 40mm may be reconstructed during scanning. At the same time, in order to reduce the image transmission delay, the number of CT slices reconstructed during scanning is small (e.g., less than the slice number threshold). For example, the number of reconstructed slices may be 4, 8, or 16 slices. The number of reconstructed slices can be positively correlated with the image data transmission delay, that is, the more slices reconstructed, the greater the transmission delay of the complete image sequence composed of multiple three-dimensional medical image frames.

[0088] S102, during the acquisition of each slice, the slice processing steps are performed in parallel on the currently acquired slice to obtain the slice processing result of the current slice. The slice processing steps include: determining the slice image features of the currently acquired slice, determining the neighboring slices with the intra-frame stacking order adjacent to the current slice from the previously acquired slices, and completing the slice image features of the current slice in the slice stacking direction according to the slice image features of the neighboring slices to obtain spatial image features containing the neighboring spatial image information of the current slice, and obtaining the slice processing result of the current slice based on the spatial image features.

[0089] Real-time intraoperative navigation image processing algorithms may involve one or more processing steps such as segmentation, detection, and tracking, including surgical instrument segmentation, needle tip tracking and prediction, lesion location prediction, or puncture target prediction. In related technologies, image processing algorithms require a complete 3D image frame to begin processing; for example, surgical instrument segmentation only begins after receiving a complete 3D image frame (e.g.,...). Figure 3b In traditional medical imaging, surgical instruments are typically segmented into a 3D medical image frame containing the complete outline of the surgical instrument by stacking multiple layers in the head-corner direction. Only after this frame is a complete 3D medical image frame can the position of the surgical instrument in 3D space be identified. If each layer of the complete 3D image frame is processed separately, the image processing algorithm faces significant challenges in handling incomplete data because each 2D layer is not a complete dataset, resulting in lower accuracy. Taking surgical instrument segmentation in real-time mode as an example, if a complete 3D image frame in a real clinical scene includes four layers, then the surgical instrument is located on each of the four layers, with each layer containing a portion of the surgical instrument. Figure 3b The surgical instruments shown are within a complete three-dimensional image frame, while the received image is as follows: Figure 3aThe surgical instruments on the first slice shown are incomplete.

[0090] However, processing all images only after the complete 3D image frame has been received can lead to an over-concentration of computational resource requirements across multiple algorithms. For example, in real-time perspective scanning, each reconstruction yields four slices. Figure 5a As shown, after receiving the four complete slices of the nth 3D image frame during the operation, i.e., after receiving a complete 3D image frame, the 3D image processing algorithm is executed. The 3D image processing algorithm may include 3D image segmentation, detection, recognition, prediction, etc. Since multiple 3D image processing algorithms are time-consuming, the delay between receiving a complete 3D image frame and the image algorithm completing the processing and returning the result is long. At the same time, during the process of executing the image processing algorithm on the current 3D image frame, new 3D image frames are received and cannot be processed in time, resulting in a large delay in updating the image processing results. The updated data cannot be processed in time, which cannot meet the real-time requirements of intraoperative image processing in real-time mode.

[0091] In response to this, on the one hand, considering the large delay in image processing algorithms caused by multiple 3D image processing requirements, and the fact that computational resources are too concentrated on post-processing of 3D image frames while computational resources are not fully utilized during image transmission, this application will extract features from each currently acquired slice in parallel during the slice acquisition process to obtain the slice image features corresponding to the current slice. Then, in the slice stacking direction, the slice image features of the current slice will be completed based on the slice image features of neighboring slices to obtain spatial image features containing the neighboring spatial image information of the current slice.

[0092] In other words, during the process of acquiring two-dimensional slices, such as the time period when a complete three-dimensional medical image frame has not been acquired, this application will simultaneously process the currently acquired slice using image processing algorithms while acquiring other two-dimensional slices, such as... Figure 4 As shown, this application avoids waiting until all layers belonging to the same 3D medical image are acquired before processing. Compared to related technologies that wait until a complete 3D medical image frame is acquired before processing, this application can distribute the independent processing based on 2D layers in advance and in a reasonable and orderly manner throughout the entire process. This evenly distributes the computational requirements, makes full use of the idle computational resources originally available during image transmission, and avoids the computational requirements being too concentrated after the complete 3D image frame is received, which would lead to excessively long image processing time. At the same time, 2D image processing and layer transmission are carried out in parallel, further improving the real-time performance of image processing and solving the problem of difficulty in meeting real-time requirements in image processing. Figure 5bAs shown, the following example illustrates the process of obtaining four slices each time through real-time fluoroscopic scanning. In this embodiment, after obtaining the current slice, the current slice can be preprocessed and 2D image processing can be performed on it in a timely manner. Furthermore, other slices can be received while the current slice is being processed. Figure 5b In the example shown, the slices of each 3D medical image frame are sent sequentially, that is, after all slices of the current 3D medical image frame are sent, the slices of the next 3D medical image frame are sent. It is understood that in some other embodiments, multiple slices may not be sent in the order of the 3D image frames. For example, they can be sent according to the reconstruction order of each slice, with the reconstructed slices sent first. After the terminal obtains the slice, it can determine the 3D medical image frame to which the slice belongs based on the slice information associated with the slice. Subsequently, the spatial image features or related information corresponding to each of the multiple slices belonging to the same 3D medical image frame can be integrated based on this information.

[0093] On the other hand, for processing the acquired current slice using two-dimensional image processing algorithms, some implementations can utilize related traditional algorithms, such as U-net networks and Spatial-Time Memory (STM) networks for image segmentation. U-net is widely used as the primary tool for segmentation tasks in medical imaging, including CT scans, Magnetic Resonance Imaging (MRI), X-rays, and microscopy. STM networks are designed for video segmentation, primarily performing segmentation of two-dimensional image data. They can fuse information from all previous frames to aid in segmentation of the current frame.

[0094] However, U-net networks are primarily designed for non-time-series image data. In real-time CT scenarios, the resulting 3D medical image frames are time-series 3D frames. Effectively utilizing the information from each frame in the time series significantly improves the performance of image processing algorithms. If only U-net is used, only individual frames can be processed, failing to effectively utilize the temporal relationships between frames in the sequence. While STM networks can fuse information from all preceding frames to aid in segmentation of the current frame, each object processed in an STM network is a complete 2D video frame. In real-time CT, the complete data is a single 3D medical image frame. Each independent slice within a 3D medical image frame is incomplete 2D data. Applying an STM network would require the entire computation process to be concentrated after the entire 3D medical image frame is collected, i.e., located within... Figure 4In the 3D image processing algorithm, there will also be problems that lead to excessive concentration of computing resources and long processing delays. If each two-dimensional layer in each frame is processed separately, the accuracy and performance of the image processing algorithm will be reduced because the two-dimensional layer is incomplete data.

[0095] In this embodiment of the present application, multiple previously acquired slices can be identified first. The previously acquired slices can refer to multiple slices acquired in the same operation process, such as slices acquired in the same surgical operation or examination process.

[0096] Then, after receiving the current slice, other slices with intra-frame stacking order adjacent to the current slice can be directly determined from the multiple slices that have been acquired. These other slices with intra-frame stacking order adjacent to the current slice are also called neighboring slices. Intra-frame stacking order adjacent to the current slice can be understood as the difference between the intra-frame stacking order of a certain slice and the intra-frame stacking order of the current slice being less than a preset threshold. For example, if the intra-frame stacking order of the current slice is k and the intra-frame stacking order of another slice is m, when the difference between |km| is less than the preset threshold, the slice can be determined to be a neighboring slice of the current slice.

[0097] In some embodiments, neighboring layers may include at least one of preceding and following layers. A preceding layer refers to a layer whose intra-frame stacking order is adjacent to and precedes the current layer, while a following layer refers to a layer whose intra-frame stacking order is adjacent to and follows the current layer. By determining neighboring layers from the acquired layers, the spatial image information of the current layer can be supplemented using other acquired complete 2D layers, avoiding waiting and effectively improving the real-time performance of image processing.

[0098] Since neighboring slices are located above or below the current slice in spatial position, the slice image features of the current slice can be completed by utilizing the slice image features of neighboring slices. Based on the feature completion result, spatial image features containing neighboring spatial image information of the current slice are obtained, thus supplementing the spatial information of the vertical position of the current slice in the current 3D medical image frame. This allows the construction of a complete 3D spatial feature representation with vertical position information, providing the necessary vertical spatial information when processing incomplete slice data. In some embodiments, if the current slice is the first slice in the first 3D medical image frame, neighboring slices may not be obtained. In such cases, only the slice image features of the first slice can be extracted without completing its neighboring spatial image information. Alternatively, after extracting the slice image features of the first slice, the slice image features of subsequent neighboring slices can be obtained before completion, resulting in spatial image features of the first slice containing neighboring spatial image information.

[0099] Furthermore, the layer processing result of the current layer can be obtained based on the spatial image features of the current layer. In some exemplary embodiments, the spatial image features of the current layer can be used as the layer processing result, or the current layer can be further processed based on the obtained spatial image features, and the further processing result can be used as the layer processing result.

[0100] S103, based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame, obtain the target object tracking result of the three-dimensional medical image frame.

[0101] Specifically, after obtaining the slice processing result of the current slice, the newly obtained slice can be used as the current slice, and then the execution of step S102 is returned until the slice processing results of multiple slices belonging to the same three-dimensional medical image frame are obtained. The target object tracking result of the three-dimensional medical image frame can be obtained based on the slice processing results of multiple slices.

[0102] In the aforementioned target object tracking method, layers used to compose a three-dimensional medical image frame can be acquired. Multiple layers can be stacked in the layer stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked. During the acquisition of each layer, layer processing steps are performed in parallel on the currently acquired layer to obtain the layer processing result of the current layer. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the intra-frame stacking order adjacent to the current layer from the previously acquired layers; and completing the layer image features of the current layer in the layer stacking direction according to the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer based on the spatial image features. Then, based on the layer processing results of multiple layers belonging to the same three-dimensional medical image frame, the target object tracking result of the three-dimensional medical image frame is obtained. In this embodiment, on the one hand, by performing two-dimensional image processing on the currently acquired slice in parallel during the acquisition of each slice, the process of independently processing based on two-dimensional slices can be distributed in advance in a reasonable and orderly manner throughout the entire process. This makes full use of the original idle computing resources during the transmission of each slice image and avoids the computing demand being too concentrated after the reception of the complete three-dimensional medical image frame, effectively improving the real-time performance of image processing. On the other hand, by using the slice image features of neighboring slices to complete the slice image features of the current slice, spatial image features containing the neighboring spatial image information of the current slice are obtained. This can complete the neighboring spatial information of incomplete slices. When performing image processing based solely on the current slice, it can effectively improve the accuracy and reliability of the slice processing results. Thus, when tracking target objects, it can effectively balance the real-time performance and accuracy of image processing.

[0103] In an exemplary embodiment, step S102, determining the neighboring layers whose intra-frame stacking order is adjacent to the current layer from previously acquired layers, may include the following steps:

[0104] If the current slice's intra-frame stacking order is first, then the slice whose intra-frame stacking order follows the current slice from the previously acquired slices is determined as the neighboring slice of the current slice; if the current slice's intra-frame stacking order is between first and last, then the slice whose intra-frame stacking order precedes the current slice and / or the slice whose intra-frame stacking order follows the current slice from the previously acquired slices is determined as the neighboring slice of the current slice; if the current slice's intra-frame stacking order is last, then the slice whose intra-frame stacking order precedes the current slice from the previously acquired slices is determined as the neighboring slice of the current slice.

[0105] In practical applications, if the current layer is the first in the intra-frame stacking order, for example, if four layers are reconstructed in each scan and the current layer is the first layer, then it can be determined that there will be relevant spatial information below the current layer. At this time, the missing spatial information is also the spatial information below. Therefore, the layers that are stacked after the current layer in the intra-frame stacking order can be determined as the adjacent layers of the current layer.

[0106] If the current layer's intra-frame stacking order is between the first and last positions, that is, the current layer is in the middle position of multiple intra-frame layers, then it can be determined that the current layer has both upper and lower spatial information. Therefore, the preceding layer with an intra-frame stacking order before the current layer and / or the following layer with an intra-frame stacking order after the current layer can be determined from the multiple previously acquired layers as the neighboring layers of the current layer.

[0107] If the current slice is the last slice in the frame stacking order, that is, the current slice is the last slice in the 3D medical image frame, then it can be determined that there will be relevant spatial information above the current slice. At this time, the previous slice in the frame stacking order that precedes the current slice can be determined from the multiple slices that have been acquired in the past, and it can be used as the neighboring slice of the current slice.

[0108] In this embodiment, by selecting a spatially appropriate layer from among the acquired layers as a neighboring layer according to the intra-frame stacking order of the current layer, it helps to improve the accuracy of supplementing neighboring spatial image information.

[0109] In one embodiment, the front layer can be determined by the following steps:

[0110] Identify the acquired preceding three-dimensional medical image frame; from the multiple slices corresponding to the preceding three-dimensional medical image frame, identify at least one slice whose intra-frame stacking order precedes the current slice as the preceding slice before the current slice; and / or, if a preceding slice of the same frame is acquired, identify the preceding slice of the same frame as the preceding slice before the current slice.

[0111] In some cases, neighboring slices and the current slice may belong to different 3D medical image frames. In practical applications, when transmitting the slice of the current 3D medical image frame, the complete preceding 3D medical image frames can already be obtained. The preceding 3D medical image frames refer to the 3D medical image frames whose frame order precedes the 3D medical image frame to which the current slice belongs. For example, if the current slice belongs to the k-th 3D medical image frame, then one or more 3D medical image frames preceding the k-th 3D medical image frame can be identified as the preceding 3D medical image frames.

[0112] For example, a 3D medical image frame that is directly adjacent to the 3D medical image frame to which the current slice belongs and whose frame order precedes that 3D medical image frame can be considered as the preceding 3D medical image frame. For instance, the (k-1)th 3D medical image frame can be considered as the preceding 3D medical image frame of the kth 3D medical image frame. Alternatively, there can be a certain time interval, such as the kxth 3D medical image frame being considered as the preceding 3D medical image frame of the kth 3D medical image frame, where x is a positive integer greater than or equal to 2. In some cases, the accuracy of spatial image features is negatively correlated with the frame interval between the preceding 3D medical image frame and the 3D medical image frame to which the current slice belongs; that is, the smaller the frame interval, the higher the accuracy of the obtained spatial image features.

[0113] Then, layers whose stacking order precedes the current layer within the frame can be identified as the preceding layers. For example, if the current layer is the k-th layer in the n-th frame, the (k-1)-th layer in the (n-1)-th frame can be considered the preceding layer. This means that multiple layers whose stacking order precedes the current layer within the frame can be considered as preceding layers, thereby adding more complete spatial information above the incomplete layered image data.

[0114] Since the preceding three-dimensional medical image frame can often be obtained when acquiring the current slice, the preceding slice can be obtained by using the slice of the preceding three-dimensional medical image frame, which can avoid waiting and obtain the spatial information above in a timely and effective manner.

[0115] In other embodiments, neighboring layers may belong to the same 3D medical image frame as the current layer. For example, if the transmission of layers has high real-time requirements, when the current layer is acquired, the preceding layers of the same frame may also have been acquired. The preceding layers of the same frame are those belonging to the same 3D medical image frame as the current layer and whose intra-frame stacking order precedes the current layer. Therefore, when a preceding layer of the same frame is acquired, it can be determined as the preceding layer before the current layer. Since the preceding layers of the same frame and the current layer belong to the same 3D medical image frame, determining the preceding layers of the same frame as the preceding layers of the current layer increases the accuracy of the spatial information above the current layer. It is understandable that if the previous layer of the same frame is not obtained, the previous layer can be obtained based on the layers in the previous 3D medical image frames. If both the previous layer of the same frame and the previous 3D medical image frame are obtained at the same time, it can be determined whether to use the previous layer of the same frame as the previous layer based on the interval between the previous layer of the same frame and the current layer. For example, if the frame interval between the previous layer of the same frame and the current layer is greater than a threshold, the previous layer can be selected in the previous 3D medical image frames.

[0116] In an exemplary embodiment, the following steps can be used to determine the back layer:

[0117] Identify the acquired preceding three-dimensional medical image frames; from the multiple layers corresponding to the preceding three-dimensional medical image frames, identify at least one layer whose intra-frame stacking order is later than the current layer as the subsequent layer after the current layer.

[0118] Among them, the preceding three-dimensional medical image frame is the three-dimensional medical image frame whose frame order precedes the three-dimensional medical image frame to which the current slice belongs.

[0119] In the specific implementation, after determining the acquired preceding 3D medical image frames, layers whose stacking order is later than the current layer within the frame can be identified as subsequent layers. For example, if the current layer is the k-th layer of the n-th frame, the (k+1)-th layer of the (n-1)-th frame can be considered as the subsequent layer. This means that multiple layers whose stacking order is later than the current layer within the frame can be considered as subsequent layers, thereby adding more complete spatial information below the incomplete layer image data.

[0120] Since the preceding three-dimensional medical image frame can often be obtained when acquiring the current slice, the subsequent slice can be obtained based on the slice of the preceding three-dimensional medical image frame, which can avoid waiting and obtain the spatial information below the current slice in a timely and effective manner.

[0121] In an exemplary embodiment, the adjacent layers include preceding and / or subsequent layers determined from the layers corresponding to the preceding 3D medical image frames, wherein the preceding 3D medical image frames are 3D medical image frames whose frame order precedes the 3D medical image frames to which the current layer belongs.

[0122] Accordingly, in step S102, the layer image features of the current layer are completed in the layer stacking direction based on the layer image features of neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer. This may include the following steps:

[0123] Based on the slice image features of the preceding slice, predict the slice image features of the image content in the preceding slice at the time of acquisition of the current slice, and / or, based on the slice image features of the following slice, predict the slice image features of the image content in the following slice at the time of acquisition of the current slice; in the slice stacking direction, based on the slice image features of the image content in the preceding slice and / or following slice at the time of acquisition of the current slice, complete the slice image features of the current slice to obtain spatial image features containing neighboring spatial image information of the current slice.

[0124] The acquisition time can be the time when the target object is scanned. The image content in the slice can be obtained by scanning the target object with medical imaging equipment at a certain time. The slice image content acquired at different acquisition times may differ.

[0125] In practice, upon acquiring the current slice, the slice corresponding to the spatial position in the preceding 3D medical image frame can be received, obtaining at least one of the preceding and following slices. However, the acquisition time corresponding to the slice in the preceding medical image frame differs from the acquisition time corresponding to the current slice. Consequently, the image content of the preceding and following slices obtained based on the preceding 3D medical image frame may also differ from the image content of the slice in the vertical space obtained at the acquisition time corresponding to the current slice. If the slice image features of the current slice are directly completed based on the slice image features of the preceding and following slices, there is still room for improvement in the accuracy of the feature completion effect.

[0126] In this embodiment, the image features of the image content in the preceding layer at the current layer acquisition time can be predicted based on the image features of the preceding layer. Similarly, the image features of the image content in the following layer at the current layer acquisition time can be predicted based on the image features of the following layer. In other words, by predicting the changes in the preceding and following layers, the image features of the adjacent upper and lower layers at a later time can be predicted based on the image features of the image content at a previous time.

[0127] Furthermore, by using the predicted image content in the preceding and / or following layers at the acquisition time of the current layer in the layer stacking direction, the layer image features of the current layer can be supplemented, which can effectively improve the accuracy of the obtained spatial image features, so that the spatial image features can more accurately and effectively reflect the vertical spatial information of the current layer at the corresponding acquisition time.

[0128] In an exemplary embodiment, each slice image feature is a three-dimensional feature; accordingly, based on the slice image features of the image content in the previous slice and / or the subsequent slice at the time of acquisition of the current slice, the slice image features of the current slice are completed to obtain spatial image features containing the neighboring spatial image information of the current slice, which may include the following steps:

[0129] Along the stacking direction of the layers, at least two of the following are stacked sequentially: the layer image features of the image content in the previous layer at the time of acquisition of the current layer, the layer image features of the current layer, and the layer image features of the image content in the subsequent layer at the time of acquisition of the current layer, to obtain the feature stitching result; based on the feature stitching result, spatial image features containing neighboring spatial image information of the current layer are obtained.

[0130] Specifically, firstly, in the layer stacking direction, at least two of the following can be stacked sequentially: the layer image features of the image content in the preceding layer at the time of acquisition in the current layer, the layer image features of the current layer, and the layer image features of the image content in the following layer at the time of acquisition in the current layer, to obtain a feature stitching result. The feature stitching result must include at least the layer image features of the current layer. In other words, when performing feature stitching along the stacking direction of the layers, since the main purpose is to supplement the layer image features of the current layer, the stitched layer image features can at least include the layer image features of the current layer. For example, it is possible to stitch only the layer image features of the image content in the previous layer at the time of acquisition of the current layer with the layer image features of the current layer, or it is possible to stitch only the layer image features of the current layer with the layer image features of the image content in the subsequent layer at the time of acquisition of the current layer. Of course, it is also possible to stitch together the layer image features of the image content in the previous layer at the time of acquisition of the current layer, the layer image features of the current layer, and the layer image features of the image content in the subsequent layer at the time of acquisition of the current layer.

[0131] After obtaining the feature stitching result, spatial image features containing neighboring spatial image information of the current slice can be obtained based on the feature stitching result. In some embodiments, after the feature stitching is completed, no further processing of the feature stitching result can be performed (e.g., other image content in the current slice is not changed), and the feature stitching result can be used as the spatial image feature containing neighboring spatial image information of the current slice.

[0132] In other embodiments, during the process of completing the slice image features of the current slice, feature padding can be used to specify information. The order of feature stitching and padding can be set according to actual conditions. In some embodiments, feature stitching can be performed first, followed by feature padding. For example, after obtaining the feature stitching result, blank three-dimensional features can be stitched onto the slice image features of the current slice along other directions to obtain the feature stitching result. That is, after stacking the predicted slice image features onto the slice image features of the current slice, blanking can be achieved by padding with three-dimensional features with fixed feature values. In other embodiments, two-dimensional padding can be performed first. For example, three-dimensional features can be filled onto the slice image features of the current slice first, and then the filled slice image features of the current slice can be stitched together with the predicted slice image features. It is understood that those skilled in the art can change the order of feature stitching and feature padding according to actual conditions.

[0133] In this context, "other directions" refers to directions where no feature stacking has been performed on the current slice's image features. This can be understood as follows: since each slice's image features are three-dimensional, if the slice stacking direction is determined based on the top and bottom of the current slice's image features, and the predicted slice features are stitched together above and below, then blank three-dimensional features can be stitched in front of, behind, to the left of, and to the right of the current slice's image features, thus obtaining the feature stitching result. By filling in blank three-dimensional features in other directions, more contextual information can be provided for edge image information, preventing the loss of important edge features during convolution, and helping to improve the performance of tasks such as object detection and segmentation.

[0134] In this embodiment, since multiple layers are stacked into a three-dimensional medical image frame according to the layer stacking direction, when stacking and stitching the layer image features corresponding to layers at different spatial locations, they can also be stacked according to the layer stacking direction, so that the stacked layer image features match the spatial distribution of multiple layers in the three-dimensional medical image frame, thereby improving the accuracy of the feature stitching results.

[0135] To enable those skilled in the art to better understand the above embodiments, the following example illustrates the embodiments of this application, but it should be understood that the embodiments of this application are not limited thereto.

[0136] In this example, a feature extraction network can be used to extract features from the currently acquired slices. Figure 6aThe diagram shows a structural schematic of the feature extraction network, which includes convolutional layers and concatenated convolutional layers. The concatenated convolutional layer, also known as the CTFRealtime concatenated convolutional layer, can be used to concatenate the image feature extraction results of a specific position in a previous frame with the features of the current slice position in the current frame. Figure 6b This illustrates a specific structure for splicing convolutional layers.

[0137] In practical applications, the acquired current slice can be input into a feature extraction network, and then the current slice (i.e., the k-th slice in the n-th frame) can be encoded to extract the slice image features F corresponding to the current slice. In addition, the feature extraction network can also be used to extract features F from the slice adjacent to the current frame above it (i.e., the (k-1)-th slice in the (n-1)-th frame). k-1 F is obtained by extracting features from the slice adjacent to the bottom of the previous frame (the (k+1)th slice of the (n-1)th frame). k+1 It is understandable that feature F k-1 and feature F k+1 The corresponding slice can be processed immediately upon acquisition, instead of waiting for the next slice (e.g., the k-th slice of frame n). The extracted slice image features can be stored for later retrieval. For example, the feature extraction formulas for the (k-1)-th slice of frame n-1, the k-th slice of frame n, and the (k+1)-th slice of frame n-1 can be as follows, where f() represents the feature extraction of the input slice using a feature extraction network.

[0138] F k-1 =f(Image) (n-1k-1) (1)

[0139] F = f(Image) nk (2)

[0140] F k+1 =f(Image) (n-1k+1 (3)

[0141] Then, the image features of the layers corresponding to adjacent positions in the previous frame can be predicted by a prediction layer (such as an F2F network prediction layer) to obtain the layer data of the adjacent positions at the current acquisition time corresponding to the current layer. The feature F of the adjacent upper position in the previous frame... k-1 After passing through the prediction layer, the data F' of the features adjacent to the previous frame at the current time is obtained. k-1 The feature F adjacent to the bottom of the previous frame k+1 After passing through the prediction layer, the data F' of the features adjacent to the lower frame in the previous frame is obtained at the current time. k+1The processing can be described as follows, where F2F() is the prediction layer.

[0142] F′ k-1 =F2F(F k-1 (4)

[0143] F′ k+1 =F2F(F k+1 (5)

[0144] Furthermore, the three feature maps can be stitched together in the physical space direction of the layers. Since each layer was obtained from a CT helical scan, they are arranged along a head-to-toe direction in physical space; therefore, this direction is defined as the z-direction, which is also the layer stacking direction. Correspondingly, as... Figure 6b As shown, the three feature maps are concatenated along the z-axis to obtain F1. Meanwhile, padding is used to leave blank spaces in the other four directions where no concatenation occurs, similar to the concatenation process, except that the concatenated feature maps are all zeros. Zero Finally, the feature map F3 is obtained after splicing and padding.

[0145] F1 = Cat z (F′ k-1 , F, F′ k+1 (6)

[0146] F2 = Cat x (F Zero F1, F Zero (7)

[0147] F3 = Cat y (F Zero F2, F Zero (8)

[0148] The feature maps after stitching and padding are then processed by convolutional layers for feature fusion, finally obtaining the spatial image features F' of the kth layer in the nth frame.

[0149] F′=g(F3)

[0150] It is understood that the aforementioned "previous frame" can be a 3D medical image frame collected at the previous moment of the current frame, a previously acquired 3D medical image frame at a fixed time interval from the current frame, or multiple previously acquired 3D medical image frames. In this example, only the features of the upper and lower layers of the previous frame are used during stitching. In other embodiments, information from multiple upper and lower layers can be used to add more complete vertical spatial information to the incomplete 2D layered image data.

[0151] In some embodiments, if the target object is segmented based on spatial image features, the image segmentation deep learning network used can also be a two-dimensional image processing network, such as ResNet, SegNet, etc. Only the 2D convolutional layer in the network structure needs to be replaced with the CTFRealtime concatenation convolutional layer to realize the processing of incomplete two-dimensional image data, achieve high algorithm accuracy, and realize image processing functions such as image segmentation and detection.

[0152] In this example, by stitching image information from a specific position in a previous frame to the current slice position in the current frame, spatial information about the vertical position of the current slice in the current frame is supplemented. This allows for the construction of a complete three-dimensional space with vertical position information during the segmentation of the current slice in the current frame. This provides the vertical spatial information required for processing the incomplete two-dimensional image, solves the problem of incomplete segmentation caused by the lack of vertical spatial information during the segmentation of the current slice in the current frame, and improves the segmentation accuracy.

[0153] In related technologies, image processing in the real-time CT image processing workflow involves image segmentation, image detection, and image prediction. These image processing steps are performed after receiving a complete 3D image frame, which can lead to overly concentrated computational demands and prevent timely processing. To address this, this application can break down the various image processing requirements involved in real-time CT mode, satisfying different types of image processing needs at different stages of image transmission.

[0154] The various image processing requirements may include a first image processing requirement, which can be understood as an image processing requirement solely based on a two-dimensional slice image. In some exemplary embodiments, in step S102, obtaining the slice processing result of the current slice based on spatial image features may include the following steps: segmenting the target object of the current slice based on the spatial image features of the current slice to obtain a local segmentation result. That is, during the acquisition of multiple slices, target object segmentation of the current slice can be performed in parallel based on the spatial image features corresponding to the current slice. Since the image information corresponding to the slice may only include a part of the target object, the segmentation result obtained by segmenting the target object of the current slice is also called a local segmentation result.

[0155] Accordingly, in step S103, obtaining the target object tracking result of the three-dimensional medical image frame based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame may include the following steps: stitching together the local segmentation results of multiple slices belonging to the same three-dimensional medical image frame, and obtaining the target object tracking result of the three-dimensional medical image frame based on the stitched local segmentation results.

[0156] Specifically, for each current slice obtained, the local segmentation result of the current slice can be obtained based on the spatial image features, such as... Figure 3a As shown, by processing each currently acquired slice in parallel during the slice acquisition process, the local segmentation results of each slice can be obtained in a timely manner, avoiding the need to wait until all slices of the same 3D medical image frame are acquired before processing them collectively. Furthermore, after acquiring the local segmentation results of multiple slices of the 3D medical image frame, the target object segmentation result of the 3D medical image frame can be obtained by stitching the various local segmentation results together. This result can be used as the target object tracking result of the 3D medical image frame.

[0157] In this embodiment, on the one hand, by segmenting multiple layers of a complete three-dimensional medical image frame separately and then stitching together the multiple local segmentation results, the target object tracking result of the complete three-dimensional medical image frame can be obtained, effectively improving the timeliness of target object segmentation. On the other hand, since this embodiment segments the current layer based on spatial image features containing neighboring spatial image information of the current layer, the spatial image information can supplement the neighboring spatial information of the current layer. Thus, when segmenting the current layer, a more complete three-dimensional space with vertical position information can be constructed, providing the vertical spatial information required by the currently incomplete two-dimensional layer. This solves the problem of incomplete segmented objects caused by the lack of vertical spatial information when segmenting the current layer, improves the segmentation accuracy, and ensures the accuracy of the segmentation result while segmenting the target object in the current layer in a timely manner.

[0158] In some cases, multiple image processing requirements may include a second image processing requirement. This second image processing requirement can achieve better processing results when processed within a complete 3D medical image frame. For example, the processing of the second image processing requirement may depend on a complete 3D medical image frame. In some examples, the second image processing requirement may include tip tracking of surgical instruments in 3D image detection; image processing in 3D space can yield more accurate tip tracking results.

[0159] In an exemplary embodiment, the slice processing result includes the spatial image features of the current slice, that is, the spatial image features obtained for the current slice can be used as the basis for the processing. Accordingly, in step S103, obtaining the target object tracking result of the three-dimensional medical image frame based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame may include the following steps:

[0160] The spatial image features of multiple layers belonging to the same 3D medical image frame are fused to obtain fused spatial image features; based on the fused spatial image features, the target object tracking results of the 3D medical image frame are obtained.

[0161] In some implementations, the processing of a complete 3D image frame includes a feature extraction process. Taking the segmentation of a complete 3D image frame using a Unet network as an example, this includes an encoder for feature extraction of the 3D image frame and a decoder for uploading.

[0162] In this embodiment, since features are extracted from the current layer during the acquisition of each layer, the spatial image features of the current layer can be stored and then processed for the next layer. When the spatial image features of multiple layers belonging to the same 3D medical image frame are obtained, these multiple spatial image features can be fused to obtain fused spatial image features. This avoids repeated extraction and calculation of image features and accelerates the image processing flow of the complete 3D medical frame. The feature extraction for the complete 3D medical image frame is broken down into feature extraction for each layer and evenly distributed during the layer transmission process, improving the speed of image feature extraction. Subsequently, by inputting the fused spatial image features into the decoder, the target object tracking result of the 3D medical image frame can be decoded. Thus, this embodiment effectively avoids repeated calculations in the image feature extraction process by performing feature extraction, image segmentation, and stitching in parallel during the acquisition of the video stream, significantly accelerating the frame processing flow of the complete 3D medical image frame and improving the real-time performance of the 3D image processing algorithm.

[0163] For example, such as Figure 7 As shown, the layer processing results of all two-dimensional layers belonging to the same three-dimensional medical image frame can be directly used in the image processing flow of the complete three-dimensional medical image frame. There is no need for the encoder to extract features. The results are directly input into the decoder to obtain the inference results, such as the prediction results of the needle tip position in the three-dimensional medical image frame.

[0164] In one exemplary embodiment, the method may further include the following steps:

[0165] During the process of acquiring the current slice, the target preprocessing task corresponding to the current slice is executed in parallel to obtain the preprocessing task result.

[0166] In related technologies, data processing for each 3D medical image frame transmitted during real-time scanning is primarily performed frame-by-frame. Preprocessing often involves waiting for a complete 3D medical image frame to be acquired before processing multiple layers of the same frame simultaneously. However, the inventors discovered in practice that this preprocessing method leads to excessive concentration of computational resources, consuming significant resources during preprocessing and hindering the normal execution of other concurrent tasks. Furthermore, because preprocessing multiple layers is completed concurrently before subsequent layer processing, the real-time performance of image processing results is difficult to meet.

[0167] In this embodiment, a target preprocessing task corresponding to the current slice can be determined and executed before the complete current slice is received, thus obtaining the corresponding preprocessing task result. The target preprocessing task corresponding to the current slice can be related to the currently acquired slice, but its execution process does not depend on the preprocessing task of the currently acquired slice. In other words, the preprocessing result of the target preprocessing task will participate in the subsequent real-time calculation process of the current slice, but the execution process of the target preprocessing task does not depend on the slice data of the currently acquired slice. Through the processing method of this embodiment, the target preprocessing task can be pre-positioned in the image transmission process of the current slice, executed in parallel with the transmission of the current slice, thereby starting the target preprocessing task before the complete current slice is received.

[0168] Accordingly, the spatial image features of multiple layers belonging to the same three-dimensional medical image frame are fused to obtain fused spatial image features, which may include the following steps:

[0169] Based on the spatial image features of multiple layers belonging to the same 3D medical image frame and the preprocessing results obtained, feature fusion is performed to obtain the fused spatial image features.

[0170] Specifically, in some embodiments, the results of the preprocessing task can be applied to the fusion process of spatial image features. By performing the target preprocessing task during the acquisition of each current slice, the preprocessing task results related to the fusion of spatial image features can be obtained. Furthermore, when fusing the spatial image features of multiple slices belonging to the same 3D medical image frame, feature fusion can be performed based on the spatial image features of the multiple slices belonging to the same 3D medical image frame and the preprocessing task results corresponding to each of the multiple slices obtained in advance, to obtain the fused spatial image features.

[0171] It is understood that, in some other embodiments, the results of the preprocessing task can also be applied to other stages of the slice processing process. When the target preprocessing task is related to the current slice being acquired, and the task execution process does not depend on the current slice being acquired, those skilled in the art can set the task content of the target preprocessing task and the application method of the obtained preprocessing task results in subsequent slice processing processes according to the actual situation. For example, if the current slice involves using the slice image features corresponding to the previous slice to complete the slice image feature when performing slice image feature completion, the target preprocessing task may include extracting the slice image features of the previous slice, or searching for and reading the previously stored slice image features of the previous slice. For example, in some exemplary embodiments, it is necessary to extract features from the image and mask of the previous 3D medical image frame to obtain key and value values, and then calculate them with the key and value values ​​obtained from the feature extraction of the current 3D medical image frame. After processing by the decoder, the segmentation result of the current frame is obtained. The calculation process of extracting features from the image and mask of the previous frame to obtain key and value values ​​does not require the data of the current frame, so this part of the calculation can be moved forward to the image transmission process and run in parallel with the image transmission.

[0172] For example, such as Figure 8 As shown, during the transmission of multiple 3D medical image frames, data preprocessing can be performed in advance by executing a target preprocessing task. After obtaining the current slice, 2D image processing can be performed on the current slice in combination with the preprocessing task results to obtain the slice processing result. In some examples, if the slice processing result includes the local segmentation result of the current slice, it can be displayed. If the slice processing result is the spatial image feature of the slice, then after obtaining the spatial image features corresponding to multiple slices belonging to the same 3D medical image frame, 3D image features can be stitched together. 3D image processing can then be performed based on the stitched 3D image features to obtain the 3D image processing result.

[0173] In this embodiment, by performing target preprocessing tasks independent of the current slice in advance during the acquisition of the current slice, the computing resources in the image transmission process can be fully utilized, and data preprocessing related to the current slice can be performed in advance. This allows data preprocessing, two-dimensional slice image processing, and three-dimensional medical image frame transmission to be executed in parallel, processing the received slice data in real time and reducing image processing latency.

[0174] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0175] Based on the same inventive concept, this application also provides a target object tracking device for implementing the target object tracking method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more target object tracking device embodiments provided below can be found in the limitations of the target object tracking method described above, and will not be repeated here.

[0176] In one exemplary embodiment, such as Figure 9 As shown, a target object tracking device is provided, comprising:

[0177] Image acquisition module 901 is used to acquire slices for composing a three-dimensional medical image frame; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked;

[0178] The two-dimensional processing module 902 is used to perform layer processing steps in parallel on the currently acquired layer during the acquisition of each layer to obtain the layer processing result of the current layer. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the intra-frame stacking order adjacent to the current layer from the previously acquired layers; and completing the layer image features of the current layer in the layer stacking direction according to the layer image features of the neighboring layers to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer according to the spatial image features.

[0179] The 3D processing module 903 is used to obtain the target object tracking result of the 3D medical image frame based on the slice processing results of multiple slices belonging to the same 3D medical image frame.

[0180] In one embodiment, the two-dimensional processing module 902 is used for:

[0181] If the current layer is the first in the intra-frame stacking order, then the next layer in the intra-frame stacking order after the current layer is determined from the previously acquired layers and is taken as the neighboring layer of the current layer.

[0182] If the intra-frame stacking order of the current slice is between the first and last slices, then the preceding slice with an intra-frame stacking order before the current slice and / or the following slice with an intra-frame stacking order after the current slice are determined from the previously acquired slices and are taken as the neighboring slices of the current slice.

[0183] If the current tile is the last in the intra-frame stacking order, then the previous tile whose intra-frame stacking order precedes the current tile is determined from the previously acquired tiles and is taken as the neighboring tile of the current tile.

[0184] In one embodiment, the two-dimensional processing module 902 is used for:

[0185] Identify the preceding three-dimensional medical image frames that have been acquired; the preceding three-dimensional medical image frames are the three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current slice belongs;

[0186] From the multiple layers corresponding to the previous three-dimensional medical image frame, at least one layer whose intra-frame stacking order precedes the current layer is determined as the previous layer preceding the current layer.

[0187] And / or,

[0188] If a frame is found to be in the preceding layer, the preceding layer is defined as the layer preceding the current layer; the preceding layer is a layer that belongs to the same three-dimensional medical image frame as the current layer and whose intra-frame stacking order precedes the current layer.

[0189] In one embodiment, the two-dimensional processing module 902 is used for:

[0190] Identify the preceding three-dimensional medical image frames that have been acquired; the preceding three-dimensional medical image frames are the three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current slice belongs;

[0191] From the multiple layers corresponding to the preceding three-dimensional medical image frame, at least one layer whose intra-frame stacking order is later than the current layer is determined as the subsequent layer after the current layer.

[0192] In one embodiment, the adjacent layers include the preceding and / or following layers determined from the layers corresponding to the preceding three-dimensional medical image frames, wherein the preceding three-dimensional medical image frames are three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current layer belongs.

[0193] The two-dimensional processing module 902 is used for:

[0194] Based on the layer image features in the preceding layer, predict the layer image features of the image content in the preceding layer at the current layer acquisition time, and / or, based on the layer image features in the following layer, predict the layer image features of the image content in the following layer at the current layer acquisition time.

[0195] In the stacking direction of the layers, based on the layer image features of the image content in the preceding layer and / or the following layer at the time of acquisition of the current layer, the layer image features of the current layer are completed to obtain spatial image features containing the neighboring spatial image information of the current layer.

[0196] In one embodiment, each of the layered image features is a three-dimensional feature;

[0197] The two-dimensional processing module 902 is used for:

[0198] Along the stacking direction of the layers, at least two of the following are stacked sequentially: the layer image features of the image content in the previous layer at the current layer acquisition time, the layer image features of the current layer, and the layer image features of the image content in the subsequent layer at the current layer acquisition time, to obtain the feature stitching result.

[0199] Based on the feature stitching result, spatial image features containing neighboring spatial image information of the current slice are obtained.

[0200] In one embodiment, the layer processing result includes a local segmentation result obtained by segmenting the target object of the current layer based on the spatial image features of the current layer;

[0201] The three-dimensional processing module 903 is used for:

[0202] The local segmentation results of multiple layers belonging to the same three-dimensional medical image frame are stitched together, and the target object tracking result of the three-dimensional medical image frame is obtained based on the stitched local segmentation results.

[0203] In one embodiment, the layer processing result includes the spatial image features of the current layer;

[0204] The three-dimensional processing module 903 is used for:

[0205] The spatial image features of multiple layers belonging to the same three-dimensional medical image frame are fused to obtain fused spatial image features.

[0206] Based on the fused spatial image features, the target object tracking results of the three-dimensional medical image frame are obtained.

[0207] In one embodiment, the method further includes:

[0208] During the process of acquiring the current slice, the target preprocessing task corresponding to the current slice is executed in parallel to obtain the preprocessing task result; the target preprocessing task corresponding to the current slice is a preprocessing task that is related to the current slice being acquired and whose task execution process does not depend on the current slice being acquired.

[0209] The three-dimensional processing module 903 is used for:

[0210] Based on the spatial image features of multiple layers belonging to the same three-dimensional medical image frame and the obtained preprocessing task results, feature fusion is performed to obtain fused spatial image features.

[0211] Each module in the aforementioned target object tracking device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0212] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores CT data, such as three-dimensional medical image frames and / or multiple layers that make up three-dimensional medical image frames. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a target object tracking method.

[0213] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a target object tracking method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0214] Those skilled in the art will understand that Figure 10 and Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0215] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0216] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0217] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0218] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0219] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0220] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0221] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target object tracking method, characterized in that, The method includes: The slices used to compose a three-dimensional medical image frame are obtained based on the slice stream transmission results; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked; there is a time interval between the acquisition of multiple slices corresponding to the same three-dimensional medical image frame; During the acquisition of each slice, slice processing steps are performed in parallel on the currently acquired slice to obtain the slice processing result of the current slice. The acquisition of each slice includes at least a time period during which no other slices belonging to the same three-dimensional medical image frame as the current slice are acquired. The slice processing is performed in parallel with the acquisition of the other slices. The slice processing steps include: determining the slice image features of the currently acquired slice; determining the neighboring slices with the same intra-frame stacking order as the current slice from the previously acquired slices; predicting the slice image features of the image content in the neighboring slices at the acquisition time of the current slice based on the slice image features of the neighboring slices; completing the slice image features of the current slice in the slice stacking direction based on the predicted slice image features at the acquisition time of the current slice to obtain spatial image features containing the neighboring spatial image information of the current slice; and obtaining the slice processing result of the current slice based on the spatial image features. Based on the slice processing results of multiple slices belonging to the same 3D medical image frame, the target object tracking result of the 3D medical image frame is obtained.

2. The method according to claim 1, characterized in that, The step of determining the neighboring layers whose intra-frame stacking order is adjacent to the current layer from previously acquired layers includes: If the current layer is the first in the intra-frame stacking order, then the next layer in the intra-frame stacking order after the current layer is determined from the previously acquired layers and is taken as the neighboring layer of the current layer. If the intra-frame stacking order of the current slice is between the first and last slices, then the preceding slice with an intra-frame stacking order before the current slice and / or the following slice with an intra-frame stacking order after the current slice are determined from the previously acquired slices and are taken as the neighboring slices of the current slice. If the current tile is the last in the intra-frame stacking order, then the previous tile whose intra-frame stacking order precedes the current tile is determined from the previously acquired tiles and is taken as the neighboring tile of the current tile.

3. The method according to claim 2, characterized in that, The following steps are used to determine the front layer: Identify the preceding three-dimensional medical image frames that have been acquired; the preceding three-dimensional medical image frames are the three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current slice belongs; From the multiple layers corresponding to the previous three-dimensional medical image frame, at least one layer whose intra-frame stacking order precedes the current layer is determined as the previous layer preceding the current layer. And / or, If a frame is found to be in the preceding layer, the preceding layer is defined as the layer preceding the current layer; the preceding layer is a layer that belongs to the same three-dimensional medical image frame as the current layer and whose intra-frame stacking order precedes the current layer.

4. The method according to claim 2 or 3, characterized in that, The following steps are used to determine the position in the back layer: Identify the preceding three-dimensional medical image frames that have been acquired; the preceding three-dimensional medical image frames are the three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current slice belongs; From the multiple layers corresponding to the preceding three-dimensional medical image frame, at least one layer whose intra-frame stacking order is later than the current layer is determined as the subsequent layer after the current layer.

5. The method according to claim 1, characterized in that, The adjacent layers include the preceding and / or subsequent layers determined from the layers corresponding to the preceding three-dimensional medical image frames, wherein the preceding three-dimensional medical image frames are three-dimensional medical image frames whose frame order precedes the three-dimensional medical image frames to which the current layer belongs. The step of predicting the image content of the neighboring layers at the current layer acquisition time based on the image features of the neighboring layers, and then, in the layer stacking direction, completing the image features of the current layer based on the predicted image features at the current layer acquisition time to obtain spatial image features containing the neighboring spatial image information of the current layer, includes: Based on the layer image features in the preceding layer, predict the layer image features of the image content in the preceding layer at the current layer acquisition time, and / or, based on the layer image features in the following layer, predict the layer image features of the image content in the following layer at the current layer acquisition time. In the stacking direction of the layers, based on the layer image features of the image content in the preceding layer and / or the following layer at the time of acquisition of the current layer, the layer image features of the current layer are completed to obtain spatial image features containing the neighboring spatial image information of the current layer.

6. The method according to claim 5, characterized in that, Each of the aforementioned layered image features is a three-dimensional feature; The step of completing the layer image features of the current layer based on the layer image features of the image content in the preceding layer and / or the following layer at the time of acquisition of the current layer, to obtain spatial image features containing the neighboring spatial image information of the current layer, includes: Along the stacking direction of the layers, at least two of the following are stacked sequentially: the layer image features of the image content in the previous layer at the time of acquisition in the current layer, the layer image features of the current layer, and the layer image features of the image content in the subsequent layer at the time of acquisition in the current layer, to obtain a feature stitching result; the feature stitching result includes the layer image features of the current layer. Based on the feature stitching result, spatial image features containing neighboring spatial image information of the current slice are obtained.

7. The method according to claim 1, characterized in that, The layer processing result includes the local segmentation result obtained by segmenting the target object in the current layer based on the spatial image features of the current layer; The step of obtaining the target object tracking result of the three-dimensional medical image frame based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame includes: The local segmentation results of multiple layers belonging to the same three-dimensional medical image frame are stitched together, and the target object tracking result of the three-dimensional medical image frame is obtained based on the stitched local segmentation results.

8. The method according to claim 1 or 7, characterized in that, The layer processing result includes the spatial image features of the current layer; The step of obtaining the target object tracking result of the three-dimensional medical image frame based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame includes: The spatial image features of multiple layers belonging to the same three-dimensional medical image frame are fused to obtain fused spatial image features. Based on the fused spatial image features, the target object tracking results of the three-dimensional medical image frame are obtained.

9. The method according to claim 8, characterized in that, The method further includes: During the process of acquiring the current slice, the target preprocessing task corresponding to the current slice is executed in parallel to obtain the preprocessing task result; the target preprocessing task corresponding to the current slice is a preprocessing task that is related to the current slice being acquired and whose task execution process does not depend on the current slice being acquired. The process of fusing the spatial image features of multiple layers belonging to the same three-dimensional medical image frame to obtain fused spatial image features includes: Based on the spatial image features of multiple layers belonging to the same three-dimensional medical image frame and the obtained preprocessing task results, feature fusion is performed to obtain fused spatial image features.

10. A target object tracking device, characterized in that, The device includes: An image acquisition module is used to acquire slices for composing a three-dimensional medical image frame based on slice stream transmission results; multiple slices are stacked in the slice stacking direction according to the intra-frame stacking order to form a three-dimensional medical image frame containing the target object to be tracked; there is a time interval between the acquisition of multiple slices corresponding to the same three-dimensional medical image frame; A two-dimensional processing module is used to perform layer processing steps in parallel on the currently acquired layer during the acquisition of each layer to obtain the layer processing result of the current layer. The process of acquiring each layer includes at least a time period during which no other layers belonging to the same three-dimensional medical image frame as the current layer are acquired. The layer processing is performed in parallel with the acquisition of the other layers. The layer processing steps include: determining the layer image features of the currently acquired layer; determining the neighboring layers with the same intra-frame stacking order as the current layer from the previously acquired layers; predicting the layer image features of the image content in the neighboring layers at the time of acquisition of the current layer based on the layer image features of the neighboring layers; completing the layer image features of the current layer in the layer stacking direction based on the predicted layer image features at the time of acquisition of the current layer to obtain spatial image features containing the neighboring spatial image information of the current layer; and obtaining the layer processing result of the current layer based on the spatial image features. The three-dimensional processing module is used to obtain the target object tracking result of the three-dimensional medical image frame based on the slice processing results of multiple slices belonging to the same three-dimensional medical image frame.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images