A semi-automatic bootstrap medical image sequence labeling method

Abstract

The application discloses a kind of semi-automatic bootstrap medical image sequence marking method, comprising the following steps: facing single frame or multiple frames DICOM and its derived sequence, frame analysis and export lossless image sequence, establish sequence index to support hierarchical batch access, sampling and cross-frame operation, estimate imaging window (sound window) and crop by interframe pixel variation statistics, support manual frame selection, reuse history crop frame and skip abnormal sequence, when marking, based on prompt point set and neighborhood brightness distribution adaptive threshold determination, connected extraction generates effusion target mask;Magnetic tracing is provided to highlight boundary and bone surface boundary is generated along texture fitting, optionally reuse last frame prompt point / mask or optical flow propagation to subsequent frame, and support cross-frame interpolation, and can be integrated segmentation model automatic marking, correction and incremental training to form closed loop;Mask is saved with file tail additional lossless compression section, supports cover update, does not affect general reading and can be restored.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and computer-aided annotation technology, and in particular to a semi-automatic bootstrapping method for labeling medical image sequences. Background Technology

[0002] Medical image segmentation and detection models typically rely on large amounts of pixel-level labeled data. Existing annotation methods mostly involve manual outlining, smearing, or polygon drawing frame by frame, which suffers from low efficiency, high labor intensity, and poor consistency among annotators. When the input is a continuous sequence of frames (such as dynamic scanning, interventional procedure records, etc.), these problems become even more prominent: adjacent frames are highly similar but still require repeated operations, and due to factors such as noise, occlusion, and tissue deformation, the boundaries of a single frame are often blurred, making manual judgment highly subjective.

[0003] Furthermore, medical images are often saved in DICOM format for clinical storage, and a single file may contain a large number of frames. Different devices / software output interface layouts vary, and the position, scale, and overlaid text of the imaging window in the entire image may change. If used directly for annotation and training, it will introduce a large number of irrelevant areas and reduce annotation efficiency. Current practices usually rely on manually setting the cropping box sequence by sequence or preprocessing in external software, which is difficult to process in batches and is prone to problems such as missed cropping, incorrect cropping, and inconvenience in reuse.

[0004] To reduce annotation costs, existing interactive annotation tools (Fedorov A, et al. 3D Slicer as an Image Computing Platform for the Quantitative Imaging Network (2012)) typically offer functions such as threshold selection, connected component extraction, or "smart scissors / magnetic lasso," but they are mostly single-frame based and lack capabilities such as reusing cue points for sequences, cross-frame propagation, and rapid sampling annotation. Some cross-frame propagation methods require additional sequence registration or complex settings, still requiring frequent manual intervention. Regarding the storage of annotation results, common solutions include saving the original image and the mask as separate files or writing the mask to the image as an overlay channel. This easily leads to problems such as file mismatch, complex path management, mask accuracy limited by image size, or incompatibility with general image reading software. Therefore, a semi-automatic labeling method for medical image sequences is needed, which allows operators to obtain high-quality labels with less interaction, and supports batch preprocessing, cross-frame interpolation propagation, and file formats that accommodate both general reading and lossless mask saving (Yushkevich PA, et al. User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability (2006). Summary of the Invention

[0005] This invention aims to solve the problems of low efficiency, large amount of repetitive work across frames, inconsistent imaging window cropping, and difficulty in managing labeled data in existing medical image sequence annotation. It provides a labeling method that reduces the number of interactions, improves labeling consistency, and facilitates batch processing and storage management while ensuring labeling quality. It can be used for labeling, dataset construction, and verification.

[0006] The present invention is achieved by at least one of the following technical solutions.

[0007] A semi-automatic bootstrapping method for labeling medical image sequences includes the following steps: S1. Obtain the medical image sequence data to be labeled and establish a sequence index structure; S2. Determine the effective imaging window region based on medical image sequence data and crop the medical image sequence data to obtain the cropped imaging window image sequence. S3. For closed area targets, based on the set of prompt points in the operator's target area and the distribution of neighboring pixels of the prompt points, an adaptive threshold is determined, and connected component extraction is performed under the threshold constraint to generate target annotation mask. S4. For semi-open boundary targets, receive sparse cue points given by the operator and automatically generate boundary line masks by matching adjacent cue points along the texture based on magnetic delineation. S5. Based on the correlation between adjacent frames in the sequence, perform cross-frame multiplexing and / or propagation of the cue point set and / or annotation results to generate cue point sets and / or annotation results for subsequent frames; S6. Write the cropped image sequence and the corresponding annotation results into an image file, and append a lossless compressed annotation data segment to the end of the image file to restore and update the annotation mask without affecting general image reading.

[0008] Furthermore, in step S1, the establishment of the sequence index structure includes: establishing a hierarchical management structure according to sequence-frame number-tag type; wherein, the tag type may include closed area targets, semi-open boundary targets, ray targets and other custom tags.

[0009] Further, step S2 includes: in a multi-frame sequence scenario, obtaining a change intensity map by statistically analyzing the pixel changes between frames, and extracting the bounding rectangle of the largest connected region from the change intensity map as the cropping boundary; writing the cropping boundary into the auxiliary channel of the image to form a cropping mark, and then reading the cropping mark to perform batch cropping on the image; when there is an error in the automatic estimation or the input is single-frame data, the operator can manually select the cropping boundary and reuse the historical cropping boundary.

[0010] Furthermore, step S3 also includes: receiving the narrow edge-filling line mask drawn by the operator along the target boundary, and merging the narrow edge-filling line mask with the target annotation mask extracted from the connected region, so as to achieve edge filling and / or hole filling of the gap region.

[0011] Further, step S4 specifically includes: advancing along the connection direction at a set step size between adjacent prompt points, and searching for candidate positions in a local range along the vertical direction during the advancement process; calculating a cost function for the candidate positions, the cost function including at least a pixel intensity difference term and a center constraint term; selecting the candidate position with the minimum cost to form a fitting path, and smoothing the fitting path to generate the boundary line mask.

[0012] Furthermore, for ray-like structures in the puncture needle path, ray annotation is completed using two points: the ray direction is determined based on the two points and the ray annotation result is generated. When saving, only the two-point information is stored, and when reading, the ray is reconstructed from the two-point information, thereby reducing interaction and storage overhead.

[0013] Further, step S5 includes the following steps: Optical flow-based inter-frame tweening propagation: Select consecutive frame segments from the same sequence, each segment comprising at least three frames, where the first and last frames are labeled anchor frames; for each intermediate frame, calculate the dense displacement field between the intermediate frame and the first and last frames, and propagate the label mask and / or cue points of the anchor frames to the intermediate frames according to the deformation of the dense displacement field; attenuate the propagation mask based on the difference between the deformed image of the anchor frames and the image of the intermediate frames; then, perform weighted fusion of the propagation results at both ends according to the relative position of the intermediate frames in the sequence to obtain the intermediate frame label mask; wherein, the propagation mask for semi-open boundary targets is thickened before deformation propagation to reduce the impact of smoothing on the detectability of the boundary lines; and after weighted fusion, extract the first super-threshold pixel by column to output a single-pixel boundary line mask.

[0014] Further, step S5 includes: recording the operator's interaction sequence on the first frame, replaying the interaction sequence on the second frame to generate the annotation result of the second frame; and when it is detected that there is already an annotation result in the second frame, performing overwrite, append, or skip processing on the annotation result of the second frame. The interactive operation sequence includes a sequence of prompt point coordinates and their order information.

[0015] Further, step S6 includes: after writing the medical image into an image file in a preset image format, appending a labeling data segment to the end of the image file. The labeling data segment includes: a compressed byte sequence obtained by lossless compression of the labeling mask array, the length information of the compressed byte sequence, the shape information of the labeling mask array, the data type information, and a preset tail identifier; wherein, the shape information of the labeling mask array is used to decouple the resolution of the labeling mask array from the resolution of the medical image to adapt to the labeling mask array determined by the interactive rendering accuracy; and when the tail identifier is detected to already exist at the end of the image file, the existing labeling data segment is truncated before a new labeling data segment is appended, so that the image content of the image file can be read without parsing the labeling data segment.

[0016] A computer device according to the present invention includes: a memory and a processor, and a computer program stored in the memory, which, when executed on the processor, implements the method.

[0017] Compared with the prior art, the present invention has at least the following beneficial effects: (1) By using adaptive imaging window estimation and cropping, the preprocessing cost caused by differences in different devices / interfaces is reduced, and manual correction and reuse of historical cropping results are supported. (2) By using a set of prompts, adaptive thresholds and connected component extraction, a stable region mask can be obtained with a small number of interactions, reducing the subjective differences in fuzzy boundaries; (3) The boundary is automatically generated by magnetic tracing along the high-brightness texture, which reduces the amount of manual tracing and improves the consistency of the boundary; (4) By using optical flow interpolation propagation and cue point / mask reuse, the repetitive work per frame is significantly reduced and the efficiency of sequence annotation is improved; (5) By appending lossless compressed annotation data segments to the end of the image file, the "image + mask" single file management is realized. The mask is saved losslessly and supports overwrite updates without affecting general image reading, reducing the risk of mismatch and facilitating offline verification and incremental training. (6) Through hash-based reproducible sampling and bookmark management, it supports the rapid selection of representative frames from massive sequences for priority labeling and facilitates review and export; (7) By recording and replaying the interactive operation sequence of the previous frame, adjacent frames can be reused quickly, further reducing repetitive work; (8) By using two-point ray annotation and reconstruction, the interaction and storage overhead can be reduced while ensuring the annotation expressive ability. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the functional modules of the marking system of the present invention as an example.

[0019] Figure 2 This is a schematic diagram of the file structure with lossless compressed annotation data segments appended to the end of the image file as an example.

[0020] Figure 3 This is a schematic diagram illustrating the steps of a complete marking process as shown in the example. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings. To enable those skilled in the art to reproduce the invention, this section provides a detailed description of the implementation methods of each functional module, and provides embodiments at the end of the document. For ease of explanation, the embodiments use ultrasound imaging sequences as examples, but the present invention is not limited to medical imaging types and can also be applied to annotation scenarios of other medical imaging sequences (such as endoscopy, microscopy, CT / MR reconstruction sequences, etc.).

[0022] like Figure 1 , Figure 2 As shown, the method of the present invention can be executed by a computer device and provides multiple mutually cooperating modules through an interactive interface. The modules include at least: The data import / export and sequence indexing module is used to acquire medical image sequence data to be labeled and to build a sequence index.

[0023] The imaging window cropping module addresses the issue of inconsistent effective imaging window positions caused by different device / interface layouts. It can automatically estimate the imaging window in multi-frame sequence scenarios by utilizing inter-frame variation statistics and provides a two-stage mechanism of "cropping mark → batch cropping" for reuse and error correction.

[0024] The closed region intelligent delineation module is used to annotate closed region targets (such as low-contrast areas like effluent) by using prompt points / brush trajectories + threshold tolerance + connected region extraction, so as to reduce the image-by-image sketching edge.

[0025] The semi-open boundary magnetic tracing module is used to annotate semi-open boundary targets (such as highlighted boundaries like bone surfaces) using magnetic tracing. The operator only needs to provide sparse cue points, and the system can generate boundary lines by matching the texture between adjacent cue points.

[0026] The module for labeling and reconstructing ray-like structures is used to label ray-like structures such as puncture needle paths using two points.

[0027] The cross-frame multiplexing and replay module is used to record the interaction sequence of the first frame and replay it in subsequent frames to quickly generate initial annotations.

[0028] The optical flow interpolation module is used to perform optical flow interpolation propagation on consecutive frame segments of the same sequence and automatically generate annotation results for intermediate frames to further reduce frame-by-frame interaction.

[0029] The module for saving and exporting annotation data in a single file is used to save the annotation mask in a lossless format (single file) by appending a standard image data segment to the end and updating it over time. This reduces the risk of mismatch caused by saving the original image and the mask in separate files, and saves the mask losslessly without affecting the image content read by general image reading software.

[0030] In one embodiment, for each frame of the image, the following are maintained: image data (raw or cropped) and an array of annotation masks corresponding to that frame. The annotation mask array may use bit flag encoding for multiple types of labels, thereby carrying multiple categories of annotation information within an array; and optionally, some annotations may be written to an auxiliary channel of the image in a visual manner (e.g., as an overlay display) so that the annotation results can be quickly previewed even without parsing appended annotation segments at the end.

[0031] like Figure 3 As shown, a semi-automatic bootstrapping method for labeling medical image sequences in this embodiment includes the following steps: S1. Obtain the medical image sequence data to be labeled and establish a sequence index, including the following steps: (S11) Input data type: Input data may include single-frame DICOM, multi-frame DICOM, or image sequence folders exported from DICOM.

[0032] (S12) DICOM → Sequence Export: For multi-frame DICOM files, they can be parsed frame by frame according to a standard format and exported as a lossless image sequence (e.g., PNG) arranged in chronological order. This parsing and frame-by-frame export are standard format readings according to rules and can be used as system import / preprocessing functions, but are not intended to limit the scope of this invention.

[0033] (S13) Sequence Identification and Naming: To facilitate sequence association and consistency verification, the exported file name can adopt the rule of "frame number + sequence identifier". For example, use a 6-digit incrementing frame number as a prefix and the sequence path, DICOM key meta information or hash value of the sequence content as a suffix to obtain a naming method like "000000_<sequence identifier>.png"; wherein, the sequence identifier can be generated by MD5, SHA or other hash functions, and the present invention does not limit the specific generation method.

[0034] (S14) Sequence Index Structure: After exporting, a sequence index is created, and the data is organized in a hierarchical structure of "sequence - frame number - tag type". It supports batch access, sampling access and cross-frame operation. The tag type may include closed area targets, semi-open boundary targets, ray targets and other custom tags.

[0035] (S15) Reproducible Sampling: To quickly generate the first batch of labeled samples and reduce the initial labeling cost, reproducible sampling can be performed on massive frames. One type of sampling implementation involves the operator setting a seed. ,interval With offset And select a hash function. The system will match the candidate filenames (name) with the seed filenames. After concatenating the strings, calculate the hash to obtain an integer. And perform modulo operation filtering, for example, satisfying... The selected frames are added to the annotation set. To accommodate different sequence lengths, the number of candidate frames within the same sequence is set to be . It can be further Mapped to ,Will Restricted to This avoids out-of-bounds deviations and maintains stable and reproducible sampling.

[0036] (S16) Bookmarks: To facilitate review and export, keyframes can be added to a bookmark set. The bookmark set can be stored as a relative path list or a key-value set and used as a filtering condition during export.

[0037] S2. Determine the effective imaging window region based on medical image sequence data and crop the medical image sequence data to obtain a cropped imaging window image sequence, including the following steps: (S21) Grouping by resolution: Group the input sequence by resolution (and optional directory level), preferably only calculating the clipping boundary within the same group with the same resolution and layout to avoid cross-resolution misuse.

[0038] (S22) Calculation of intensity variation map: Select multiple frames of images within the same group (denoted as...). ,in The number of frames included in the statistics. Represents an image sequence. Indicates the first position in the sequence Zhang Image, then This indicates taking the first to N images from the sequence (grayscale or a single channel of RGB) and calculating the average image. : ; Recalculate the difference plot The difference map is then thresholded (e.g., setting difference thresholds for the upper and lower halves, setting differences smaller than the thresholds to zero to mask low-amplitude noise and fixed overlapping areas), resulting in a sparse difference map; subsequently, the variation intensity map is accumulated. : .

[0039] (S23) Maximum Connected Component and Circumscribed Rectangle: Let the binarization threshold be... Binarize the intensity change map (e.g., take...) or Then perform connected component analysis, extract the largest connected component and obtain its bounding rectangle. As the clipping boundary, The coordinates of the left and right boundaries, The coordinates are the upper and lower boundaries. To accommodate situations where there are differences in the upper and lower halves of the layout, the image can be divided into two halves by height, and then the connected regions can be analyzed separately before combining the boundaries.

[0040] (S24) Writing cropping marks: To facilitate batch processing and manual reuse, the cropping boundaries can be written into the auxiliary channel of the image to form "cropping marks". One implementation is to set the pixels of the auxiliary channel outside the cropping box to zero, and draw a 1-pixel wide rectangular border in the auxiliary channel as a preset boundary value (e.g., 255). At this time, the image size does not change, and only the parsable cropping marks are carried.

[0041] (S25) Batch cropping submission: Read the image carrying cropping marks, locate the minimum / maximum coordinates of the rectangular border pixels in the auxiliary channel and verify the border closure; if the verification passes, perform actual cropping on the image content and save the cropped image. To avoid loss of edge information due to overly tight cropping, a preset number of pixels (e.g., 2 pixels upwards / leftwards) can be added as padding during cropping.

[0042] (S26) Manual selection and reuse: When the input is single-frame data or there is an error in automatic estimation, the operator can manually select the cropping boundary and reuse the boundary in other sequences or frames with the same layout; it can also provide the ability to clear cropping marks, re-estimate and resubmit cropping.

[0043] S3. For a closed target region, receive the operator's cue point set within the target region and adaptively determine a threshold based on the distribution of neighboring pixels of the cue points. Under the threshold constraint, perform connected component extraction to generate a target annotation mask, including the following steps: (S31) Interactive Input: The operator clicks or drags within the target area to form a set of prompt points / brush trajectories; the system discretizes the trajectory into a set of operation tuples arranged in chronological order, for example... These represent the position, brush size, brush hardness, and whether it is an erase operation, respectively.

[0044] (S32) Local ROI Constraints: Generate a gradient brush template based on the brush size and hardness, and overlay it onto the drawing map. In this context, it is used to limit the range of candidate regions; in one type of implementation, it can be... Normalization Weights and Construction This results in pixels outside the ROI being suppressed in subsequent connectivity extraction.

[0045] (S33) Threshold tolerance and adaptability: The input grayscale image (or single-channel image) is normalized under ROI constraints, and connected component extraction is performed with a fixed range of pixel values ​​at the cue point as a reference (e.g., OpenCV's floodFill, using FLOODFILL_FIXED_RANGE to make the threshold comparison relative to the seed point value). The threshold tolerance and smoothing parameter can be adjusted by the operator. Since the connected component extraction is based on the seed point pixel, it presents a relatively adaptive region growing effect in different brightness regions.

[0046] (S34) Smoothing: Gaussian smoothing is performed on the image before connectivity extraction to reduce the impact of speckle noise. One implementation is to construct an odd kernel size based on smoothness and set the variance to be related to smoothness, thereby achieving adjustable smoothing intensity.

[0047] (S35) Mask Update and History: Connected components from the connectivity extraction output are written to the target mask fill_map; during an erasure operation, the corresponding connected components are removed from the fill_map. The system can record a historyQue and a snapshot of the fill_map / drawing_map after each update, forming a history of undoable / redo steps to support iterative annotation correction.

[0048] (S36) Liquid edge / hole patching thick line: When gaps appear in the extraction of connected regions in narrow channels, local occlusions or noise interference, the "edge / hole patching thick line" tool can be provided: The operator draws a strip of thick line mask of a certain width along the gap (e.g., fixed line width of 5 pixels), and merges the thick line mask with the target mask of the closed region (e.g., taking the maximum value per pixel or logical OR); In order to make the connection of the thick line to the gap more robust, the thick line mask can also be Gaussian smoothed before merging with the target mask, thereby realizing edge patching and / or hole patching.

[0049] S4. For semi-open boundary targets, receive sparse cue points provided by the operator and automatically generate boundary line masks by fitting adjacent cue points along the texture based on magnetic outlines, including the following steps: (S41) Input and preprocessing: Receive the sparse cue point sequence given by the operator; perform Gaussian smoothing on the input image to obtain a smooth image, so as to reduce the impact of speckle noise on the boundary search.

[0050] (S42) Step and Normal Candidate: For any two adjacent cue points The centerline point is obtained by sampling along the connection direction at a preset step size. For each centerline point, construct its normal direction and generate a set of candidate locations within a local range of the normal. .

[0051] (S43) Cost function and greedy search: candidate positions Calculate the cost function The location with the minimum cost is selected as the boundary point. One type of cost function may include an intensity difference term and a center constraint term, for example: .

[0052] in For the pixel intensity at the candidate location, For reference strength (which can be the interpolation of the starting point strength or the starting and ending point strength), Centerline point The weights are centered and constrained. In practice, a locally greedy approach involving left and right movement along the normal direction can be adopted: comparing the cost of the current position with its left and right neighboring positions, iterating until a local minimum is reached, thus obtaining the normal offset sequence; where... The allowable offset envelope range can be adjusted by the "Magnetic Strength" parameter.

[0053] (S44) Smoothing and drawing: Perform one-dimensional Gaussian smoothing on the obtained normal offset sequence, and shrink the offset amplitude near the endpoints to reduce endpoint jitter; then add the center line points and offsets to obtain the boundary point sequence and draw it as a boundary line mask.

[0054] (S45) Bone surface representation and processing techniques of “boundary line + logical region”: In one type of implementation, the semi-open boundary class annotation such as bone surface adopts the representation method of “boundary line + logical region”: only the bone surface boundary line is saved during storage, and the area below the boundary line is defined as the bone tissue area; before optical flow interpolation, smooth fusion and other processing, the boundary line can be thickened and extended downward to enhance detectability. After processing, the first super-threshold pixel is extracted by column to restore it as the single pixel upper boundary line.

[0055] For ray-like structures such as puncture needle paths, ray marking can be completed at two points: the first point is the ray emission point. The second point is a point on the ray. The system determines the direction of the ray based on two points. It also generates ray annotation results (e.g., extending across the entire image). Only two points of information are stored during saving, and the ray is reconstructed from these two points during retrieval, thus reducing interaction and storage overhead.

[0056] S5. Based on the correlation between adjacent frames in the sequence, perform cross-frame multiplexing and / or propagation of the cue point set and / or annotation results to generate cue point sets and / or annotation results for subsequent frames, including the following steps: (S51) Applicable Prerequisites and Verification: Select consecutive frame segments from the same sequence, wherein the segment includes at least three frames, and the first and last frames are anchor frames that have been annotated. The system can perform sequence consistency verification (e.g., same directory, same sequence identifier and consecutive frame numbers) and anchor constraint verification (both the first and last frames are annotated) to reduce the probability of misoperation; wherein, the sequence identifier can be a field agreed upon by the project and the specific generation method is not limited.

[0057] (S52) Two-end propagation: Assume that consecutive frame segments are numbered 0 to 1 in chronological order. (common (frames), where 0 is the first frame anchor point, For the last frame anchor point, For intermediate frame index; for any intermediate frame Calculate its relationship with the first frame respectively. Last frame The dense displacement field (optical flow) or deformation field between the anchor frames is used to propagate the annotation mask of the anchor frame to the intermediate frame through displacement field deformation, thus obtaining the propagation results from both ends. and The calculation of dense displacement fields can be performed using traditional optical flow or deep learning optical flow networks; this invention does not limit the specific algorithm.

[0058] (S53) Difference Attenuation: To suppress propagation errors caused by occlusion, noise, or inconsistent deformation, the propagation mask can be attenuated based on the difference between the deformed image of the anchor frame and the intermediate frame image. One implementation is: Let Indicates will The displacement field deformation calculated by the double-end propagation in step (S52) is resampled to Let the coordinate system be The scaling factor (optional) then the difference plot for , And make a propagation mask ; The propagation results from the last frame are similar.

[0059] (S54) Smoothing and Blending: Let Let be the normalized position of the intermediate frame in the segment. Let be the Gaussian smoothing operator, and let For the smoothing scale parameter (which can vary) If the changes are balanced to account for the uncertainties at both ends, then the propagation results at both ends can be smoothed and weighted and fused according to their time positions. This weighted fusion result (frame sequence) is denoted as... ,in Represents the first of the sequence Frame. For example, , Then on Thresholding yields the target mask for the intermediate frame.

[0060] (S55) Thickening of semi-open boundaries and extraction of upper boundaries: For semi-open boundary annotations such as bone surfaces, it is preferable to thicken and extend the boundary lines downwards before propagation and smoothing to enhance detectability; after fusion, extract the first super-threshold pixel by column to restore it as a single pixel upper boundary line.

[0061] (S56) Propagation of a ray at two points: For a ray at two points, the two points can be propagated by sub-pixel deformation according to the displacement field, and the coordinates of the two points can be interpolated according to their relative positions after propagation at both ends; when saving, only the information of the two points is still stored.

[0062] To reduce repetitive work between adjacent frames, the sequence of interactive operations in the first frame can be recorded and replayed in subsequent frames to quickly generate initial annotations.

[0063] (1) Operation sequence recording: Record the brush operation tuple sequence of the closed area and the necessary tool parameters (threshold tolerance, smoothing parameters, etc.) as a replayable operation sequence, and maintain the undoable / redo step history.

[0064] (2) Replay of adjacent frames: When switching to the second frame, the operation sequence of the first frame is used as the "previous frame history" cache; when the operator triggers replay, the operation sequence is injected into the queue to be processed in the original order, and the mouse operation of the previous frame is replayed in the frame according to the connectivity extraction logic of the intelligent delineation module of the closed area target, so as to generate the initial mask of the second frame.

[0065] (3) Conflict handling strategy: When it is detected that there is already a labeling result in the second frame, it can be processed according to the preset strategies such as overwrite, append, skip, apply only when empty or ask for confirmation; among them, the overwrite strategy can clear the target mask of the second frame first and then perform replay, the append strategy can perform union of the replay result and the original mask, and the skip strategy does not perform replay.

[0066] S6. Write the cropped image sequence and corresponding annotation results into an image file, and append a lossless compressed annotation data segment to the end of the image file to restore and update the annotation mask without affecting general image reading. This includes the following steps: (S61) Data layout of the appended segment: The implemented appended segment can be written in the following order: lossless compressed mask byte sequence, compressed byte sequence length (e.g., 8-byte unsigned integer), each dimension of the mask array shape (e.g., writing 8-byte unsigned integers dimension by dimension), number of shape dimensions, mask array dtype string, dtype string length, and append identifier (e.g., preset string END_OF_ZIP). An example of its structure is shown below. Figure 2 As shown.

[0067] (S62) Decoupling shape / dtype from resolution: The shape and dtype information of the labeled mask array is used to decouple the mask resolution from the medical image resolution: The mask resolution can be determined by the sampling accuracy of the interactive drawing interface, without having to be consistent with the image size, thereby preserving finer annotations without changing the image resolution.

[0068] (S63) Overwrite update: When a tail identifier is detected at the end of the image file, the starting position of the old annotation data segment can be located back from the tail metadata and the old segment can be truncated. Then, a new annotation data segment can be added to support overwrite update. This allows the image content to be read without parsing the annotation data segment and the annotation mask array can be restored without loss when needed.

[0069] To facilitate dataset construction and sharing, an export function is provided. The export is implemented as follows: the operator selects the target directory for export and generates a list of files to be exported based on filtering conditions such as "all / bookmarks / selected"; then, byte-level copying (e.g., copying file by file) is performed on the source files to ensure that the "tail appended annotation segment" in the single file structure is not destroyed.

[0070] In one alternative implementation, a trained segmentation model can be integrated into the method of this invention to assist in automatic labeling and allow for interactive correction by the operator.

[0071] (1) Temporal context input: The input of the segmentation model may include the current frame and a preset number of adjacent frames before the current frame, so as to improve the temporal consistency by utilizing short-term temporal context; in engineering implementation, adjacent frames can be collected forward by the sequence identifier and frame number continuity in the file name. This invention does not limit the specific collection method.

[0072] (2) Initial annotation generation: The segmentation model outputs an initial annotation probability map (which may contain multiple channels, such as closed region targets, semi-open boundary targets, ray targets, etc.), and then obtains an initial binary mask through threshold and smoothing parameters.

[0073] (3) Interactive local acceptance / rejection: The operator can perform local acceptance or rejection updates on the initial mask; one implementation is to perform flood filling on the connected component where the clicked point is located and mark the connected component as acceptance or rejection state, thereby achieving fast correction.

[0074] (4) Write back: Write the corrected annotation results back to the image file in the "single file append" manner for subsequent review and training; and optionally add the correction results to the training set for offline retraining or incremental model update, thus forming an iterative process of "automatic labeling - manual correction - model update".

[0075] Taking ultrasound as an example, a semi-automatic bootstrapping method for medical image sequence labeling includes the following steps: (1) Import and organization: Export the multi-frame DICOM as a lossless sequence file and name it according to "frame number + sequence identifier"; establish an index structure of "sequence - frame number - tag type". Set seed / interval / offset to perform reproducible sampling, for example, extract one frame every few frames as the first batch of labeled frames, and add keyframes to bookmarks.

[0076] (2) Cropping: Automatic estimation of the imaging window is performed for each sequence to obtain the cropping boundary and write it into the auxiliary channel to form a cropping mark; the operator checks several frames and manually selects and corrects any errors found; then the cropping is submitted in batches and the cropped images are saved.

[0077] (3) Delineation and edge patching of closed areas: Open the first frame of the cropped sequence, click / drag to draw a prompt point / brush trajectory in the target liquid area, adjust the threshold tolerance and smoothing parameters to generate a liquid mask; use the edge patching / hole patching thick line tool to draw a strip mask for the gaps and merge it with the liquid mask; save to get a single file image with the tail annotation segment.

[0078] (4) Replay of adjacent frames: Switch to the next frame and trigger the replay of the previous frame to obtain the initial mask; if the changes between frames are small, only a small amount of correction is needed to complete the annotation; if the changes are large, the outline is redrawn or the edges are locally filled.

[0079] (5) Bone surface and needle path: For semi-open boundaries such as bone surface, add sparse prompt points and enable magnetic tracing to automatically generate boundary lines; for ray structures such as needle path, click two points to generate ray annotations and save only two point information.

[0080] (6) Optical flow interpolation (optional): Select consecutive frame segments in the same sequence and ensure that the first and last frames have been labeled. Perform optical flow interpolation propagation to generate intermediate frame labels. Quickly review and locally correct the interpolation results.

[0081] (7) Export: Export files by conditions such as "bookmark / select / all". Since the export uses file copying, the exported result still maintains the single file structure of "image + appended annotation segment", which can be directly used for verification, training or sharing.

[0082] (6) Optical flow interpolation (optional): Select consecutive frame segments in the same sequence and ensure that the first and last frames have been labeled. Perform optical flow interpolation propagation to generate intermediate frame labels. Quickly review and locally correct the interpolation results.

[0083] (7) Export: Export files by conditions such as "bookmark / select / all". Since the export uses file copying, the exported result still maintains the single file structure of "image + appended annotation segment", which can be directly used for verification, training or sharing.

[0084] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A semi-automatic bootstrapping method for labeling medical image sequences, characterized in that, Includes the following steps: S1. Obtain the medical image sequence data to be labeled and establish a sequence index structure; S2. Determine the effective imaging window region based on medical image sequence data and crop the medical image sequence data to obtain the cropped imaging window image sequence. S3. For closed area targets, based on the set of prompt points in the operator's target area and the distribution of neighboring pixels of the prompt points, an adaptive threshold is determined, and connected component extraction is performed under the threshold constraint to generate target annotation mask. S4. For semi-open boundary targets, receive sparse cue points given by the operator and automatically generate boundary line masks by matching adjacent cue points along the texture based on magnetic delineation. S5. Based on the correlation between adjacent frames in the sequence, perform cross-frame multiplexing and / or propagation of the cue point set and / or annotation results to generate cue point sets and / or annotation results for subsequent frames; S6. Write the cropped image sequence and the corresponding annotation results into an image file, and append a lossless compressed annotation data segment to the end of the image file to restore and update the annotation mask without affecting general image reading.

2. The semi-automatic bootstrap medical image sequence labeling method according to claim 1, characterized in that, In step S1, the establishment of the sequence index structure includes: establishing a hierarchical management structure according to sequence-frame number-tag type; wherein, the tag type may include closed area targets, semi-open boundary targets, ray targets and other custom tags.

3. The semi-automatic bootstrap medical image sequence labeling method according to claim 1, characterized in that, Step S2 includes: in a multi-frame sequence scenario, obtaining a change intensity map by statistically analyzing the pixel changes between frames, and extracting the bounding rectangle of the largest connected region from the change intensity map as the cropping boundary; writing the cropping boundary into the auxiliary channel of the image to form a cropping mark, and then reading the cropping mark to perform batch cropping on the image; when there is an error in the automatic estimation or the input is single-frame data, the operator can manually select the cropping boundary and reuse the historical cropping boundary.

4. The semi-automatic bootstrap medical image sequence labeling method according to claim 1, characterized in that, Step S3 further includes: receiving the narrow edge-filling line mask drawn by the operator along the target boundary, and merging the narrow edge-filling line mask with the target annotation mask extracted from the connected region to achieve edge filling and / or hole filling in the gap region.

5. The semi-automatic bootstrap medical image sequence labeling method according to claim 1, characterized in that, Step S4 specifically includes: advancing along the connection direction between adjacent prompt points at a set step size, and searching for candidate positions within a local range along the vertical direction during the advancement process; calculating a cost function for the candidate positions, the cost function including at least a pixel intensity difference term and a center constraint term; selecting the candidate position with the minimum cost to form a fitting path, and smoothing the fitting path to generate the boundary line mask.

6. The semi-automatic bootstrap medical image sequence labeling method according to claim 1, characterized in that, For ray-like structures of puncture needle paths, ray annotation is completed through two points: the ray direction is determined based on the two points and the ray annotation result is generated. When saving, only the two point information is stored, and when reading, the ray is reconstructed from the two point information, thereby reducing interaction and storage overhead.

7. The semi-automatic bootstrap medical image sequence labeling method according to claim 1, characterized in that, Step S5 includes the following steps: Optical flow-based inter-frame tweening propagation: Select consecutive frame segments from the same sequence, each segment comprising at least three frames, where the first and last frames are labeled anchor frames; for each intermediate frame, calculate the dense displacement field between the intermediate frame and the first and last frames, and propagate the label mask and / or cue points of the anchor frames to the intermediate frames according to the deformation of the dense displacement field; attenuate the propagation mask based on the difference between the deformed image of the anchor frames and the image of the intermediate frames; then, perform weighted fusion of the propagation results at both ends according to the relative position of the intermediate frames in the sequence to obtain the intermediate frame label mask; wherein, the propagation mask for semi-open boundary targets is thickened before deformation propagation to reduce the impact of smoothing on the detectability of the boundary lines; and after weighted fusion, extract the first super-threshold pixel by column to output a single-pixel boundary line mask.

8. A semi-automatic bootstrapping method for labeling medical image sequences according to claim 7, characterized in that, Step S5 includes: recording the operator's interaction sequence on the first frame, replaying the interaction sequence on the second frame to generate the annotation result of the second frame; and when it is detected that there is already an annotation result in the second frame, performing overwrite, append, or skip processing on the annotation result of the second frame. The interactive operation sequence includes a sequence of prompt point coordinates and their order information.

9. A semi-automatic bootstrapping method for labeling medical image sequences according to claim 1, characterized in that, Step S6 includes: writing the medical image into an image file in a preset image format, and then appending a labeling data segment to the end of the image file. The labeling data segment includes: a compressed byte sequence obtained by lossless compression of the labeling mask array, the length information of the compressed byte sequence, the shape information of the labeling mask array, the data type information, and a preset tail identifier. The shape information of the labeling mask array is used to decouple the resolution of the labeling mask array from the resolution of the medical image to adapt to the labeling mask array determined by the interactive rendering accuracy. When the tail identifier is detected to already exist at the end of the image file, the existing labeling data segment is truncated before a new labeling data segment is appended, so that the image content of the image file can be read without parsing the labeling data segment.

10. A computer device, characterized in that, include: A memory and a processor, and a computer program stored in the memory, which, when executed on the processor, implements the method as described in any one of claims 1 to 9.

Images