A method for constructing an index for a medical ultrasound database

Abstract

The application provides an index construction method for a medical ultrasound database, and relates to the technical field of ultrasound data processing, and comprises the following steps: reading a plurality of image sequences from a DICOM ultrasound database, and generating a diagnosis intention vector for each frame; analyzing a diagnosis report, and dividing the report into a plurality of segments; performing lesion detection and segmentation on each frame of image to obtain lesion information, and identifying a unified lesion object by using a cross-frame aggregation strategy; extracting a feature vector for each lesion object to obtain a lesion object feature vector; establishing an alignment mapping of the lesion object and the report segment by using a triple constraint, and generating a lesion report mapping table; and constructing a multi-layer index structure, and storing the lesion report mapping table and the multi-layer index structure in parallel. The application forms a multi-layer index structure comprising a metadata index, a text index, a lesion object vector index and a cross-modal embedding index, so that efficient organization of medical ultrasound data and lesion-level accurate retrieval are realized.

Description

Technical Field

[0001] This invention relates to the field of ultrasound data processing technology, and in particular to an index construction method for medical ultrasound databases. Background Technology

[0002] Ultrasound imaging, a crucial tool for clinical diagnosis, generates data stored in DICOM format, typically containing dozens to hundreds of consecutive image sequences along with corresponding diagnostic reports. Current Picture Archiving and Communication Systems (PACS) primarily rely on Study UIDs or Series UIDs for indexing, returning all image data as a unit for the entire examination during retrieval. This coarse-grained retrieval method forces physicians to browse numerous images frame-by-frame when searching for specific lesions, resulting in low efficiency. Furthermore, existing indexing methods mainly rely on global image feature extraction, failing to effectively distinguish lesion areas from background tissue. In ultrasound images, lesions often occupy only a small portion of the image, while background tissue and speckled noise occupy the majority of the area. This makes global feature-based retrieval susceptible to background interference, leading to insufficient retrieval accuracy.

[0003] During ultrasound examinations, the doctor's actions contain crucial diagnostic intent information, such as using calipers to mark lesion dimensions, freezing specific frames for observation, enabling color Doppler to observe blood flow, and switching to elastography to assess stiffness. This operational information is stored in the overlay and metadata of the DICOM file, reflecting the doctor's varying levels of attention to different frames. However, existing retrieval systems do not fully utilize these diagnostic behavioral signals and cannot distinguish between key frames that doctors focus on and ordinary scan frames. Furthermore, ultrasound diagnosis emphasizes multi-view comprehensive judgment; doctors scan the same lesion from different angles, resulting in significant differences in the image appearance of the same lesion from different perspectives, making it difficult for existing indexing methods to identify them as belonging to the same lesion. In addition, diagnostic reports and images are stored separately, lacking fine-grained semantic connections between them. The system cannot accurately establish a mapping relationship between report descriptions and specific image frames, making accurate cross-modal retrieval difficult.

[0004] Patent application CN116825266A discloses a diagnostic report generation system based on medical imaging big data. This system collects existing case data and medical images, classifies them by target examination sites to establish symptom sets and index keywords, delineates lesions and extracts features from newly acquired ultrasound images, and then matches the new images with existing case images to infer diagnostic results. This patented method improves case retrieval efficiency through keyword indexing and utilizes AI models to assist in lesion delineation and disease identification. However, this system primarily focuses on lesion region segmentation and case similarity matching of single images, failing to address the temporal management of multiple consecutive frames in ultrasound examinations, and lacking a fine-grained semantic alignment mechanism between image features and diagnostic report text. Therefore, it struggles to support the needs for refined indexing at the lesion level and cross-modal retrieval. Summary of the Invention

[0005] In view of this, the present invention provides an index construction method for medical ultrasound databases. By parsing the doctor's operation behavior in DICOM data to extract diagnostic intent features, a cross-frame aggregation strategy is used to identify the same lesion in multiple consecutive frames as a unified object. A comprehensive representation of the lesion object is constructed by integrating image features, morphological features, dynamic sign features and diagnostic intent features. A fine-grained alignment mapping between the lesion object and the diagnostic report fragment is established. Finally, a multi-layer index structure is formed, including metadata index, text index, lesion object vector index and cross-modal embedding index, thereby realizing efficient organization of medical ultrasound data and accurate lesion-level retrieval.

[0006] The technical solution of this invention is implemented as follows:

[0007] This invention provides an index construction method for medical ultrasound databases, comprising:

[0008] S1. Read a multi-frame image sequence from the DICOM ultrasound database, parse the overlay layer to extract doctor's operational behavior information, generate a diagnostic intent vector for each frame to represent the doctor's attention to that frame; parse the diagnostic report and divide the report into several segments;

[0009] S2. Perform lesion detection and segmentation on each frame of the image to obtain lesion information, including lesion mask, lesion type label, lesion sign attributes and lesion bounding box. Use a cross-frame aggregation strategy to identify the same lesion in multiple consecutive frames as a unified lesion object based on the lesion information.

[0010] S3. Extract feature vectors for each lesion object, use convolutional neural network to extract global image features, perform radial sampling with the centroid of the lesion mask as the pole and generate radial ring signature through Fourier transform, use temporal coding network to extract dynamic sign features, map the diagnostic intent vector to diagnostic intent features, and concatenate global image features, radial ring signature, dynamic sign features and diagnostic intent features to obtain lesion object feature vector.

[0011] S4. Establish an alignment mapping between lesion objects and report fragments using temporal consistency constraints, protocol consistency constraints, and semantic similarity constraints, and generate a lesion report mapping table as an auxiliary data structure;

[0012] S5. Construct a multi-level index structure, including a metadata index layer, a text index layer, a lesion object vector index layer, and a cross-modal embedding index layer. The lesion object vector index layer adopts an inverted index structure, storing the lesion report mapping table in parallel with the multi-level index structure.

[0013] Preferably, in step S1, the diagnostic intent vector is an 8-dimensional vector. ,in, Indicates whether a measuring caliper is present. This represents the normalized value of the number of calipers used for measurement. Indicates whether there is a color-coded border. This indicates the proportion of the area of ​​the colored border to the total image area. Indicate whether there are any textual annotations. Indicates whether the frame is frozen. Indicates whether a mode switch has occurred. This represents the normalized value of the relative position of the frame in the sequence.

[0014] Preferably, in step S2, the cross-frame aggregation strategy includes:

[0015] Establish aggregation criteria, including temporal continuity, spatial overlap, consistency of symptoms, and diagnostic significance. Figure 1 To the point of being compatible;

[0016] Construct a connection graph between frames. If two frames simultaneously satisfy all four criteria, then connect them with an edge. The edge weight is determined by the combined score of the four criteria.

[0017] A graph-based connected component extraction algorithm is used to extract connected components in the graph, and each connected component corresponds to a lesion object;

[0018] Among them, the temporal continuity requirement is that the timestamp interval between two frames is less than a threshold. Spatial overlap requires a crossover ratio of two lesion masks. Greater than the threshold , and These are lesion masks in two frames; consistency of signs requires that the lesion type labels and lesion sign attributes in the two frames be the same or similar; diagnostic significance... Figure 1 Consistency requires that the similarity between the diagnostic intent vectors of two frames be higher than a threshold.

[0019] Preferably, in step S3, the method for generating the radial ring signature is as follows:

[0020] Center of gravity of lesion mask As the pole, along Equally spaced angular directions Radial sampling is performed, where k = 0, 1, ..., K-1; sampling is performed from the centroid outwards along this direction, and the texture intensity sequence is recorded. ,in , The maximum radius of the lesion; normalize the radial length to the interval. The normalized sampling function is obtained. ,in Constructing a two-dimensional feature matrix Perform a Fourier transform on the angular dimension and take the low-frequency component as the radial ring signature of the lesion. .

[0021] Preferably, in step S3, the method for extracting dynamic feature characteristics is as follows:

[0022] For lesions containing color Doppler ultrasound, the time series of blood flow signals is extracted, and the sum of the color pixel intensities or the blood flow area of ​​the lesion region in each frame is calculated to obtain the time series. ;

[0023] For lesions that include elastography, extract the time series of the mean elasticity values ​​of the lesion region. ;

[0024] The time series is normalized and resampled to a fixed length using interpolation. Then, a Dynamic Time Warping (DTW) distance or a temporal coding network is used to map the time series into a fixed-dimensional embedding vector. ;

[0025] If the lesion does not contain a dynamic imaging segment, then the dynamic signs and characteristics... Let it be the zero vector.

[0026] Preferably, the process of constructing the lesion object vector index layer includes:

[0027] A mask weight vector is constructed from the feature vector of the lesion object. The mask weight vector is used to characterize the difference in importance of the image regions corresponding to different dimensions of the feature vector.

[0028] Using an inverted index and product quantization structure, mask-weighted distance is used for calculation during clustering and quantization. The mask-weighted distance is defined as follows:

[0029] ;

[0030] in, Represents the feature vectors of two lesion objects; for and Mask weighted distance; Let be the mask weights for the k-th dimension features; and for and The k-th dimension component; d is the total dimension of the feature vector.

[0031] Preferably, the process of constructing the mask weight vector is as follows:

[0032] For the global image features, trace back to the convolutional feature map from which they originate and compute the feature map at each spatial location. Corresponding receptive field area Coverage of internal lesion masking: ;in, Masking lesions in pixels The value at; The total number of pixels in the receptive field is given; spatial weights are mapped to various dimensions of the feature vector through global average pooling or flattening operations to obtain the mask weight vector of the global image feature part. ;

[0033] For the radial ring signature, dynamic sign features, and diagnostic intent features, since they do not have a dimension-wise spatial correspondence, a uniform weight is set to a constant. ;

[0034] The complete mask weight vector is obtained by concatenating the weights of each component. ,in Represents an n-dimensional vector of all 1s; , , These are the dimensions of radial circular signature, dynamic sign features, and diagnostic intent features, respectively.

[0035] Preferably, the temporal consistency constraint requires that the description order of the report segments be consistent with the temporal order of image acquisition. This constraint evaluates the overall alignment scheme. The global timing relationship, the timing consistency score is defined as:

[0036] ;

[0037] in This is an indicator function that takes the value 1 when the condition is met, and 0 otherwise. This indicates a report excerpt. The corresponding lesion object; This is the timestamp of the lesion. This represents the total number of reported segments; and For the index of the report segment and ; The score represents the proportion of all reported segment pairs that meet the time-consistent criteria.

[0038] The protocol consistency constraint requires that the anatomical location described in the report fragment be consistent with the scanning procedure to which the lesion belongs. Analyze the anatomical locations it describes For the lesion object The scanning steps match the time period of occurrence and image features to the standard scanning protocol template. Define the protocol consistency score:

[0039] ;

[0040] If the location described in the fragment matches the scanning step where the lesion is located, the score is 1; otherwise, it is 0.

[0041] Semantic similarity constraints require that the textual semantics of the report fragment match the image features of the lesion object, and that the report fragment be... Encoded as text feature vectors , to the lesion object Projecting the feature vectors into the cross-modal embedding space yields Calculate the cosine similarity:

[0042] ;

[0043] in, Represents the dot product of vectors; This represents the Euclidean norm of a vector.

[0044] Preferably, the optimization objective of the alignment mapping is to maximize the total alignment score, which is defined as:

[0045] ;

[0046] in These are the weighting coefficients;

[0047] The optimization objective is expressed as:

[0048] ;

[0049] That is, to find the alignment mapping that maximizes the total score. The constraints include that each report segment corresponds to at least one lesion object, one lesion object can correspond to multiple report segments, and the correspondence must satisfy the condition that the temporal order does not overlap. The optimal alignment is solved using dynamic programming or Viterbi algorithm.

[0050] Preferably, the multi-layered index structure includes a metadata index layer, a text index layer, a lesion object vector index layer, and a cross-modal embedding index layer;

[0051] The metadata indexing layer creates B-tree indexes and hash indexes for the metadata fields of Study, Series, and Patient;

[0052] The text indexing layer builds an inverted index on the text content of the report fragments and a vector index on the text feature vectors of the report fragments;

[0053] The lesion object vector index layer adopts a mask-weighted inverted index and a product quantization structure;

[0054] The cross-modal embedding index layer projects image feature vectors and text feature vectors into a unified embedding space to build a hybrid vector index, and distinguishes the data type of the entries by the type field;

[0055] The lesion report mapping table provides a two-way association query between lesion objects and report fragments.

[0056] The present invention has the following advantages over the prior art:

[0057] (1) This invention achieves a transformation from traditional study-level coarse-grained retrieval to lesion-level refined retrieval by constructing a multi-layered index structure for medical ultrasound databases. This method supports multimodal retrieval, including image retrieval of images, text retrieval of text, image retrieval of text, and text retrieval of images, improving the organization efficiency and retrieval accuracy of medical ultrasound data and providing more accurate case references for clinical diagnosis;

[0058] (2) This invention extracts physician operational behavior information by parsing the overlay layer of DICOM ultrasound data and generates a diagnostic intent vector, which effectively represents the physician's attention to each frame. Compared with traditional image content-based analysis methods, the diagnostic intent vector can distinguish between key frames that physicians focus on and ordinary scan frames, making index construction more in line with the real needs of clinical diagnosis;

[0059] (3) This invention adopts the principles of temporal continuity, spatial overlap, consistency of symptoms, and diagnostic significance. Figure 1The cross-frame aggregation strategy based on the four consistency criteria identifies the same lesion in multiple consecutive frames as a unified lesion object. This method solves the problem of the same lesion appearing multiple times due to repeated scanning from different angles by doctors in ultrasound diagnosis, and avoids indexing different view frames of the same lesion as independent objects, thereby improving the accuracy and completeness of the retrieval results and enabling the system to return complementary view reference images of the same lesion.

[0060] (4) This invention proposes a method for radial sampling using the centroid of the lesion mask as the pole and generating a radial ring signature through Fourier transform. Compared with traditional morphological feature extraction methods based on boundary point sequences or moment descriptors, the radial ring signature obtains robustness to rotation, small-amplitude translation and scale changes by performing Fourier transform on the angular dimension and taking the low-frequency component;

[0061] (5) This invention targets dynamic imaging modes such as color Doppler and elastography, extracts the time series of blood flow signals and elastic values, and generates dynamic feature characteristics using dynamic time warping or temporal coding networks. This method extends traditional static image features to temporal dynamic features, enabling the indexing system to support retrieval needs based on dynamic features;

[0062] (6) This invention proposes a mask-weighted approximate vector indexing method, which introduces lesion mask weights during the clustering and quantization process of the inverted file and the product quantization structure. This method reduces the interference of background tissue and speckle noise in ultrasound images on retrieval accuracy and improves the accuracy of lesion-level retrieval;

[0063] (7) This invention establishes a triple mechanism of temporal consistency constraints, protocol consistency constraints, and semantic similarity constraints, and uses dynamic programming or the Viterbi algorithm to optimize the alignment mapping between lesion objects and report fragments. The synergistic effect of the triple constraints achieves fine-grained alignment between lesion objects and report fragments, avoids the high cost of manual annotation, and provides accurate semantic mapping relationships for cross-modal retrieval. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a flowchart of the method of the present invention;

[0066] Figure 2 This is a schematic diagram illustrating the diagnostic intent vector generation of the present invention;

[0067] Figure 3This is a schematic diagram of cross-frame aggregation according to the present invention;

[0068] Figure 4 This is a schematic diagram of the radial ring signature generation of the present invention;

[0069] Figure 5 This is a schematic diagram of the alignment mapping optimization process of the present invention;

[0070] Figure 6 This is a schematic diagram of the mask weight vector construction of the present invention. Detailed Implementation

[0071] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0072] like Figure 1 As shown, this invention provides an index construction method for medical ultrasound databases, including:

[0073] S1. Read a multi-frame image sequence from the DICOM ultrasound database, parse the overlay layer to extract doctor's operational behavior information, generate a diagnostic intent vector for each frame to represent the doctor's attention to that frame; parse the diagnostic report and divide the report into several segments;

[0074] S2. Perform lesion detection and segmentation on each frame of the image to obtain lesion information, including lesion mask, lesion type label, lesion sign attributes and lesion bounding box. Use a cross-frame aggregation strategy to identify the same lesion in multiple consecutive frames as a unified lesion object based on the lesion information.

[0075] S3. Extract feature vectors for each lesion object, use convolutional neural network to extract global image features, perform radial sampling with the centroid of the lesion mask as the pole and generate radial ring signature through Fourier transform, use temporal coding network to extract dynamic sign features, map the diagnostic intent vector to diagnostic intent features, and concatenate global image features, radial ring signature, dynamic sign features and diagnostic intent features to obtain lesion object feature vector.

[0076] S4. Establish an alignment mapping between lesion objects and report fragments using temporal consistency constraints, protocol consistency constraints, and semantic similarity constraints, and generate a lesion report mapping table as an auxiliary data structure;

[0077] S5. Construct a multi-level index structure, including a metadata index layer, a text index layer, a lesion object vector index layer, and a cross-modal embedding index layer. The lesion object vector index layer adopts an inverted index structure, storing the lesion report mapping table in parallel with the multi-level index structure.

[0078] In one embodiment of the present invention, step S1 includes:

[0079] Each ultrasound examination corresponds to a DICOM file or Series, containing dozens to hundreds of images acquired during continuous scanning. The image data is analyzed frame by frame, extracting the pixel data, timestamp, imaging mode label, and scanning parameters for each frame. The timestamp is used for subsequent temporal aggregation of lesions and image-text alignment; the imaging mode label distinguishes between B-mode ultrasound grayscale, color Doppler, and elastography modes; and the scanning parameters include technical parameters affecting image appearance such as probe frequency, gain, and depth.

[0080] The Overlay layer and Structured Report of the DICOM file are parsed to extract information about the doctor's actions. The Overlay layer stores measurement annotations, color-coded borders, and text annotations drawn by the doctor on the image. Measurement annotations include the position and length of calipers used to mark the size of lesions. Color-coded borders are rectangular or polygonal areas used to select areas of interest. Text annotations are text descriptions added by the doctor to the image. Freeze and save events are recorded through metadata fields, indicating that the frame was frozen and saved as a keyframe by the doctor. Mode switching events are identified by changes in the imaging mode label of adjacent frames, such as switching from B-mode grayscale to color Doppler mode;

[0081] like Figure 2 As shown, a diagnostic intent vector is generated for each frame. This is an 8-dimensional fixed-dimensional vector. Wherein, This indicates whether a measuring caliper is present; if the frame contains at least one measuring caliper, then... ,otherwise . This represents the normalized value of the number of measuring calipers. The normalized value is obtained by dividing the number of measuring calipers in the frame by the maximum number of measurements in a single frame in the entire sequence. This indicates whether a color-coded border exists; if the frame contains a color-coded border, then... ,otherwise . This represents the proportion of the area of ​​the color frame to the total area of ​​the image. It is calculated by dividing the total number of pixels in the color frame by the total number of pixels in the image. This indicates whether there are any text annotations. If the frame contains text annotations, then... ,otherwise . This indicates whether the frame is a frozen save frame. If the frame is marked as frozen save, then... ,otherwise . This indicates whether a mode switch has occurred; if the imaging mode of this frame differs from the previous frame, then... ,otherwise . This represents the normalized value of the relative position of the frame in the sequence. It is calculated by dividing the frame number by the total number of frames in the sequence, and its value ranges from [value missing]. Frames containing measurements or annotations have a higher diagnostic intent weight, indicating that the physician has paid special attention to that frame;

[0082] The diagnostic report is parsed and segmented into several segments based on a structured template or a semantic segmentation algorithm. The structured template is suitable for reports following standard formats, separating segments by recognizing paragraph headings or serial numbers. The semantic segmentation algorithm is suitable for reports in free-text format, dividing segments by recognizing sentence boundaries and semantic units; each segment describes an independent lesion or sign. Keywords and medical entities are extracted from each segment, and a pre-trained medical text encoding model, such as Bio BERT or PubMed BERT, is used to map the segment into a fixed-dimensional text feature vector, typically with dimensions of 768 or 1024.

[0083] In one embodiment of the present invention, step S2 includes:

[0084] For each frame of the image, a lesion detection and segmentation model is used to identify lesion regions within the image. This model is based on deep learning architectures such as Mask R-CNN or U-Net and is trained on a medical ultrasound dataset. The model output includes a lesion mask. This is a binary image, when pixels The value is 1 when the area belongs to the lesion region and 0 when it belongs to the background region. Lesion type label. This indicates the classification of lesions, such as solid nodules, cystic nodules, mixed nodules, calcifications, etc. Lesion characteristics. This is a set of attributes, including clinical descriptive dimensions such as echo type, boundary clarity, morphological regularity, and internal blood flow signal. (Lexicectomy bounding box) This is the smallest rectangular bounding box that encloses the lesion. In a single ultrasound scan, multiple different lesions may be detected; for example, a thyroid scan might find one nodule in the left lobe and another in the right lobe, while a liver scan might reveal multiple cysts or hemangiomas.

[0085] like Figure 3As shown, cross-frame aggregation is performed on lesions in ultrasound sequences. The purpose of cross-frame aggregation is to identify the same lesion in multiple consecutive frames as the same lesion object. Because doctors scan the same lesion from different angles, the same lesion will appear repeatedly in multiple consecutive frames. The cross-frame aggregation strategy is based on four criteria: temporal continuity, spatial overlap, consistency of signs, and diagnostic significance. Figure 1 To the point of being compatible;

[0086] Temporal continuity requires that the time stamp interval between two frames be less than a threshold. If frame and frame The timestamps are respectively and ,satisfy The two frames then satisfy temporal continuity. Threshold The typical time interval is 2 to 5 seconds, depending on the ultrasound scan speed setting. Temporal continuity ensures that the aggregated frames come from the same continuous scan segment; if the time interval is too long, it indicates that the doctor has moved the probe and may have observed lesions in different areas.

[0087] Spatial overlap requires the cross-joint ratio of two lesion masks. Greater than the threshold The intersection-union ratio (IU) measures the degree of overlap between two masks, and its value ranges from [value range missing]. A higher value indicates a higher degree of overlap. Threshold Typical values ​​are 0.3 to 0.5. Spatial overlap ensures that the aggregated structures are the same physical entities that are close in location, rather than different lesions that are far apart.

[0088] Consistency of signs requires lesion type labels in two frames. and Same or similar, lesion signs and attributes and The similarity is higher than a threshold. For example, the same lesion should appear as either a solid nodule or a cystic lesion in different frames. If one frame shows a cyst and another shows a solid nodule, then they are clearly not the same lesion. The similarity of sign attributes is measured by calculating the cosine similarity or Hamming distance of the attribute vectors.

[0089] Diagnostic intention Figure 1 Consistency requires diagnostic intent vectors for two frames. and The similarity is higher than the threshold. Doctors will highlight and measure key lesions, and these operations reflect a consistent diagnostic intent pattern across multiple frames. The similarity of diagnostic intent vectors is calculated using cosine similarity, with a typical threshold value of 0.7.

[0090] Construct a connection graph between frames. Nodes in the graph represent frames where lesions are detected. If two frames simultaneously satisfy the four criteria mentioned above, an edge is connected between them. The edge weight is determined by the combined score of the four criteria: temporal continuity score, spatial overlap score, sign consistency score, and diagnostic significance score. Figure 1 The weighted sum of consistency scores. A graph-based connected component extraction algorithm is used to extract connected components in the graph, with each connected component corresponding to a lesion object. Connected component extraction can be implemented using depth-first search or breadth-first search algorithms;

[0091] Lesion object The record contains a set of frames. This indicates in which frames the lesion appears. Viewpoint label set It identifies the scanning perspective by image orientation or probe angle, such as cross-sections, longitudinal sections, and oblique sections. Representative frame. Typically, the frame with the highest diagnostic intent weight is selected, i.e., the frame with the largest weighted sum of the components of the diagnostic intent vector. This frame usually contains the most measurement annotations and doctor attention signals. Mask set This includes the mask of the lesion object in each frame. Lesion type and sign attributes. The uniform type and attributes are determined from all frames contained in the lesion object through voting or averaging. After cross-frame aggregation, all frames in the database are organized into several lesion objects, each representing an independent clinical finding and becoming the basic unit for subsequent index construction.

[0092] This cross-frame aggregation strategy achieves a unified representation of the same lesion under multi-view scanning, avoiding indexing different view frames of the same lesion as independent objects, and improving the accuracy and completeness of the retrieval results.

[0093] In one embodiment of the present invention, step S3 includes:

[0094] For each lesion object, multiple types of feature representations are extracted, namely static morphological features, dynamic sign features, and diagnostic intent features, which are finally combined into a complete lesion object feature vector.

[0095] Static morphological characteristics It consists of two parts. The first part is global image features. Feature extraction is performed on representative frames of the lesion using deep convolutional neural networks such as ResNet, DenseNet, or EfficientNet. The feature map from the last convolutional layer of the network is taken; this feature map has a spatial structure and its size is typically [size missing]. ,in and The height and width of the feature map, This represents the number of channels. Each spatial location of the feature map. The receptive field corresponds to a region of the original image. The feature map is obtained by performing global average pooling or flattening. 3D feature vector Record the feature map location and receptive field information corresponding to each dimension k of the feature vector;

[0096] like Figure 4 As shown, the second part is a radial circular signature. The center of gravity of the lesion mask. As the pole, along K equally spaced angular directions Radial sampling is performed, where k = 0, 1, ..., K-1. Typical values ​​for K are 64 or 128. Sampling is performed along this direction from the centroid outwards, recording the texture intensity sequence. ,in , The maximum radius of the lesion is calculated as the maximum distance from the centroid to the lesion boundary. During sampling, the grayscale value of one pixel is taken at fixed intervals along the radial direction; the number of sampling points is typically 32 to 64. The radial length is then normalized to an interval... The normalized sampling function is obtained. ,in The normalization method is as follows: Constructing a two-dimensional feature matrix The rows of the matrix correspond to the normalized radial positions. Column corresponding angle The matrix elements are the sampled texture intensity values. A Fourier transform is performed on the angular dimension to obtain the frequency domain representation. The low-frequency component is taken as the radial ring signature of the lesion. The method for selecting low-frequency components is to preserve the first part of the spectrum. There are coefficients, among which Typical values ​​are 8 to 16. Radial ring signature flattening is... dimensional vector, Number of radial sampling points; it should be noted that Figure 4 The text uses a red gradient to represent texture intensity values, with dark red → light red indicating a transition from high to low intensity.

[0097] This radial ring signature is robust to rotation, small-amplitude translation, and scale changes. Rotation invariance comes from the Fourier transform of the angular dimension, translation robustness from normalized sampling with the centroid as the pole, and scale invariance from the normalization of the radial length. This feature can uniformly represent the same lesion from different viewpoints.

[0098] Static morphological characteristics Global image features With radial ring signature splicing, that is , dimension ;

[0099] Dynamic signs and characteristics Extraction of lesions involving dynamic imaging. Dynamic imaging refers to imaging modes with time-series changes, such as color Doppler and elastography, while conventional B-mode ultrasound grayscale ultrasound is a static imaging technique.

[0100] For the color Doppler segment, the time series of blood flow signals is extracted. Color coding in the color Doppler image represents the direction and velocity of blood flow; red indicates blood flow towards the probe, and blue indicates blood flow away from the probe. The sum of color pixel intensities or blood flow area of ​​the lesion region in each frame is calculated to obtain the time series. The blood flow area is calculated by counting the number of colored pixels within the lesion mask;

[0101] For the elastography segment, the time series of the mean elasticity value of the lesion region is extracted. Elastography uses color coding to represent tissue stiffness, with soft tissue typically displayed in red and hard tissue in blue. The average elastic value within the lesion mask is calculated as the elastic feature of that frame.

[0102] The time series is normalized by subtracting the mean and dividing by the standard deviation. The series is then resampled to a fixed length using interpolation, typically 32 to 64 time points. Dynamic Time Warping (DTW) distance or temporal coding networks such as LSTM or Transformer are used to map the time series into fixed-dimensional embedding vectors. Dynamic time warping methods calculate the distance between time series and embed the distance matrix into Euclidean space through multidimensional scaling analysis. Temporal coding network methods use the last hidden state of an LSTM or Transformer as the embedded representation of the time series. Embedding vector dimension... Typical values ​​range from 128 to 256. If the lesion does not contain a dynamic imaging segment, then the dynamic signs are... Let it be the zero vector;

[0103] Diagnostic intent characteristics Diagnostic intent vector of the frame represented by the lesion object The mapping is obtained by mapping the 8-dimensional diagnostic intent vector. Mapped to via the fully connected layer 3D feature vector The fully connected layer contains one hidden layer with 64 neurons and the ReLU activation function. The output layer has [number of neurons]. . Typical values ​​range from 64 to 128. This feature encodes the physician's level of attention to the lesion and their operational behavior patterns;

[0104] Lesion object The complete feature vector consists of three concatenated parts, namely The total dimension is ;

[0105] After the above steps, each lesion object in the database obtains a feature vector of fixed dimensions. This vector comprehensively encodes the static morphology, dynamic signs, and diagnostic intent of the lesion.

[0106] like Figure 5 As shown, in one embodiment of the present invention, step S4 includes:

[0107] A semantic correspondence between lesion objects and report fragments is established using a triple-constraint alignment method based on temporal consistency, protocol consistency, and semantic similarity. This method defines a general scoring function to evaluate the matching quality between any report fragment and any lesion object, and then uses an optimization algorithm to find the alignment mapping that maximizes the total score.

[0108] Standard scanning protocol templates were established for different types of ultrasound examinations. For example, the standard protocol for thyroid ultrasound includes transverse scan of the left thyroid gland, longitudinal scan of the left thyroid gland, isthmus scan, transverse scan of the right thyroid gland, longitudinal scan of the right thyroid gland, and cervical lymph node scan. Each step is associated with a predicted frame range and anatomical landmark features. The standard protocol for breast ultrasound includes the upper outer quadrant, lower outer quadrant, upper inner quadrant, lower inner quadrant, and nipple region of the left breast, and the corresponding region of the right breast. The protocol templates were established by analyzing a large number of standard examinations and stored as configuration files. The structure of diagnostic reports was analyzed to identify the descriptive patterns they follow. For example, thyroid reports are typically described in the order of left lobe, isthmus, right lobe, and lymph nodes.

[0109] Report fragment sequence Arranged in the order they appear in the report, among which Total number of reported fragments. Lesion object sequence. Sort by the timestamp of the first appearance of the lesion, among which This represents the total number of lesion objects. Define the alignment mapping. This indicates a report excerpt. Corresponding lesion object The mapping It is the function to be optimized, and the optimization goal is to find the mapping that maximizes the total score among all possible alignment schemes;

[0110] The temporal consistency constraint requires that the description order of the reported segments be consistent with the temporal order of image acquisition. This constraint evaluates the overall alignment scheme. The global timing relationship. The timing consistency score is defined as:

[0111] ;

[0112] in This is an indicator function that takes the value 1 when the condition is met, and 0 otherwise. This indicates a report excerpt. The corresponding lesion object; This is the timestamp of the lesion. This represents the total number of reported segments; and For the index of the report segment and ; The score represents the proportion of time-consistent segment pairs among all reported segment pairs, and its value ranges from [value missing]. This constraint encourages the earlier parts of the report to correspond to earlier lesions, and the later parts to correspond to later lesions; for example, if the report has three segments... Describe in order, alignment scheme Map them to lesion objects respectively If the timestamps of these three lesion objects satisfy Then all fragments , and The timing relationships are all consistent, and the timing consistency score is . If the timestamp order is Then only fragment pairs and Meeting timing consistency, score is ;

[0113] The protocol consistency constraint requires that the anatomical location described in the report fragment be consistent with the scanning procedure to which the lesion object belongs. For report fragments... Analyze the anatomical locations it describes It identifies lesions through keyword matching. For example, it identifies keywords such as "left lobe," "right lobe," and "isthmus." The scanning steps match the time period of occurrence and image features to the standard scanning protocol template. Define the protocol consistency score:

[0114] ;

[0115] A score of 1 is given if the location described in the segment matches the scan step in which the lesion is located; otherwise, a score of 0 is given. This constraint leverages the standardized procedures followed by ultrasound examinations to prevent mismatches between descriptions of the left lobe and lesions in the right lobe; for example, if the reported segment... The description is "hypoechoic nodule at the lower pole of the left thyroid gland". If the lesion object Scanning steps ,but If the lesion object Scanning steps ,but This is a general scoring function that can calculate the protocol consistency score between any report fragment and any lesion object;

[0116] Semantic similarity constraints require that the textual semantics of the report fragment match the image features of the lesion object. The report fragment... Encoded as text feature vectors We use a pre-trained text encoding model from the medical field to represent lesion objects. eigenvectors Projecting onto the cross-modal embedding space yields The cross-modal embedding space is trained through contrastive learning, ensuring that semantically related images and text are close in the embedding space. Projection is implemented through fully connected layers. Cosine similarity is calculated.

[0117] ;

[0118] in, Represents the dot product of vectors; This represents the Euclidean norm of the vectors. The similarity values ​​range from... , usually A higher similarity score indicates a better semantic match between the text and the image; for example, if the report fragment... The text feature vector describes "hypoechoic solid nodules with indistinct boundaries". In the embedded space, lesions with similar symptoms Image feature vectors If they are close, then The value is relatively high, for example, 0.85. This is in contrast to a cystic lesion. similarity It might only be 0.32. This is also a general scoring function that can calculate the semantic similarity between any report fragment and any lesion object;

[0119] The overall optimization process consists of the following steps. The first step is to define a general scoring function and a protocol consistency function. and semantic similarity function It can calculate any report fragment. With any lesion object The scores between them. The second step is to define the alignment map. This mapping maps each report fragment to a lesion object. The third step calculates the specific alignment scheme. The total score, defined as the total alignment score:

[0120] ;

[0121] in The weighting coefficients are: The three components are the average of the temporal consistency score, the average of the protocol consistency score, and the average of the semantic similarity score, all normalized to . Range. The weighting coefficients are adjusted according to the actual data distribution; a typical value is... Note the total score. It is an alignment mapping The functions, different Corresponding to different total scores. For protocol consistency and semantic similarity, a general scoring function... and Alignment mapping is applied to the total score. Determined matching pairs The local scores of all report segments are averaged. The temporal consistency constraint... Directly evaluate the overall alignment scheme To determine the global temporal relationship, it is necessary to examine the temporal order between all segments;

[0122] The fourth step is to find the optimal alignment mapping, with the optimization objective being:

[0123] ;

[0124] That is, to find the alignment mapping that maximizes the total score. ;

[0125] The constraints include: each report segment must correspond to at least one lesion object. One lesion object can correspond to multiple report segments, and the correspondence must satisfy the condition of no temporal overlap. Temporal non-overlap means that if segments... and satisfy ,and Corresponding lesion , Corresponding lesion ,but and The time period cannot appear Prior to The ending situation;

[0126] The optimal alignment is solved using dynamic programming. A state transition graph is constructed, where state nodes represent matching pairs between report fragments and lesion objects. The edges represent the alignment choices between adjacent segments. Starting from the initial state, the states are gradually expanded, accumulating scores until the final state is reached. The Viterbi algorithm is used to find the path with the maximum total score, which corresponds to the optimal alignment scheme. ;

[0127] Generate a lesion report mapping table and store triples. , indicating the lesion object With report excerpt Correspondingly, the confidence level is The confidence score is calculated as the local score of the matching pair in the optimal alignment. A bidirectional index is built to find the corresponding report fragment from the lesion object and the corresponding lesion object from the report fragment. This mapping table is generated during the index building phase and stored alongside the subsequent multi-level indexes as an auxiliary data structure.

[0128] This triple-constraint alignment method achieves fine-grained semantic mapping between lesion objects and report fragments, avoiding the high cost of manual annotation and providing accurate semantic associations for cross-modal retrieval.

[0129] In one embodiment of the present invention, step S5 includes:

[0130] Construct a multi-layered index structure, including a metadata index layer, a text index layer, a lesion object vector index layer, and a cross-modal embedding index layer;

[0131] The metadata index layer indexes the metadata fields for Study, Series, and Patient. B-tree indexes are created for fields such as Patient ID, Examination Date, Examination Type, Anatomical Location, and Diagnosis to support range queries. Hash indexes are created for unique identifier fields such as Study UID to support exact matches. The metadata index is implemented using a relational database or key-value store.

[0132] The text index layer builds an inverted index on the text content of the report fragments. It performs word segmentation and extracts medical terms and keywords, using medical dictionaries such as UMLS or MeSH to normalize synonyms. An inverted list of terms to fragment IDs is built to support keyword retrieval. Simultaneously, a vector index, such as an HNSW index, is built on the text feature vectors of the report fragments to support semantic similarity retrieval. The HNSW index is based on a hierarchical, navigable small-world graph structure, supporting efficient approximate nearest neighbor queries.

[0133] The lesion object vector index layer stores the feature vectors of lesion objects and is constructed using a mask-weighted inverted index and the product quantization IVF-PQ method.

[0134] like Figure 6 As shown, the process of constructing the mask weight vector is as follows:

[0135] For global image feature parts This involves tracing the convolutional feature map back to its source. Each spatial location of the feature map... Receptive field area corresponding to the original image Calculate the mask coverage at this location:

[0136] ;

[0137] in Masking lesions in pixels The value at this location is either 0 or 1; This represents the total number of pixels in the receptive field. If the CNN features are obtained through global average pooling, then the k-th dimension feature... The weight of the k-th channel of the corresponding feature map is the average of the mask coverage at all spatial locations.

[0138] ;

[0139] in Let represent the height and width of the feature map. If the CNN features are obtained by flattening, then establish a mapping relationship between the flattened index and the feature map position, where the k-th dimension feature corresponds to the position... The weight is Obtain the mask weight vector of the CNN feature parts. The weight value range is The feature dimension weights corresponding to the lesion area are close to 1, while the feature dimension weights corresponding to the background area are close to 0.

[0140] For radial ring signatures Dynamic symptoms and characteristics Diagnostic intent characteristics Since they do not have a one-dimensional spatial correspondence, a uniform weight is set as a constant. . Typically, 1 or the average of the CNN weights is used. Construct the complete mask weight vector. ,in Let n be an n-dimensional vector of all 1s, with a total dimension of 1. , and eigenvectors The dimensions are consistent;

[0141] Define the mask weighted distance as:

[0142] ;

[0143] in, Represents the feature vectors of two lesion objects; for and Mask weighted distance; Let be the mask weights for the k-th dimension features; and for and The k-th dimension component; d is the total dimension of the feature vector. This distance formula assigns higher weight to the feature dimension corresponding to the lesion region and lower weight to the feature dimension corresponding to the background region, making the similarity calculation more focused on the morphological differences of the lesion itself;

[0144] The lesion object vector index is constructed using an inverted index and a product quantization (IVF-PQ) structure, introducing mask weights during clustering and quantization. The clustering stage uses a weighted k-means algorithm, and the distance metric is mask-weighted distance. The formula for updating cluster centers is:

[0145] ;

[0146] in For the k-th cluster, This represents element-wise product. After iterative optimization, K cluster centers are obtained. Typical values ​​for K range from 256 to 1024;

[0147] The product quantization stage divides the d-dimensional feature vector into M sub-vectors, each sub-vector having a dimension of 1. Typical values ​​for M are 8 to 16. A codebook is generated by clustering each subspace, using mask-weighted distance during clustering as well. The codebook is based on the average of the subspace mask weights. ,in Indicates the first Each subspace contains a set of feature dimension indices, and the codebook size is adaptively adjusted. A larger codebook, such as 256 cluster centers, is used for subspaces with higher weights to improve accuracy, while a smaller codebook, such as 64 cluster centers, is used for subspaces with lower weights to save storage.

[0148] The storage stage is for each cluster center Construct an inverted index. The inverted index stores the lesion object ID, quantization code, sparse representation of the mask weight vector, and corresponding Study and Series information for each cluster. The sparse representation of the mask weight vector only stores dimensions where the weights significantly deviate from the mean, reducing storage overhead.

[0149] For key lesions with high diagnostic intent weights, in addition to storing the master quantization code, the quantization residuals are also stored. ,in This is the quantized and reconstructed feature vector. The residuals are stored after secondary coarse-grained quantization. During retrieval, the coarse screening stage uses master quantization encoding for fast approximate calculation, and the fine ranking stage uses residuals to reconstruct more accurate feature vectors for the top candidates, and calculates mask-weighted distances for re-ranking.

[0150] The cross-modal embedding index layer projects image and text feature vectors into a unified embedding space. A cross-modal embedding model is trained using contrastive learning loss such as InfoNCE to ensure that semantically related images and text are close in the embedding space. A hybrid vector index is built, containing embedding vectors of both lesion objects and report fragments, distinguishing the data type of entries by a type field. The hybrid vector index is implemented using approximate nearest neighbor index structures such as HNSW or Annoy.

[0151] The lesion report mapping table provides a bidirectional association query between lesion objects and report segments, stored as key-value pairs. It supports querying the corresponding report segment ID list based on the lesion object ID, and querying the corresponding lesion object ID list based on the report segment ID.

[0152] The aforementioned multi-layered index structure works together to support multimodal retrieval functionality. The metadata index layer is used for coarse screening and range queries; the text index layer supports keyword retrieval and text similarity retrieval; the lesion object vector index layer is the core index supporting image retrieval of images; the cross-modal embedding index layer supports image retrieval of text and text retrieval of images; and the lesion report mapping table provides related information for cross-modal queries.

[0153] This multi-layered indexing structure expands the search capabilities from a single retrieval mode to a multi-modal retrieval mode, improving the system's adaptability to different clinical scenarios. The mask-weighted indexing method reduces the interference of background tissue and speckle noise in ultrasound images on retrieval accuracy, allowing the search to focus more on the lesion itself.

[0154] In a specific embodiment of the present invention, the index constructed using the above method can realize image retrieval of images, text retrieval of text, image retrieval of text, and text retrieval of images;

[0155] The image retrieval process is as follows:

[0156] The doctor provides a single image frame or selects a Region of Interest (ROI), extracts features from the input image to perform lesion detection and segmentation, and generates a lesion mask. and eigenvectors The feature extraction process is consistent with the index construction process, including optional static features, dynamic sign features, and diagnostic intent features. Based on the metadata of the query image, such as examination type and anatomical location, a coarse screening is performed through the metadata index layer to limit the candidate range. For example, searching only among lesion objects in thyroid ultrasound, in the lesion object vector index layer, a mask-weighted distance is used for approximate nearest neighbor retrieval. First, the query vector is calculated. The mask-weighted distance to each cluster center is used to select the nearest clusters, such as the Top-10 clusters. Then, quantization encoding is used to quickly calculate the approximate distance in the inverted list of these clusters to obtain the Top-N candidate lesion objects. The stored residual encoding is used to reconstruct a more accurate feature vector for the Top-N candidates. The mask-weighted distance with the query vector is calculated for fine ranking to obtain the final Top-K similar lesion objects. For each retrieved lesion object, all frames it contains, especially frames from different perspectives, are obtained to provide similar images from the same perspective and reference images from complementary perspectives. The corresponding report fragment summary, diagnostic conclusion, patient information and other auxiliary information are obtained through the lesion report mapping table and returned to the doctor for diagnostic reference.

[0157] The text retrieval process is as follows:

[0158] When a doctor inputs a text description, such as "hypoechoic nodule with unclear boundaries," the query text is segmented and medical entity extracted to generate a keyword list. Keyword matching is then performed in the inverted index of the text index layer to obtain a set of candidate report fragments. The query text is then encoded into a text feature vector. The system performs vector retrieval in the report fragment vector index to find the most semantically similar fragments. It then performs a mixed sorting based on keyword matching scores and vector similarity scores, returning the Top-K similar report fragments along with their corresponding complete reports and case information.

[0159] The process of retrieving text from an image is as follows:

[0160] The doctor provides a query image or ROI, and extracts lesion features from the image to generate a feature vector of the lesion object. Projecting the feature vectors onto the cross-modal unified embedding space yields... The system retrieves text-only results in the cross-modal unified embedding index layer, obtaining report fragments that are semantically closest to the query image. Simultaneously, it finds similar lesion objects through the lesion object vector index layer, and then searches for report fragments associated with these lesion objects through the lesion report mapping table. The system sorts the results based on cross-modal similarity, alignment confidence, and protocol consistency, and returns the Top-K related report fragments and diagnostic descriptions.

[0161] The process of retrieving images from text is as follows:

[0162] The doctor inputs a text description, and the query text is encoded into a text feature vector. The results are projected into a cross-modal unified embedding space. In the cross-modal unified embedding index layer, results that only return lesion object types are retrieved to obtain lesion objects that match the semantics of the query text. At the same time, matching report fragments are found in the text index layer. The lesion objects associated with these report fragments are found through the lesion report mapping table. The candidate results from multiple sources are sorted and typical frames containing diagnostic intent signals such as measurement annotations are returned first. The Top-K related lesion images, their metadata, and report summaries are returned.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing an index for a medical ultrasound database, characterized by, Comprise: S1, read a plurality of image sequences from a DICOM ultrasound database, parse Overlay superposition layer to extract doctor operation behavior information, generate diagnosis intention vector for each frame to represent the doctor's attention to the frame; parse the diagnosis report, and cut the report into several segments; S2, lesion detection and segmentation are performed on each frame to obtain lesion information, including lesion mask, lesion type label, lesion sign attribute and lesion bounding box, and a cross-frame aggregation strategy is used to identify the same lesion in continuous multiple frames as a unified lesion object according to the lesion information; S3, a feature vector is extracted for each lesion object, a convolutional neural network is used to extract global image features, a radial sampling is performed with the lesion mask center as the pole point, and a radial annular signature is generated through Fourier transform, a time encoding network is used to extract dynamic sign features, a diagnosis intention vector is mapped to a diagnosis intention feature, and the global image features, the radial annular signature, the dynamic sign features and the diagnosis intention feature are spliced to obtain a lesion object feature vector; S4, a time consistency constraint, a protocol consistency constraint and a semantic similarity constraint are used to establish the alignment mapping of the lesion object and the report segment, and a lesion report mapping table is generated as an auxiliary data structure; The temporal consistency constraint requires that the order of the description of the shots is consistent with the temporal order of the image acquisition, this constraint evaluates the global alignment scheme of the global temporal relationship, the temporal consistency score is defined as: ； wherein is an indicator function that takes value 1 when the condition is satisfied and 0 otherwise; denotes a report segment corresponding lesion object; is the timestamp of the lesion object; is the total number of report segments; and is the index of the report segment and ; is the total number of all report segment pairs, and the score indicates the proportion of the segment pairs that satisfy the temporal consistency among all segment pairs. The protocol consistency constraint requires that the anatomical site described by the report fragment is consistent with the scan step to which the lesion object belongs, for report fragments , parsing the anatomical site it describes , for lesion objects , matching to the scan step of the standard scan protocol template according to the time period and image features it appears in , defining a protocol consistency score: ； If the part described in the segment is consistent with the scanning step where the lesion is located, the score is 1, otherwise 0; The semantic similarity constraint requires the text semantics of the report fragment to match the image features of the lesion object, and the report fragment is encoded into a text feature vector The feature vector of the lesion object is projected into the cross-modal embedding space to obtain The cosine similarity is calculated: ； where • denotes the vector dot product; denotes the Euclidean norm of a vector. S5, a multi-layer index structure is constructed, including a metadata index layer, a text index layer, a lesion object vector index layer and a cross-modal embedding index layer, wherein the lesion object vector index layer uses an inverted index structure, and the lesion report mapping table is stored in parallel with the multi-layer index structure.

2. The method of claim 1, wherein, In step S1, the diagnosis intention vector is an 8-dimensional vector wherein, represents whether there is a measurement caliper, represents a quantity normalized value of the measurement caliper, represents whether there is a color frame marking, represents a proportion of the color frame area to the image area, represents whether there is a text annotation, represents whether it is a frozen save frame, represents whether a mode switch occurs, represents a relative position normalized value of the frame in the sequence.

3. The method of claim 1, wherein the medical ultrasound database is constructed by indexing. In step S2, the cross-frame aggregation strategy comprises: Set aggregation criteria, including time continuity, spatial overlap, sign consistency and diagnosis intention consistency; Construct a connection graph between frames, and if two frames meet all four criteria, connect an edge, and the edge weight is determined by the comprehensive score of the four criteria; A graph-based connected component extraction algorithm is used to extract connected components in the graph, and each connected component corresponds to a lesion object; wherein the time continuity requires the timestamp interval of the two frames to be less than a threshold ; the spatial overlap requires the intersection over union of the two frames of lesion masks to be greater than a threshold , and are the lesion masks of the two frames respectively; the sign consistency requires the lesion type labels and the lesion sign attributes of the two frames to be the same or similar; the diagnosis intention consistency requires the similarity of the diagnosis intention vectors of the two frames to be higher than a threshold.​ 4. The method of claim 1, wherein the medical ultrasound database is constructed by using a method of claim 3. In step S3, the generation method of the radial annular signature is: Center of gravity of lesion mask As the pole, along Equally spaced angular directions Radial sampling is performed, where k = 0, 1, ..., K-1; sampling is performed from the centroid outwards along this direction, and the texture intensity sequence is recorded. ,in , The maximum radius of the lesion; normalize the radial length to the interval. The normalized sampling function is obtained. ,in Constructing a two-dimensional feature matrix Perform a Fourier transform on the angular dimension and take the low-frequency component as the radial ring signature of the lesion. .

5. The method of claim 1, wherein the medical ultrasound database is constructed by indexing. In step S3, the extraction method of the dynamic sign feature is: For the lesion objects containing color Doppler, the time series of blood flow signals are extracted, the total intensity of color pixels or the blood flow area in the lesion region of each frame is calculated, and the time series is obtained ; For a lesion object containing elastography, extracting a mean elasticity value time series of the lesion region ; normalizing the time series and resampling to a fixed length using interpolation, mapping the time series to a fixed dimension embedding vector using dynamic time warping (DTW) distance or a time series encoding network ; Wherein, if the lesion object does not contain a dynamic imaging segment, the dynamic sign feature is set to a zero vector.

6. The method of claim 1, wherein the medical ultrasound database is constructed by using a method of claim 1. The construction process of the lesion object vector index layer comprises: A mask weight vector is constructed for the lesion object feature vector, and the mask weight vector is used to represent the importance difference of the image area corresponding to different dimensions of the feature vector; An inverted file and a product quantization structure are used, and a mask weighted distance is used for calculation in the clustering and quantization process, and the mask weighted distance is defined as: ； wherein, represents 2 lesion object feature vectors; is and mask weighted distance of ; is the mask weight of the kth dimension feature; and is and the kth dimension component of and d is the total dimension of the feature vector.

7. The method of claim 6, wherein the method further comprises: The construction process of the mask weight vector is: For the global image feature part, the convolution feature map tracing its source, calculate the feature map each spatial position The corresponding receptive field area The coverage of the internal lesion mask: ; wherein, The value of the lesion mask at the pixel ; The total number of pixels of the receptive field area; through the global average pooling or flattening operation, the spatial position weight is mapped to each dimension of the feature vector, and the mask weight vector of the global image feature part is obtained ; For the radial annular signature, dynamic sign feature and diagnostic intent feature part, since there is no spatial correspondence dimension by dimension, a uniform weight is set as a constant ; The parts of the weight are spliced to obtain a complete mask weight vector wherein represents an n-dimensional all-1 vector; 、 、 are the dimensions of the radial annular signature, dynamic sign feature and diagnostic intent feature, respectively.

8. The method of claim 1, wherein the medical ultrasound database is constructed by using a method of claim 7. The optimization objective of the alignment mapping is to maximize the total alignment score, and the total alignment score is defined as: ； wherein are weight coefficients; The optimization objective is expressed as: ； finding an alignment mapping that maximizes the total score The constraint conditions include that each report segment corresponds to at least one lesion object, one lesion object can correspond to multiple report segments, and the correspondence relationship satisfies the time sequence non-intersection. The dynamic programming algorithm or Viterbi algorithm is used to solve the optimal alignment.

9. The method of claim 1, wherein the medical ultrasound database is constructed by indexing. The multi-layer index structure comprises a metadata index layer, a text index layer, a lesion object vector index layer and a cross-modal embedding index layer; The metadata index layer establishes B-tree index and hash index for the metadata fields of Study, Series and Patient; The text index layer establishes an inverted index for the text content of the report segment and a vector index for the text feature vector of the report segment; The lesion object vector index layer uses a mask weighted inverted file and a product quantization structure; The cross-modal embedding index layer projects the image feature vector and the text feature vector into a unified embedding space to establish a mixed vector index, and distinguifies the data type of the entry through a type field; The lesion report mapping table provides bidirectional association query between the lesion object and the report segment.

Images