Ureteral calculus positioning method and system based on multi-modal ultrasound

Abstract

The embodiment of the application provides a ureteral calculus positioning method and system based on multi-modal ultrasound, relates to the technical field of medical ultrasound imaging, and first acquires multi-modal ultrasound imaging data covering different scanning angles and various imaging principles; then, based on the anatomical layered structure of the ureter, the data is jointly regionally divided to obtain joint ultrasound region data divided according to the anatomical level; then, the different modal ultrasound signal propagation performances corresponding to each region data are analyzed, and a cross-modal signal feature set related to the calculus is extracted; a dynamic association link of the cross-modal signal feature set and the imaging performance of the calculus is established; finally, the spatial relationship between the cross-modal signal features and the anatomical structure is integrated in combination with the anatomical layered position to generate a precise positioning result of the calculus in the ureter, including information such as the anatomical segment where the calculus is located, the relative position to the lumen wall and the distance relationship with the surrounding key structure, thereby greatly improving the accuracy and reliability of ureteral calculus positioning.

Description

Technical Field

[0001] This application relates to the field of medical ultrasound imaging technology, and more specifically, to a method and system for locating ureteral stones based on multimodal ultrasound. Background Technology

[0002] Ureteral stones are a common and serious condition affecting the health of patients in the diagnosis of urinary system diseases. Accurate localization of ureteral stones plays a crucial role in the formulation of subsequent treatment plans and the evaluation of treatment effectiveness.

[0003] Currently, existing ultrasound imaging techniques have certain limitations in the field of ureteral stone localization. Traditional ultrasound imaging is often based on a single imaging principle, resulting in relatively limited image information that is difficult to comprehensively and accurately reflect the complex situation of the ureter and its stones. Different tissues have different reflectivity to ultrasound signals, and single-modality ultrasound imaging cannot fully utilize these differences to accurately distinguish different anatomical layers of the ureter and the location of the stones. Moreover, when faced with different scanning angles and complex anatomical structures, single-modality imaging cannot effectively integrate information from multiple aspects, thus limiting the accuracy and reliability of stone localization. Furthermore, existing methods lack flexibility in handling signal changes under different imaging conditions, making it difficult to adaptively adjust the localization strategy according to the actual situation, further affecting the effectiveness of ureteral stone localization. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and system for locating ureteral stones based on multimodal ultrasound.

[0005] According to a first aspect of this application, a method for locating ureteral stones based on multimodal ultrasound is provided, the method comprising: The system acquires multimodal ultrasound imaging data based on multiple imaging principles. The multimodal ultrasound imaging data based on multiple imaging principles includes continuous imaging sequences at different scanning angles and ultrasound reflection information corresponding to different modes. The multiple imaging principles cover ultrasound imaging methods such as tissue morphology imaging and blood flow signal imaging. The continuous imaging sequences cover the entire ureter and adjacent organs. Based on the anatomical layering structure of the ureter, multimodal ultrasound imaging data of various imaging principles are combined into regions. According to the differences in the anatomical boundaries of the ureteral wall mucosa, muscle layer, adventitia and lumen, tissue identification related signal performance of different modal ultrasound data is fused to obtain combined ultrasound region data divided by anatomical level. The propagation behavior of different modal ultrasound signals corresponding to the combined ultrasound regions was analyzed, including the propagation speed, reflection path and attenuation state of different modal signals in different tissue layers, and the cross-modal signal feature set related to the presence of stones was extracted. A dynamic correlation link between cross-modal signal feature set and stone imaging performance is established. The dynamic correlation link is constructed by the correspondence between cross-modal signal features and stone morphology, internal echo performance, and posterior acoustic shadow performance. The dynamic correlation link adaptively adjusts the correlation logic as imaging conditions change. By combining the anatomical layer locations corresponding to the data from each joint ultrasound region through dynamic correlation links, the cross-modal signal feature distribution status and the spatial relationship of anatomical structures are integrated to generate the location result of the stone in the ureter. The location result includes the anatomical segment where the stone is located, its relative position to the lumen wall, and its distance relationship with surrounding key structures.

[0006] According to a second aspect of this application, a ureteral stone localization system based on multimodal ultrasound is provided. The ureteral stone localization system based on multimodal ultrasound includes a processor and a readable storage medium. The readable storage medium stores a program that, when executed by the processor, implements the aforementioned ureteral stone localization method based on multimodal ultrasound.

[0007] Based on any of the above aspects, by acquiring multimodal ultrasound imaging data using various imaging principles, covering continuous imaging sequences at different scanning angles and ultrasound reflection information corresponding to different modalities, and considering multiple imaging principles including tissue morphology imaging and blood flow signal imaging, information about the ureter and its surrounding areas can be comprehensively acquired from multiple dimensions. Based on the anatomical layering structure of the ureter, the multimodal ultrasound imaging data is divided into joint regions, and the tissue identification-related signal manifestations of different modal ultrasound data are integrated to obtain joint ultrasound region data divided by anatomical level. This makes the data more structured and targeted, more accurately reflecting the tissue characteristics of each layer of the ureter, and analyzing the different modal ultrasound signal propagation manifestations corresponding to each joint ultrasound region data. By extracting cross-modal signal feature sets related to the presence of stones, key information from different modal signals is effectively integrated, improving the ability to identify stone features. A dynamic correlation link between the cross-modal signal feature set and the imaging performance of stones is established, and the correlation logic is adaptively adjusted according to changes in imaging conditions, enhancing the adaptability and flexibility of the method under different imaging environments. This ensures that the accuracy of localization is not affected by fluctuations in imaging conditions. Finally, by combining the dynamic correlation link with the anatomical layer location, the distribution state of cross-modal signal features and the spatial relationship of anatomical structures are integrated to generate localization results containing detailed information such as the anatomical segment where the stone is located, its relative position to the luminal wall, and its distance relationship with surrounding key structures. This greatly improves the accuracy and reliability of ureteral stone localization. Attached Figure Description

[0008] Figure 1 A flowchart illustrating the method for locating ureteral stones based on multimodal ultrasound provided in this application is shown. Figure 2A schematic diagram of the component structure of the ureteral stone localization system based on multimodal ultrasound provided in this application embodiment is shown. Detailed Implementation

[0009] Figure 1 A flowchart illustrating the method for locating ureteral stones based on multimodal ultrasound provided in this application is shown, and the detailed steps are described below.

[0010] Step S110: Acquire multimodal ultrasound imaging data based on multiple imaging principles. The multimodal ultrasound imaging data based on multiple imaging principles includes continuous imaging sequences at different scanning angles and ultrasound reflection information corresponding to different modes. The multiple imaging principles cover ultrasound imaging methods such as tissue morphology imaging and blood flow signal imaging. The continuous imaging sequences cover the entire ureter and adjacent organs.

[0011] In this embodiment, the tissue morphology imaging data comes from a two-dimensional grayscale ultrasound device. The imaging sequence is a continuous scan along the ureter from the renal pelvis junction to the bladder inlet. Each scanning angle corresponds to a sequence containing Z-frame images. The pixel format of a single frame image is grayscale 8-bit, and each pixel records the intensity value of the ultrasound reflection signal. The data source is the raw data output interface of the ultrasound device. During output, the patient's individual identification information has been desensitized using a differential privacy algorithm, retaining only the pixel intensity and scanning parameters related to the imaging. The blood flow signal imaging data comes from a color Doppler ultrasound device. The scanning angle and frame rate of the imaging sequence are completely matched with those of the tissue morphology imaging sequence. A single frame of data includes a grayscale background layer and a blood flow signal layer. The blood flow signal layer encodes the direction and velocity information of blood flow using pixel values. The data format is RGB three-channel, where the red and green channels encode the blood flow direction, and the blue channel encodes the blood flow velocity amplitude. The two types of data mentioned above are aligned at the frame level through the time synchronization module of the ultrasound equipment, ensuring that the tissue morphology and blood flow signal data correspond one-to-one at the same time point and the same scanning angle. The total number of frames in the continuous imaging sequence is M groups of scanning angles multiplied by Z frames of each group, covering the mucosa, muscle layer, adventitia layer and surrounding blood vessels, kidneys, bladder and other adjacent organ structures along the entire length of the ureter.

[0012] Step S120: Based on the anatomical layering structure of the ureter, multimodal ultrasound imaging data of various imaging principles are combined into regions. According to the differences in the anatomical boundaries of the ureteral wall mucosa, muscle layer, adventitia and lumen, the tissue identification related signal manifestations of different modal ultrasound data are fused to obtain combined ultrasound region data divided according to anatomical level.

[0013] In this embodiment, the joint region division is based on the four-layer anatomical structure of the ureteral wall as the core reference, corresponding to the lumen core region, mucosa, muscle layer, and adventitia. The fusion of different modal data is achieved based on the differences in ultrasound signal characteristics of each tissue layer. Among them, tissue morphology imaging data is used to identify the grayscale boundaries of the tissue, and blood flow signal imaging data is used to distinguish the blood-rich mucosa layer from the blood-sparse muscle layer and adventitia layer. The final output joint ultrasound region data includes the pixel range, signal feature set, and modal association index of each layer.

[0014] Step S121: Obtain the hierarchical classification criteria for the anatomical layered structure of the ureter. The hierarchical classification criteria are determined based on the differences in the tissue composition of the ureteral wall in human anatomy, the physiological thickness range of each layer of tissue, and the spatial morphological characteristics of the lumen. The hierarchical classification criteria cover the division of the mucosa, submucosa, muscularis propria, adventitia, and the core region of the lumen.

[0015] In this embodiment, the hierarchical division standard references the ureteral wall structure parameters commonly used in human anatomy. The lumen core region is defined as a hollow area within the ureter without tissue filling; the mucosa is defined as the epithelial tissue layer covering the inner wall of the lumen; the submucosa is defined as the connective tissue layer between the mucosa and the muscle layer; the muscle layer is defined as the muscular tissue layer containing circular and longitudinal muscles; and the adventitia is defined as the fibrous connective tissue layer surrounding the muscle layer. The thickness range of each layer is determined based on anatomical statistics of the adult population. The spatial morphological characteristics of the lumen core region are defined as a continuous hollow channel running along the ureter, with a cross-section that is roughly circular or elliptical. The division criteria also include supplementary parameters such as the intensity range of ultrasound reflection signals and the probability of blood flow signals in each tissue layer. These parameters are all stored as structured mixed text and numerical data for subsequent threshold determination in region division.

[0016] Step S122: Perform imaging region enhancement processing on the multimodal ultrasound imaging data of each imaging principle. Highlight the differences in ultrasound reflection signals of tissues at different anatomical levels by amplifying the ultrasound signal amplitude. Reduce the interference of background noise on tissue boundaries by spatial domain filtering to obtain enhanced multimodal ultrasound imaging data.

[0017] In this embodiment, appropriate enhancement strategies are adopted for tissue morphology imaging data and blood flow signal imaging data respectively. The tissue morphology imaging data aims to highlight grayscale boundaries, while the blood flow signal imaging data aims to highlight the contrast between blood flow signals and the background. The enhancement process includes six sub-steps: amplitude adjustment, noise suppression, frequency enhancement, edge enhancement, smoothing, and parameter optimization. The final output enhanced data maintains the same pixel size and frame rate as the original data, only adjusting the signal amplitude, contrast, and noise level.

[0018] For example, step S1221: Obtain the original signal amplitude range of multimodal ultrasound imaging data for each imaging principle, and determine the maximum, minimum and distribution range of the signal amplitude by analyzing the original data output by the ultrasound imaging device.

[0019] In this embodiment, for tissue morphology imaging data, all pixel grayscale values ​​of a single frame image are traversed, and the maximum, minimum, and most frequently occurring grayscale intervals are statistically analyzed. These grayscale intervals correspond to the signal intensity of the background tissue. For blood flow signal imaging data, the pixel values ​​of the red, green, and blue channels are traversed, and the maximum, minimum, and distribution intervals of each channel are statistically analyzed. The high-value intervals of the blue channel correspond to high-speed blood flow signals, while the low-value intervals correspond to background noise. The above statistical results are stored in array form, and the array elements contain six fields: modality type, channel identifier, maximum value, minimum value, start value of the distribution interval, and end value of the distribution interval.

[0020] Step S1222: The adaptive amplitude adjustment algorithm is used to optimize the signal amplitude of the multimodal ultrasound imaging data. Based on the signal intensity distribution in different regions, the amplitude gain is adjusted in a targeted manner to increase the amplitude in weak signal regions and maintain the amplitude in strong signal regions, thus obtaining the amplitude-adjusted signal.

[0021] In this embodiment, the adaptive amplitude adjustment algorithm divides the image into non-overlapping N×N pixel blocks based on the local neighborhood signal strength of each pixel. For each pixel block, the average signal strength is calculated. If the average strength is lower than a set weak signal threshold, the amplitude of all pixels within that block is multiplied by a gain coefficient greater than 1. If the average strength is in the medium signal range, the pixel amplitude remains unchanged. If the average strength is higher than a strong signal threshold, the pixel amplitude is multiplied by an attenuation coefficient less than 1 to avoid signal saturation. The gain coefficient and attenuation coefficient are determined by the ratio of the neighborhood signal strength to the global average strength. A smaller ratio results in a larger gain coefficient, and a larger ratio results in a smaller attenuation coefficient. During the adjustment process, it is ensured that the pixel amplitude does not exceed the maximum dynamic range of the device output.

[0022] Step S1223: Perform noise suppression processing on the amplitude-adjusted signal. Filter random noise in the multimodal ultrasound imaging data through spatial domain filtering to obtain the noise-suppressed ultrasound signal.

[0023] In this embodiment, median filtering is applied to the tissue morphology imaging data with a filter window size of K×K. The grayscale values ​​of the neighboring pixels of each pixel are sorted, and the median value is used to replace the original pixel value, effectively suppressing salt-and-pepper noise. Gaussian filtering is applied to the blood flow signal imaging data with a filter window size of L×L. A weighted average is calculated for the neighboring pixel values ​​of each pixel, with the weights determined based on a Gaussian distribution, where the center pixel has the highest weight and the edge pixels have the lowest weight, effectively suppressing Gaussian random noise. The filtering operation is only applied to the signal layer. The grayscale layer of the tissue morphology imaging data and the red, green, and blue channels of the blood flow signal imaging data are filtered independently. The pixel size of the filtered data is completely consistent with the original data.

[0024] Step S1224: Based on the noise-suppressed ultrasound signal, extract the frequency component information of the noise-suppressed ultrasound signal, decompose the signal into sub-signals of different frequencies using frequency decomposition technology, and analyze the characteristic frequency range corresponding to different anatomical tissue levels.

[0025] In this embodiment, Fast Fourier Transform (FFT) is used to decompose the pixel signals of each frame of the image into frequency components, converting the time-domain pixel intensity signals into frequency component distributions in the frequency domain. The signal frequency ranges corresponding to different anatomical tissue layers are statistically analyzed: the lumen core region corresponds to the high-frequency range, with higher signal frequencies due to the lack of tissue reflection; the mucosa corresponds to the mid-to-high-frequency range, with medium-frequency reflections due to the dense structure of the epithelial tissue; the muscle layer corresponds to the mid-to-low-frequency range, with low-frequency reflections due to the coarse fibrous structure of the muscle tissue; and the adventitia corresponds to the low-frequency range, with the lowest frequency reflections due to the loose structure of the connective tissue. These frequency ranges are stored as numerical intervals as the basis for subsequent frequency enhancement.

[0026] Step S1225: Amplify the sub-signals within the characteristic frequency range to increase the amplitude of the characteristic frequency sub-signals related to each anatomical level tissue, thus obtaining the amplitude-enhanced sub-signals.

[0027] In this embodiment, for tissue morphology imaging data, the amplitude of the characteristic frequency intervals corresponding to each anatomical level in the decomposed frequency domain signal is multiplied by a gain coefficient. The gain coefficient is a value greater than 1, set according to the importance of the layer signal. The gain coefficient for the lumen core region and mucosa is higher than that for the muscle layer and adventitia. For blood flow signal imaging data, amplitude enhancement is performed only on the characteristic frequency intervals corresponding to blood flow velocity in the blue channel. The red and green channels, which only encode blood flow direction, are not frequency enhanced. After amplitude enhancement, the frequency domain signal is converted back to the time domain signal through inverse fast Fourier transform to obtain image data that retains the characteristic frequency components.

[0028] Step S1226: Based on the amplitude-enhanced sub-signals, an edge enhancement algorithm is used to enhance the tissue boundary region of the multimodal ultrasound imaging data. By calculating the signal gradient of adjacent pixels, the signal contrast of the boundary region is enhanced, and the edge-enhanced multimodal ultrasound imaging data is obtained.

[0029] In this embodiment, the tissue morphology imaging data employs the Sobel edge detection algorithm to calculate pixel gradients in both the horizontal and vertical directions. The gradient magnitudes in both directions are then superimposed to obtain the edge intensity value, which is added to the original grayscale image to enhance the grayscale contrast of the boundary region. The blood flow signal imaging data employs the Laplacian edge detection algorithm to calculate the second derivative of pixels, identify the edges of the blood flow signal region, and enhance the blood flow signal amplitude in the edge region, thereby improving the clarity of the boundary between the blood flow region and the background. A gradient threshold is set during the edge enhancement process, retaining only edge regions with gradient values ​​higher than the threshold to avoid noise amplification caused by over-enhancement.

[0030] Step S1227: Perform signal smoothing processing on the edge-enhanced multimodal ultrasound imaging data. Use the neighborhood averaging algorithm to process the sawtooth distortion of the signal generated during the enhancement process to obtain smoothed multimodal ultrasound imaging data.

[0031] In this embodiment, the smoothing process uses a 3×3 pixel neighborhood averaging window. For each pixel in each frame, the average signal value of its 8 neighboring pixels is calculated. The original pixel value and the average signal value are then weighted and summed according to a set ratio to obtain a new pixel value, where the weight of the original pixel value is higher than that of the average signal value. This ensures that edge features are preserved while eliminating jagged distortion generated during enhancement. Tissue morphology imaging data and blood flow signal imaging data undergo independent smoothing processes, and the processed data maintain the original pixel size and frame rate.

[0032] Step S1228: Calculate the signal characteristic distance between tissues at each level and analyze the differences in ultrasound imaging of tissues at different anatomical levels in the smoothed multimodal ultrasound imaging data.

[0033] In this embodiment, the signal feature distance is calculated based on three dimensions: the average gray value of each tissue layer, the signal frequency range, and the blood flow signal amplitude. First, the average gray value, the center value of the characteristic frequency, and the average amplitude of the blood flow signal are extracted from the lumen core region, the mucosa layer, the muscle layer, and the adventitia layer, respectively. Then, the values ​​of each dimension are normalized to eliminate dimensional differences. Finally, the Euclidean distance between any two layers is calculated using the normalized values ​​of the three dimensions to obtain the signal feature distance matrix between each layer. The larger the value of the matrix element, the more significant the imaging difference between the corresponding two layers.

[0034] Step S1229: Adjust the enhancement processing parameters according to the difference performance, adjust the amplitude gain and filter intensity parameters until the imaging differences of different anatomical levels of tissues meet the requirements for subsequent region division settings.

[0035] In this embodiment, a minimum threshold for signal feature distance is set. If the signal feature distance between any two adjacent layers in the matrix is ​​lower than this threshold, the enhancement processing parameters are adjusted accordingly: if the distance between the mucosa and the muscle layer is insufficient, the mid-to-high frequency gain coefficient of the tissue morphology imaging data is increased, while the low-frequency gain coefficient is decreased; if the distance between the muscle layer and the adventitia is insufficient, the filtering intensity of the muscle layer region in the blood flow signal imaging data is increased to eliminate interference from weak blood flow signals within the muscle layer. After each adjustment, steps S1221 to S1228 are repeated until the signal feature distance of all adjacent layers reaches or exceeds the set threshold. At this point, the final enhanced multimodal ultrasound imaging data is output.

[0036] Step S12210: Output enhanced multimodal ultrasound imaging data after parameter adjustment, wherein the enhanced multimodal ultrasound imaging data includes ultrasound imaging features of tissues at different anatomical levels.

[0037] In this embodiment, the enhanced data is stored in DICOM format, retaining the original scanning parameters such as scanning angle, frame rate, and probe frequency, while adding metadata of enhancement processing parameters such as amplitude gain coefficient, filter window size, and edge enhancement threshold. The imaging features of each layer are stored in the private data field of the DICOM file in the form of pixel masks. The pixels corresponding to the layer in the mask are marked as 1, and the remaining pixels are marked as 0.

[0038] Step S123: Based on the enhanced multimodal ultrasound imaging data, extract the tissue boundary signal in each type of enhanced multimodal ultrasound imaging data, and identify the boundary position between different anatomical levels by analyzing the reflection coefficient variation characteristics of ultrasound signals at different tissue interfaces.

[0039] In this embodiment, for tissue morphology imaging data, the grayscale gradient map of each frame is extracted. Pixels with gradient values ​​higher than a set threshold are marked as candidate boundary points. Then, connected component analysis is performed on the candidate boundary points to filter out continuous boundary segments, corresponding to the interfaces of different tissue layers. For blood flow signal imaging data, the amplitude gradient map of the blue channel is extracted. Pixels with gradient values ​​higher than a set threshold are marked as blood flow boundary candidate points. Combined with the boundary segments of tissue morphology imaging, the boundary position between the mucosa and muscle layer is determined. Because the mucosa is rich in capillaries, the blood flow signal is significant, while the blood flow signal in the muscle layer is sparse. The boundary positions are stored in the form of pixel coordinate pairs. Each boundary position includes the start pixel coordinates and the end pixel coordinates, corresponding to continuous boundary segments.

[0040] Step S124: Cross-compare the tissue boundary signals extracted from ultrasound data of different modalities, and mark the boundary position signals whose overlap meets the set standard as joint boundary signals.

[0041] In this embodiment, the boundary segments extracted from tissue morphology imaging data are used as a reference. The boundary segments extracted from blood flow signal imaging data are compared with these boundary segments in terms of coordinates. The ratio of the number of overlapping pixels between the two boundary segments to the total number of pixels is calculated. If this ratio exceeds a set overlap threshold, the boundary segment is marked as a joint boundary signal. The overlap calculation adopts a pixel-by-pixel matching method. If the coordinate difference between a pixel of the blood flow signal boundary segment and a pixel of the tissue morphology boundary segment is within 1 pixel, they are determined to be overlapping pixels. The final output joint boundary signal includes the pixel coordinates of the boundary segments, the overlap value, and the corresponding anatomical level identifier.

[0042] Step S125: Based on the distribution range and amplitude variation of the joint boundary signal, preliminarily divide the common distribution range of each anatomical level in the multimodal ultrasound imaging data of multiple imaging principles, and mark the start and end pixel coordinate intervals of each level.

[0043] In this embodiment, each frame of image is divided into four consecutive pixel intervals based on the junction boundary signal. From the image center outwards, these intervals correspond to the core lumen region, mucosa, muscle layer, and adventitia, respectively. The starting pixel coordinates of each level are the ending coordinates of the inner junction boundary, and the ending pixel coordinates are the starting coordinates of the outer junction boundary. For regions without junction boundaries, such as the connection between the ureter and surrounding organs, supplementary division is performed based on the grayscale values ​​of tissue morphology imaging and the amplitude of blood flow signal imaging to ensure that each pixel is assigned to the corresponding anatomical level. The division results are stored as a layer pixel interval array for each frame of image. Each array element contains five fields: layer identifier, starting x-coordinate, ending x-coordinate, starting y-coordinate, and ending y-coordinate.

[0044] Step S126: Based on the tissue thickness range and spatial relationship in the hierarchical division standard, adjust the common distribution range of each anatomical level in the preliminary division according to the actual hierarchical distribution of the human ureteral anatomy, correct the range offset caused by imaging angle deviation or signal interference, and obtain the adjusted anatomical level distribution range.

[0045] In this embodiment, the thickness range parameters of each layer in the hierarchical division standard are called to convert the pixel range of each initially divided layer into the actual physical thickness. The conversion coefficient is the pixel-to-physical size conversion ratio of the ultrasound device, which is calculated based on the probe frequency and imaging depth. If the actual physical thickness of a certain layer exceeds the upper and lower limits of the standard thickness range, the pixel interval of that layer is adjusted accordingly: if the thickness is too large, the termination pixel coordinate is contracted inward; if the thickness is too small, the termination pixel coordinate is expanded outward until the actual thickness falls within the standard range. For the distortion of the hierarchical distribution caused by the imaging angle deviation, an elastic registration algorithm is used to register the initially divided pixel interval with the standard ureteral anatomical structure model, correct the distorted boundary segments, and ensure that the hierarchical distribution conforms to the actual course of the ureter.

[0046] Step S127: Based on the adjusted anatomical level distribution range, extract all ultrasound pixel data within each adjusted anatomical level distribution range, including the gray value, signal intensity value and phase information of each pixel, classify and organize them according to level and imaging modality to form initial joint level region data.

[0047] In this embodiment, for each anatomical level of each frame image, all pixel data within that level's pixel range are extracted. For tissue morphology imaging data, grayscale values ​​and signal phase information are extracted; for blood flow signal imaging data, pixel values, blood flow velocity amplitude, and phase information across the RGB channels are extracted. The data is categorized according to anatomical level and imaging modality. Each level corresponds to a dataset containing both tissue morphology and blood flow signal data. The data is stored in a two-dimensional array, where rows correspond to pixels, and columns correspond to various attribute values ​​of the pixels, such as grayscale values, phase values, and RGB channel values.

[0048] Step S128: Perform edge processing on the initial joint hierarchical region data. Process the signal aliasing region at the hierarchical boundary through pixel neighborhood similarity analysis. Adjust the hierarchical assignment of the boundary pixels by comparing the signal features of neighboring pixels to obtain the edge-processed initial joint hierarchical region data.

[0049] In this embodiment, for each boundary pixel of a layer, pixel attributes within its 3×3 neighborhood are extracted, including grayscale value, blood flow signal amplitude, and phase value. The feature similarity between the boundary pixel and its neighboring pixels in the inner and outer layers is calculated. The similarity calculation uses a cosine similarity algorithm, where the attribute value of the boundary pixel is used as a vector, and the cosine similarity is calculated with the average attribute vectors of the inner and outer layers. The boundary pixel is then assigned to a layer with higher similarity. For boundary regions with severe signal aliasing, a multimodal fusion approach is used, combining tissue morphology and blood flow signal characteristics for comprehensive judgment. If the blood flow signal amplitude of the boundary pixel is higher than a set threshold, it is assigned to the mucosa layer; if the blood flow signal amplitude is lower than the threshold, it is assigned to the muscle layer or adventitia layer based on the grayscale value.

[0050] Step S129: Arrange the initial combined hierarchical region data after edge processing in an orderly manner according to the physiological order of anatomical layering, and arrange the data of mucosa, submucosa, muscle layer and adventitia layer in sequence from the core region of the lumen outward to form combined hierarchical sequence data.

[0051] In this embodiment, the combined hierarchical sequence data is stored in a linked list structure. Each node in the linked list corresponds to an anatomical level, and the node contains information such as the pixel data set, level identifier, thickness parameters, and boundary coordinates for that level. The linked list is arranged in the following order: lumen core region node → mucosal layer node → muscle layer node → adventitia layer node. Nodes are linked by pointers to facilitate subsequent hierarchical traversal and feature extraction. Simultaneously, a timestamp is added to each node, corresponding to the frame number of the original imaging sequence, ensuring the correspondence between the sequence data and the imaging time.

[0052] Step S1210: Based on the joint hierarchical sequence data, integrate the relevant information of each imaging modality in the joint hierarchical sequence data, establish the association index between each level of data and ultrasound signals of different modalities, and finally obtain the joint ultrasound region data divided according to anatomical level. The joint ultrasound region data contains independent information of each level and retains the correlation between modalities and levels.

[0053] In this embodiment, the association index is stored in a dictionary structure. The dictionary keys are level identifiers, and the dictionary values ​​are index entries containing modality identifiers, data storage addresses, data sizes, and feature types. Each level corresponds to index entries for two modalities: tissue morphology and blood flow signals. The combined ultrasound region data is stored in a hierarchical file format. The root directory contains subdirectories for each level. Each subdirectory stores the tissue morphology data file and blood flow signal data file for the corresponding level, as well as the association index file, which is used to quickly find data at different levels and for different modalities. The data files contain detailed attribute information for each pixel, and the relationships between levels are established through the boundary coordinates in the index file.

[0054] Step S130: Analyze the propagation characteristics of different modal ultrasound signals corresponding to the combined ultrasound region data, including the propagation speed, reflection path and attenuation state of different modal signals in different tissue layers, and extract the cross-modal signal feature set related to the presence of stones.

[0055] In this embodiment, the analysis of ultrasound signal propagation behavior is based on the hierarchical division results of combined ultrasound regional data. For each level and each mode of ultrasound signal, its propagation speed, reflection path, and attenuation state are analyzed. Abnormal signal features related to stones are extracted, including signal attenuation abrupt changes, waveform distortion, frequency shift, etc. The final output cross-modal signal feature set is stored in the form of feature vectors. Each feature vector contains four fields: modality identifier, level identifier, feature type, and feature value.

[0056] Step S131: Obtain the ultrasound propagation path information of different modes corresponding to the data of each combined ultrasound region. Through the scanning parameter recording and imaging geometric model analysis of the ultrasound imaging equipment, determine the propagation direction, path length and tissue medium type of each mode ultrasound signal in the corresponding ultrasound region, and obtain the determined ultrasound propagation path information.

[0057] In this embodiment, the scanning parameters of the ultrasound imaging device include the probe's emission angle, imaging depth, and pixel spacing. The imaging geometry model is constructed based on the linear propagation characteristics of ultrasound. A three-dimensional coordinate system is established with the transducer array center of the probe as the origin. The three-dimensional coordinates of each pixel are calculated using the imaging depth and scanning angle. For each pixel at each level, the propagation direction of the ultrasound signal from the transducer to that pixel is defined as the vector from the origin to the pixel. The path length is the straight-line distance from the origin to the pixel, and the tissue medium type traversed is the tissue type corresponding to the anatomical level to which the pixel belongs. The above information is stored in array form, with each array element containing four fields: pixel coordinates, propagation direction vector, path length, and medium type.

[0058] Step S132: Based on the determined ultrasonic propagation path information, analyze the signal attenuation state on each mode of ultrasonic propagation path. By recording the intensity changes of the ultrasonic signal at each point during propagation, plot the signal intensity attenuation curve of each mode. Record the intensity loss of the ultrasonic signal from the transmitting end to the receiving end to obtain the signal intensity attenuation curve of each mode.

[0059] In this embodiment, for tissue morphology imaging data, sampling points are selected at fixed intervals along the propagation path from the transducer to the pixel, and the grayscale value of each sampling point is recorded. The path length of the sampling point is used as the abscissa, and the grayscale value is used as the ordinate to plot the signal intensity attenuation curve. For blood flow signal imaging data, the amplitude of the blue channel is recorded along the propagation path to plot the blood flow signal intensity attenuation curve. The attenuation curves are stored in the form of a two-dimensional array, where the rows of the array correspond to the sampling points, and the columns correspond to the path length and signal intensity value. The modal identifier, layer identifier, and pixel coordinates corresponding to each curve are also recorded.

[0060] Step S133: Based on the signal intensity attenuation curves of each mode, extract the abrupt change segments in the signal attenuation state of each mode. By comparing the signal intensity difference between adjacent propagation path points, identify propagation path segments where the signal intensity difference exceeds a set threshold, and obtain the identified abrupt change segments.

[0061] In this embodiment, adjacent sampling points of the attenuation curve are traversed, and the signal intensity difference between the next sampling point and the previous sampling point is calculated. If the absolute value of this difference exceeds a set mutation threshold, the path segment between these two sampling points is marked as a mutation segment. The mutation threshold is determined based on the signal attenuation rate of normal tissue at this level. The signal attenuation rate of normal tissue is linear, and the difference is within a fixed range. A mutation segment corresponds to an area where the attenuation rate increases or decreases abnormally, which may be the location where a stone exists. Each mutation segment is stored with four attributes: the starting index of the sampling point, the ending index, the maximum intensity difference, and the average attenuation rate.

[0062] Step S134: Based on the identified abrupt change segment, perform signal waveform analysis on the identified abrupt change segment, perform waveform decomposition on the ultrasound signal of the abrupt change segment, extract the number of peaks, peak interval, valley depth, and steepness of the rising and falling edges of the waveform, and obtain the signal waveform performance of the abrupt change segment.

[0063] In this embodiment, wavelet decomposition is used to decompose the original ultrasonic signal of the abrupt change segment into waveforms, obtaining wavelet coefficients at different scales. Temporal features such as the number of peaks, peak intervals, and valley depths are extracted from these wavelet coefficients: the number of peaks is the number of local maxima exceeding a set amplitude threshold in the wavelet coefficients; the peak interval is the path length difference between adjacent peaks; the valley depth is the intensity difference between a local minimum and an adjacent peak; a steep rising edge represents the ratio of the intensity change from a valley to a peak to the path length; and a steep falling edge represents the ratio of the intensity change from a peak to a valley to the path length. These features are stored numerically, with each abrupt change segment corresponding to a feature set.

[0064] Step S135: Based on the signal waveform performance of the abrupt change segment, compare the signal waveform performance of the same modal abrupt change segment in different combined ultrasound region data, compare the abrupt change waveforms of each region one by one, and statistically analyze the overlap and similarity of the waveform performance to obtain similar waveform data of the same modality across regions.

[0065] In this embodiment, for all abrupt segments of the same modality, the waveform feature set of each segment is converted into a feature vector. The cosine similarity algorithm is used to calculate the similarity between any two feature vectors. If the similarity exceeds a set similarity threshold, the waveforms of the corresponding two abrupt segments are determined to be similar. The number of all similar segment pairs, the average similarity, and the similarity distribution interval are counted to obtain cross-regional waveform similarity data of the same modality. The data is stored in the form of a statistical report, which includes four fields: modality identifier, number of similar segment pairs, average similarity, and similarity interval distribution.

[0066] For example, step S1351: Align the abrupt change segment signal waveforms of the same modality in each combined ultrasound region data according to a uniform time scale, so that the start and end points of all waveforms are consistent on the time axis, and obtain the aligned abrupt change segment signal waveforms.

[0067] In this embodiment, the time scale corresponds to the propagation path length of the ultrasound signal. The starting point of the abrupt change in the propagation path is taken as the zero point of the time axis, and the ending point is taken as the end point. Through linear interpolation or stretching, the signal waveforms of all abrupt change segments are adjusted to have the same time axis length. For abrupt change segments in tissue morphology imaging, the number of sampling points in each segment is uniformly adjusted to a fixed number, with evenly distributed sampling point intervals, based on the path length. For abrupt change segments in blood flow signal imaging, the number of sampling points in each segment is adjusted to a fixed number, based on the blood flow signal acquisition time, ensuring that the time axis range of all waveforms is completely consistent. The aligned waveforms are stored in a two-dimensional array, where rows correspond to sampling points, and columns correspond to path length and signal intensity values.

[0068] Step S1352: Extract the feature parameter set of each aligned abrupt change segment signal waveform. The feature parameter set includes the number of peaks, peak amplitude, peak interval, valley depth, rising edge slope, falling edge slope, waveform duration, and waveform symmetry parameter.

[0069] In this embodiment, the number of peaks is the number of local maxima in the waveform that exceed a set amplitude threshold, determined by traversing the sampling points of the waveform and comparing the intensity values ​​of each sampling point with those of its neighboring sampling points; the peak amplitude is the average signal intensity value of each peak point; the peak interval is the average path length difference between adjacent peak points; the valley depth is the average intensity difference between a local minimum point and its adjacent peak point; the rising edge slope is the average ratio of the intensity change from valley to peak to the path length change; the falling edge slope is the average ratio of the intensity change from peak to valley to the path length change; the waveform duration is the total path length of the abrupt change segment; and the waveform symmetry parameter is the similarity of the number of peaks and valley depths between the first and second halves of the waveform, determined by calculating the cosine similarity of the corresponding features between the first and second halves. These feature parameters are stored as numerical arrays, with one array corresponding to each aligned waveform.

[0070] Step S1353: Standardize the feature parameter set, convert each feature parameter into a unified representation range, eliminate the difference in parameter magnitude caused by the difference in imaging conditions in different combined ultrasound region data, and obtain the standardized feature parameter set.

[0071] In this embodiment, the Z-Score normalization method is used to process each feature parameter. For the same feature parameter across all abrupt changes in the same modality, the mean and standard deviation of the parameter are calculated. Then, each parameter value is converted into a standardized value using a formula, where the standardized value equals (original parameter value minus mean) divided by the standard deviation. After processing, the mean of each feature parameter is 0, and the standard deviation is 1, eliminating the differences in parameter magnitude caused by different imaging conditions such as probe frequency and imaging depth. The standardized feature parameter set is stored in a two-dimensional array, where rows correspond to abrupt changes and columns correspond to standardized feature parameters.

[0072] Step S1354: Based on the standardized feature parameter set, the similarity between the feature parameter sets of any two abrupt signal waveforms is calculated one by one using the feature similarity calculation operation. The above similarity calculation operation is based on the cosine similarity algorithm to obtain the similarity values ​​between each pair.

[0073] In this embodiment, the standardized feature parameter set of each mutation segment is converted into a feature vector. For any two feature vectors, their cosine similarity value is calculated. This value is equal to the dot product of the two vectors divided by the product of their magnitudes, and the value ranges from 0 to 1. The larger the value, the more similar the features of the two waveforms are. The pairwise similarity values ​​are stored in the form of a similarity matrix, where the rows and columns of the matrix correspond to the identifiers of the mutation segments, and the matrix elements are the similarity values ​​between the corresponding two segments.

[0074] Step S1355: Perform cluster analysis on all abrupt segment signal waveforms based on similarity values, and group waveforms with similarity values ​​that meet the set criteria into the same category to form multiple waveform cluster groups.

[0075] In this embodiment, a hierarchical clustering algorithm is used for cluster analysis. Using a similarity matrix as input, aberration segments with similarity values ​​higher than a set clustering threshold are grouped into the same group. The clustering threshold is set based on typical characteristics of clinical stone signals to ensure that waveforms within the same group correspond to the same type of stone signal. The clustering process starts with each aberration segment as a separate group, gradually merging the groups with the highest similarity until the similarity within all groups is higher than the threshold, or the number of groups reaches a set number. The final waveform clusters are stored in list form, with each group containing a list of aberration segment identifiers and the average similarity value within the group.

[0076] Step S1356: For each waveform cluster group, calculate the mean and standard deviation of each feature parameter within the group, determine the core feature parameters of the waveform cluster group, and construct a typical waveform template for the waveform cluster group based on the core feature parameters.

[0077] In this embodiment, for each waveform cluster, the mean and standard deviation of each feature parameter within the group are calculated. The core feature parameter is the mean of all feature parameters. Based on the core feature parameter, a typical waveform template is constructed: using a unified time axis as a reference, the corresponding signal waveform is generated according to the core feature parameter, including features such as the number of peaks, peak intervals, and valley depth, ensuring that the typical waveform template can represent the common features of all waveforms within the group. The typical waveform template is stored in the form of a two-dimensional array, where the rows of the array correspond to sampling points, and the columns correspond to path length and signal strength values.

[0078] Step S1357: Calculate the similarity between each waveform in the group and the typical waveform template, set a first threshold, and adjust the clustering threshold until the number of waveforms in the group whose similarity to the typical waveform template is greater than or equal to the first threshold reaches a set proportion.

[0079] In this embodiment, the first threshold is set based on the accuracy requirements of clinical diagnosis, and the set proportion is above 90% of the number of waveforms within a group. For each waveform cluster, the cosine similarity between the feature vector of each waveform within the group and the typical waveform template is calculated. The number of waveforms with a similarity greater than or equal to the first threshold is counted. If this number does not reach the set proportion, the threshold of the clustering analysis is lowered, and clustering is performed again until the requirements are met. If the number exceeds the set proportion, the threshold of the clustering analysis is raised to optimize the clustering results and ensure the consistency of waveforms within the group. The adjusted clusters are stored in list form, including the segment identifier within the group, the typical waveform template, and the proportion of similarity that meets the standard within the group.

[0080] Step S1358: Compare the typical waveform templates of all waveform clusters, exclude duplicate templates or templates whose similarity meets the set criteria, and retain typical waveform templates with unique common features.

[0081] In this embodiment, the cosine similarity of the feature vectors of any two typical waveform templates is calculated. If the similarity is higher than a set repetition threshold, the two templates are considered to be duplicates. The template with clearer features and higher intra-group similarity is retained, while the other template is excluded. If the similarity is lower than the repetition threshold, it is determined to be a template with unique common features and is retained. The repetition threshold is set based on the difference in waveform features to ensure that the retained templates correspond to different categories of stone signal manifestations. The finally retained typical waveform templates are stored in the form of template identifier, core feature parameters, and waveform data.

[0082] Step S1359: Record the overlap and similarity data corresponding to the retained typical waveform templates.

[0083] In this embodiment, the overlap is the ratio of the number of waveforms within a corresponding waveform cluster to the total number of abrupt transition segments within the same modality. The similarity data includes the core feature parameters of the template, the average similarity within the group, and the similarity distribution interval within the group. This data is stored in a structured table format, containing six fields: template identifier, modality type, overlap, list of core feature parameters, average similarity within the group, and similarity distribution interval. These fields are used for subsequent cross-modal signal feature integration and correlation analysis of stone imaging performance.

[0084] Step S136: Based on the signal waveform performance of the abrupt change segment, compare the signal waveform performance of different modal abrupt change segments within the same combined ultrasound region data, extract the common features of waveform performance between different modes, and obtain the common features of cross-modal waveforms.

[0085] In this embodiment, within the same level of combined ultrasound region, abrupt change segments corresponding to the same location in tissue morphology imaging and blood flow signal imaging are selected, and their waveform characteristics are compared. If the abrupt change segment in tissue morphology imaging is characterized by a large number of peaks and a large valley depth, while the corresponding segment in blood flow signal imaging is characterized by a sudden drop in blood flow signal amplitude and a phase abrupt change, then the combination of these two features is extracted as the common cross-modal waveform feature. The common cross-modal waveform feature is stored in the form of feature pairs, each feature pair containing four fields: tissue morphology feature type, tissue morphology feature value, blood flow signal feature type, and blood flow signal feature value.

[0086] Step S137: Combine the similar waveform data of the same mode across regions with the common features of cross-modal waveforms to form a cross-modal common waveform pattern. Based on the above cross-modal common waveform pattern, extract key parameters of the cross-modal common waveform pattern. Quantify the duration, peak interval, amplitude variation range and symmetry of the waveform using waveform analysis tools to obtain the key parameters of the cross-modal common waveform pattern.

[0087] In this embodiment, similar waveforms across regions within the same modality are integrated with common features across modalities to form a common cross-modal waveform pattern. Each pattern corresponds to a type of abnormal signal manifestation related to the presence of stones. Waveform analysis tools are used to quantify the waveform of each pattern. The duration is calculated as the path length of the abrupt change segment divided by the propagation speed of the ultrasound signal in the tissue; the peak interval is the average path length between adjacent peaks; the amplitude variation range is the difference between the maximum and minimum intensity values ​​of the waveform; and symmetry is defined as the similarity in the number of peaks and the depth of valleys between the first and second halves of the waveform. These key parameters are stored numerically, with each pattern corresponding to a set of parameters.

[0088] Step S138: Extract the abrupt change amplitude data in the attenuation state of each modal signal, and integrate the abrupt change amplitude data with the key parameters of the cross-modal common waveform mode to construct the basic cross-modal signal features related to the presence of stones.

[0089] In this embodiment, the abrupt change amplitude data is the ratio of the maximum intensity difference of the abrupt change segment to the normal attenuation rate of that level. This ratio is associated with the key parameters of the cross-modal common waveform pattern. Each basic cross-modal signal feature includes three parts: abrupt change amplitude data, a mode identifier, and a set of key parameters. The mode identifier is used to associate the corresponding cross-modal common waveform pattern, and the set of key parameters includes quantization parameters such as duration and peak interval.

[0090] Step S139: Expand the feature dimensions of the basic cross-modal signal features, incorporate the frequency variation and phase shift data of each modal ultrasound signal, obtain the main frequency distribution and frequency component changes of the signal through frequency spectrum analysis, record the phase shift state of the signal through phase detection, classify and organize the expanded basic cross-modal signal features according to feature type, establish the correspondence between features, and form a set of cross-modal signal features related to the presence of stones.

[0091] In this embodiment, a Fast Fourier Transform (FFT) is used to perform frequency spectrum analysis on the signal in the abrupt change segment to obtain the signal's dominant frequency distribution range, dominant frequency amplitude, and rate of change of frequency components, representing frequency variation. A phase difference calculation method is used to compare the signal phase of the abrupt change segment with that of the adjacent normal segment, obtaining phase shift data. This data is added to the basic cross-modal signal features to expand the feature dimensions. Then, features are categorized by type into four types: attenuation abrupt change features, waveform distortion features, frequency shift features, and phase shift features. A correspondence is established within each feature category, such as the correspondence between attenuation abrupt change features and waveform distortion features, forming the final cross-modal signal feature set. This set is stored in the form of a feature matrix, where rows correspond to features and columns correspond to various attributes of the features.

[0092] Step S140: Establish a dynamic correlation link between the cross-modal signal feature set and the stone imaging performance. The dynamic correlation link is constructed by the correspondence between the cross-modal signal features and the stone morphology, internal echo performance, and posterior acoustic shadow performance. The dynamic correlation link adaptively adjusts the correlation logic as the imaging conditions change.

[0093] In this embodiment, the dynamic association link is constructed with a directed graph structure. Nodes correspond to cross-modal signal features and stone imaging performance, edges correspond to the association relationship between features and imaging performance, and the weight of the edge represents the association strength. The structure of the link adaptively adjusts the weight of the edge and the connection relationship of the node according to the changes in imaging conditions such as scanning angle, probe frequency, and stone size, so as to ensure that the imaging performance of the stone can be accurately identified under different imaging conditions.

[0094] Step S141: Obtain typical data of stone imaging performance, the typical data including the contour morphology, internal echo intensity, internal echo uniformity, posterior acoustic shadow and lateral acoustic shadow of the stone in ultrasound imaging.

[0095] In this embodiment, the typical data for stone imaging are derived from surgically confirmed ultrasound images of ureteral stones collected from multiple clinical centers. Each case includes multiple frames of data for tissue morphology imaging and blood flow signal imaging. For each frame, the contour morphology of the stone is labeled, such as round, oval, or irregular; internal echo intensity, such as hyperechoic, intermediate, or hypoechoic; internal echo uniformity, such as uniform or non-uniform; posterior acoustic shadowing, such as present, absent, or weak; and lateral acoustic shadowing, such as present or absent. The above labeled data is stored in a structured format, with each case corresponding to a data entry, including case identifier, imaging modality, frame number, and labeling results for each imaging feature.

[0096] Step S142: Decompose the typical data of stone imaging into features. The typical data is divided into independent feature units by feature decomposition algorithm to obtain stone morphological feature elements and echo feature elements. The morphological feature elements include smooth contour, regular shape, and clear boundary. The echo feature elements include echo intensity, echo uniformity, and echo attenuation rate.

[0097] In this embodiment, a rule-based feature decomposition algorithm is used to decompose the contour morphology into three morphological feature elements: contour smoothness, shape regularity, and boundary clarity. Contour smoothness is determined by the rate of change of curvature; a rate of change of curvature below a threshold indicates smoothness. Shape regularity is determined by the area ratio of the circumcircle to the minimum enclosing rectangle; an area ratio above a threshold indicates regularity. Boundary clarity is determined by the gray-level gradient value of the contour edge; a gradient value above a threshold indicates clarity. Internal echo intensity, internal echo uniformity, and posterior acoustic shadow are decomposed into three echo feature elements: echo intensity, echo uniformity, and echo decay rate. Echo intensity is determined by the ratio of the average gray-level value of the stone region to the average gray-level value of the surrounding tissue. Echo uniformity is determined by the standard deviation of the gray-level values ​​in the stone region; a standard deviation below a threshold indicates uniformity. Echo decay rate is determined by the signal attenuation rate of the posterior acoustic shadow region; an attenuation rate above a threshold indicates rapid attenuation.

[0098] Step S143: Based on the obtained stone morphological feature elements and echo feature elements, analyze the correspondence between each signal feature and morphological feature element in the cross-modal signal feature set. By comparing the performance status of each signal feature and morphological feature element one by one, define the characterization effect of signal features on morphology.

[0099] In this embodiment, each feature in the cross-modal signal feature set is compared one by one with three morphological feature elements: if a feature is a high-frequency abrupt change feature in tissue morphology imaging, corresponding to a clear boundary of the stone, because the density difference between the stone and the surrounding tissue is large, generating a high-frequency reflection signal and a large boundary gray-scale gradient, then this feature is defined as having a direct characterizing effect on the clear boundary; if a feature is a blood flow interruption feature in blood flow signal imaging, corresponding to a smooth contour of the stone, because a smooth stone surface will completely block blood flow, causing blood flow signal interruption, while a rough surface will have some blood flow signal, then this feature is defined as having an indirect characterizing effect on the smooth contour. The above correspondence is stored in the form of an association table, which contains four fields: signal feature identifier, morphological feature element identifier, characterization type, and confidence level.

[0100] Step S144: Based on the obtained stone morphology features and echo features, simultaneously analyze the matching status of each signal feature and echo feature in the cross-modal signal feature set, quantify the fit of each signal feature and echo feature through feature matching analysis, and obtain the correlation strength value between signal features and echo.

[0101] In this embodiment, a cosine similarity algorithm is used for feature matching analysis. The numerical values ​​of signal features are used as feature vectors, and the numerical values ​​of echo feature elements are also converted into feature vectors. The cosine similarity between the two vectors is calculated. This cosine similarity value represents the correlation strength between the signal feature and the echo feature element, ranging from 0 to 1. A higher value indicates a better match. For example, in tissue morphology imaging, the correlation strength between high-echo signal features and high-echo elements in echo intensity is close to 1, while the correlation strength with low-echo elements is close to 0. The correlation strength values ​​are stored in matrix form, with rows corresponding to signal features, columns corresponding to echo feature elements, and matrix elements representing the correlation strength values.

[0102] Step S145: Based on the defined representation function and the obtained correlation strength value, construct the initial correlation relationship, and bind each signal feature with the corresponding morphological feature element and echo feature element to form a one-to-one or many-to-one correspondence.

[0103] In this embodiment, signal features are bound to corresponding features based on their representational effect on morphological features and their correlation strength with echo features: if a signal feature represents only one morphological feature and has the highest correlation strength with one echo feature, a one-to-one correspondence is formed; if a signal feature represents multiple morphological features or has a high correlation strength with multiple echo features, a many-to-one correspondence is formed. The initial association relationships are stored in a dictionary structure, where the dictionary key is the signal feature identifier, and the dictionary value is a set containing a list of morphological feature identifiers, a list of echo feature identifiers, and a list of correlation strengths.

[0104] Step S146: Use multimodal ultrasound imaging data of known stone locations in a clinical standard quantity as validation samples, apply the initial correlation to the analysis of the validation samples, and retain the valid initial correlations that are stable and correlated in the validation samples.

[0105] In this embodiment, the validation samples are multimodal ultrasound data of surgically confirmed ureteral stones collected from multiple clinical centers. The sample size meets the standard requirements for clinical research. Each sample contains complete sequence data of tissue morphology imaging and blood flow signal imaging, as well as gold standard annotations for stone location, morphology, and echogenicity. Initial associations are applied to the cross-modal signal feature analysis of each sample. If the correspondence between a signal feature and its corresponding morphological and echogenic features is consistent with the gold standard annotation in more than 80% of the samples, the initial association is retained; otherwise, it is discarded. Valid initial associations are stored in the form of an association table, containing four fields: signal feature identifier, morphological feature identifier, echogenic feature identifier, and sample consistency rate.

[0106] Step S147: Group the valid initial correlations according to the category of signal features. Classify the corresponding valid initial correlations according to the type of signal features to form feature correlation groups. The correlations in each group have similar feature bases.

[0107] In this embodiment, the signal features are categorized into four types: attenuation abrupt change features, waveform distortion features, frequency shift features, and phase shift features. Valid initial correlations corresponding to signal features belonging to the same category are grouped together to form four feature correlation groups: the attenuation abrupt change feature group contains all correlations related to signal attenuation abrupt changes; the waveform distortion feature group contains all correlations related to waveform distortion; the frequency shift feature group contains all correlations related to frequency shift; and the phase shift feature group contains all correlations related to phase shift. Each group of correlations has a similar feature basis, all originating from the same type of signal anomaly.

[0108] Step S148: Analyze the interaction order between feature association groups. By dividing the influence order and interaction mode of different types of signal features on the stone imaging performance, determine the order of action of each group of association relationships.

[0109] In this embodiment, based on the interaction principle between ultrasound signals and stones, the interaction order of the feature association groups is analyzed: the attenuation abrupt change feature group corresponds to the initial reflection signal of the stone, which is the first abnormal signal generated and therefore acts first; the waveform distortion feature group corresponds to the scattered signal inside the stone, which is a secondary signal after attenuation abrupt change and acts second; the frequency shift feature group corresponds to the resonance signal on the surface of the stone, which is a secondary signal after waveform distortion and acts again; the phase shift feature group corresponds to the phase interference signal around the stone, which is the last signal generated and acts last. Simultaneously, there are interactions between the groups; for example, the attenuation abrupt change feature affects the waveform distortion feature, therefore, the above dependencies need to be reflected in the link structure.

[0110] Step S149: Concatenate the feature association groups in the order of basic association first and supplementary association last to construct a link structure and form a preliminary dynamic association link.

[0111] In this embodiment, the groups are connected in the order of attenuation abrupt change feature group, waveform distortion feature group, frequency offset feature group, and phase offset feature group to form a directed graph structure of preliminary dynamic association links. Each feature association group corresponds to a node group in the graph, and the association relationship within a group is a directed edge between nodes, with the edge direction pointing from the signal feature node to the morphology and echo feature element node. At the same time, directed edges are added between groups to reflect the interaction between groups. For example, a node from the attenuation abrupt change feature group to a node from the waveform distortion feature group indicates that the attenuation abrupt change feature will affect the performance of the waveform distortion feature.

[0112] Step S1410: Establish dynamic correlation link adjustment rules using imaging condition change factors, integrate the adjustment rules into the preliminary dynamic correlation link, so that the preliminary dynamic correlation link adjusts the weight and action order of the correlation relationship according to the change of imaging conditions, and obtains the dynamic correlation link.

[0113] In this embodiment, the imaging condition variation factors include ultrasound scanning angle, ultrasound probe center frequency, tissue type around the stone, and maximum diameter of the stone. For each factor, a corresponding adjustment rule is established, and the rule is integrated into the directed graph structure of the preliminary dynamic association link, so that the edge weight of the link and the node connection order are automatically adjusted as the imaging conditions change.

[0114] For example, step S14101: Obtain a multimodal ultrasound imaging data sample set with different stone types, different stone sizes, and different stone anatomical locations.

[0115] In this embodiment, the sample set is derived from surgically confirmed ureteral stone data collected from multiple clinical centers. It includes different types such as calcium oxalate stones, uric acid stones, and calcium phosphate stones, with stone sizes ranging from very small to large. Anatomical locations cover the upper, middle, and lower ureter. Each sample includes multimodal ultrasound imaging sequence data, the gold standard annotation of the stone, and imaging condition parameters such as scanning angle and probe frequency. The sample size meets the standard scale requirements for training machine learning models, ensuring the generalization ability of the adjustment rules.

[0116] Step S14102: Apply the preliminary dynamic correlation link to the analysis and processing of the multimodal ultrasound imaging data sample set, and record the deviation between the correlation result of each sample in the preliminary dynamic correlation link and the actual stone imaging performance.

[0117] In this embodiment, for each sample in the sample set, a cross-modal signal feature set is extracted and input into a preliminary dynamic correlation link to obtain the correlation results of the corresponding stone morphology and echo feature elements. This result is compared with the gold standard annotation of the sample, and the deviation is recorded: if the morphological feature elements in the correlation result do not match the gold standard, a morphological deviation is recorded; if the echo feature elements do not match the gold standard, an echo deviation is recorded; if both do not match, a comprehensive deviation is recorded. The deviation is stored in a structured format, with each sample corresponding to a data entry containing five fields: sample identifier, correlation result, gold standard annotation, deviation type, and deviation degree.

[0118] Step S14103: Based on the recorded deviation performance, identify the association nodes in the preliminary dynamic association link where the association results show deviation under specific stone imaging performance.

[0119] In this embodiment, the associated nodes corresponding to different deviation types are statistically analyzed. If the deviation rate of the associated results for a certain signal feature node exceeds a set threshold under a specific stone imaging appearance, such as a hyperechoic heterogeneous stone, then the node is identified as an associated node with deviation. The deviation rate is the ratio of the number of samples with deviation for that node in the corresponding imaging appearance to the total number of samples. The threshold is set based on the accuracy requirements of clinical diagnosis. Associated nodes with deviation are stored in three fields: node identifier, corresponding imaging appearance, and deviation rate.

[0120] Step S14104: Analyze the imaging condition influencing factors corresponding to the above-mentioned associated nodes. The imaging condition influencing factors include ultrasound scanning angle, ultrasound probe center frequency, tissue type around the stone, and maximum diameter of the stone.

[0121] In this embodiment, for each associated node with deviation, the corresponding sample imaging condition parameters are analyzed, and the deviation rate under different imaging conditions is statistically analyzed: if the deviation rate increases significantly when the scanning angle is less than a certain threshold, then the ultrasound scanning angle is an influencing factor; if the deviation rate increases significantly when the probe center frequency is less than a certain threshold, then the probe center frequency is an influencing factor; if the deviation rate increases significantly when the area surrounding the stone is adipose tissue, then the tissue type surrounding the stone is an influencing factor; if the deviation rate increases significantly when the maximum diameter of the stone is greater than a certain threshold, then the maximum diameter of the stone is an influencing factor. These influencing factors are stored using four fields: node identifier, influencing factor type, influencing threshold, and deviation rate change.

[0122] Step S14105: For each key influencing factor, establish correlation adjustment rules, and clarify the weight adjustment ratio and the order of action of the correlation within different influencing factor value ranges.

[0123] In this embodiment, adjustment rules are established for the factors affecting ultrasound scanning angle: when the scanning angle is below a threshold, the correlation weight of the attenuation mutation feature group is multiplied by a decreasing ratio, and the correlation weight of the waveform distortion feature group is multiplied by an increasing ratio. Simultaneously, the effect order of the waveform distortion feature group is moved forward to before that of the attenuation mutation feature group. For the factors affecting probe center frequency, adjustment rules are established: when the probe center frequency is below a threshold, the correlation weight of the frequency shift feature group is multiplied by an increasing ratio, and the correlation weight of the phase shift feature group is multiplied by a decreasing ratio. For the factors affecting the type of tissue surrounding the stone, adjustment rules are established: when the stone is surrounded by adipose tissue, the correlation weight of the attenuation mutation feature group is multiplied by an increasing ratio to counteract the signal attenuation interference from the adipose tissue. For the factors affecting the maximum diameter of the stone, adjustment rules are established: when the maximum diameter of the stone is greater than a threshold, the correlation weight of the echo uniformity is multiplied by a decreasing ratio, because large stones are more prone to internal heterogeneity. These adjustment rules are stored in the form of a rule table, containing four fields: influencing factor type, value range, weight adjustment ratio, and change in the order of action.

[0124] Step S14106: Construct a dynamic adjustment model and integrate the correlation adjustment rules into the dynamic adjustment model. The dynamic adjustment model is used to optimize the preliminary dynamic correlation link by calling the corresponding adjustment rules according to the input imaging condition parameters.

[0125] In this embodiment, the dynamic adjustment model is a rule-based inference model. The input is imaging condition parameters, including scanning angle, probe frequency, surrounding tissue type, and maximum diameter of the stone. The output is the adjusted association link structure, including the adjustment ratio of edge weights and the change in the order of node action. The model is implemented internally using a rule engine, which converts the adjustment rules into rule statements. When the imaging condition parameters are input, the rule engine automatically matches the corresponding adjustment rule and outputs the adjustment instruction.

[0126] Step S14107: Train the dynamic adjustment model using a multimodal ultrasound imaging data sample set. Input the imaging condition parameters and deviation performance from the multimodal ultrasound imaging data sample set into the dynamic adjustment model, adjust the adjustment rule parameters in the dynamic adjustment model, and obtain the trained dynamic adjustment model.

[0127] In this embodiment, the sample set is divided into a training subset and a validation subset. The training subset is used to adjust the rule parameters, and the validation subset is used to verify the model's performance. For each sample in the training subset, the imaging condition parameters are input into the dynamic adjustment model to obtain adjustment instructions. The adjusted association links are applied to that sample, and the conformity rate between the association results and the gold standard is calculated. Based on the conformity rate adjustment rules, parameters such as the weight adjustment ratio and the change in the order of action are adjusted to maximize the conformity rate. The above process is repeated until the conformity rate of the validation subset reaches the set accuracy requirement. At this point, the trained dynamic adjustment model is obtained.

[0128] Step S14108: Apply the trained dynamic adjustment model to a new test sample set, compare the correlation performance of the dynamic correlation links before and after adjustment, and verify the optimization effect of the dynamic adjustment model.

[0129] In this embodiment, the test sample set consists of clinical multicenter ureteral stone data that was not used in training. The sample size meets the validation requirements. The initial dynamic association links and the association links adjusted by the trained dynamic adjustment model are applied to the test sample set, respectively. The concordance rate and deviation rate of the association results of the two links with the gold standard are statistically analyzed. If the concordance rate of the adjusted links is more than 10% higher than that of the initial links, and the deviation rate is more than 10% lower than that of the initial links, then the optimization effect of the dynamic adjustment model is validated. The validation results are stored in the form of a statistical report, including four fields: link type, concordance rate, deviation rate, and sample size.

[0130] Step S14109: Based on the optimized dynamic adjustment model, make comprehensive adjustments to the preliminary dynamic correlation link so that each correlation node in the preliminary dynamic correlation link can be optimized according to the changes in imaging conditions.

[0131] In this embodiment, the adjustment rules of the trained dynamic adjustment model are applied to all associated nodes of the initial dynamic association link. For each node's influencing factors, corresponding weight adjustment ratios and triggering conditions for changes in the order of action are set. When the imaging condition parameters meet the triggering conditions, the association weights and order of action of the nodes are automatically adjusted to ensure that the association link can accurately reflect the correspondence between cross-modal signal characteristics and stone imaging performance under any imaging conditions.

[0132] Step S141010: Output the adjusted dynamic correlation link, which responds to changes in imaging conditions and stone imaging performance, and maintains the correlation between signal features and stone imaging performance.

[0133] In this embodiment, the dynamic correlation link is stored in a directed graph structure file, containing node information, edge information, and adjustment rule information. The node information includes the identifiers and attributes of signal feature nodes, morphological feature element nodes, and echo feature element nodes; the edge information includes the correlation relationships between nodes, initial weights, and adjustment rule triggering conditions; the adjustment rule information includes the type of influencing factor, value range, weight adjustment ratio, and change in the order of action. The link file can be called by the subsequent positioning module to realize dynamic correlation analysis between cross-modal signal features and stone imaging performance.

[0134] Step S150: By combining the anatomical layer positions corresponding to each joint ultrasound region data through dynamic correlation links, the cross-modal signal feature distribution status and anatomical structure spatial relationship are integrated to generate the location result of the stone in the ureter. The location result includes the anatomical segment where the stone is located, its relative position to the lumen wall, and its distance relationship with surrounding key structures.

[0135] In this embodiment, the localization result is generated based on the distribution of cross-modal signal features and combined with the anatomical layering information of the combined ultrasound regional data. Through coordinate transformation, position mapping, three-dimensional modeling and other steps, the final output is a standardized localization result containing anatomical segments, relative positions and distance relationships. It also includes a bidirectional mapping relationship between ultrasound image coordinates and anatomical structure coordinates, which is convenient for clinicians to refer to and apply.

[0136] Step S151: Analyze the stone imaging manifestations corresponding to each signal feature in the dynamic correlation link. Based on the correlation relationship in the dynamic correlation link, classify the stone morphology and echo imaging manifestations reflected by each signal feature one by one, and establish a corresponding mapping table between signal features and imaging manifestations.

[0137] In this embodiment, all signal feature nodes in the dynamic correlation link are traversed. Based on the correlation edges between nodes, the stone morphology feature elements and echo feature elements corresponding to each signal feature are determined, and a corresponding mapping table is established. The mapping table contains four fields: signal feature identifier, morphology feature element identifier, echo feature element identifier, and correlation strength. For each signal feature, if the correlation strength is higher than a set threshold, the corresponding morphology and echo feature elements are marked as the main imaging manifestations reflected by that signal feature. The mapping table is used for subsequent signal feature localization analysis.

[0138] Step S152: Based on the pixel coordinate index of the joint ultrasound region data, locate the specific distribution position of each signal feature in the joint ultrasound region data, determine the pixel set corresponding to each signal feature, and delineate the spatial distribution range of the signal feature in the ultrasound image.

[0139] In this embodiment, the pixel coordinate index of the combined ultrasound region data is stored in a dictionary structure. The key of the dictionary is a hierarchical identifier, and the value is the pixel coordinate range of that hierarchical level. For each cross-modal signal feature, the corresponding pixel coordinate range is searched in the combined ultrasound region data according to its corresponding hierarchical identifier. Then, based on the extraction location of the signal feature, the specific pixel set corresponding to that feature is determined. This pixel set is a list of coordinates of pixels with abnormal signal features, and its spatial distribution range is the coordinate range of the smallest bounding rectangle of the pixel set, represented by the start x-coordinate, end x-coordinate, start y-coordinate, and end y-coordinate.

[0140] Step S153: Combining the anatomical layer locations corresponding to each combined ultrasound region data, the pixel coordinates of the signal feature distribution location are mapped to the ureteral anatomical layer structure through the coordinate transformation model between anatomical layer and ultrasound image, thereby obtaining the anatomical location coordinates corresponding to the signal features.

[0141] In this embodiment, the coordinate transformation model between anatomical layering and ultrasound images is constructed based on the imaging geometry parameters of the ultrasound equipment. It converts the two-dimensional pixel coordinates of the ultrasound image into three-dimensional coordinates of the ureteral anatomical structure. The transformation process includes: first, converting the two-dimensional pixel coordinates into three-dimensional spatial coordinates based on imaging depth, scanning angle, and pixel spacing; then, mapping the three-dimensional spatial coordinates to the coordinate system of the ureteral anatomical layering structure, and performing registration based on the three-dimensional anatomical model of the ureter. The final anatomical position coordinates are three-dimensional coordinates in the ureteral anatomical coordinate system, containing coordinate values ​​for the x, y, and z axes, corresponding to the length, circumferential, and radial directions of the ureter.

[0142] Step S154: Based on the anatomical location coordinates corresponding to the signal features, analyze the distribution density and intensity concentration of the signal features, and extract the anatomical location with the most concentrated signal features from the mapped location information as the core location point.

[0143] In this embodiment, the anatomical coordinates of the signal features are clustered. The K-means clustering algorithm is used to divide the coordinate points into multiple clusters, each corresponding to a concentrated signal region. The average signal feature intensity of each cluster is calculated, and the center coordinates of the cluster with the highest average value are selected as the core location point. The distribution density is the ratio of the number of coordinate points within each cluster to the total number of coordinate points. Intensity concentration is represented by the standard deviation of the signal feature intensity within each cluster; a smaller standard deviation indicates a more concentrated intensity. The core location point is stored in three-dimensional anatomical coordinates, including x, y, and z coordinate values ​​and the corresponding average intensity.

[0144] Step S155: Based on the core location point, analyze the specific hierarchical affiliation of the core location point in the anatomical layered structure, and classify the core location point as being located in the lumen core region, mucosal layer or other layers of the ureter. Combined with the physiological location of the stone, define the positioning range.

[0145] In this embodiment, the three-dimensional anatomical coordinates of the core location point are compared with the hierarchical range of the ureteral anatomical layer structure. If the radial position corresponding to the coordinates is within the core region of the lumen, it is classified as the core region of the lumen; if it is within the mucosal layer, it is classified as the mucosal layer; if it is within the muscular or adventitia layer, the judgment is made based on the type of signal characteristics. Since stones are usually located in the lumen or mucosal layer, if the core location point is located in the muscular or adventitia layer, it may be signal interference, and it needs to be verified in conjunction with other signal characteristics. The defined positioning range is the area around the core location point with a certain radial length, circumferential angle, and longitudinal distance. The specific range is set based on the average size of the stone.

[0146] Step S156: Integrate the anatomical coordinates and hierarchical information of the core location points to establish a three-dimensional anatomical positioning model of the core location points.

[0147] In this embodiment, the three-dimensional anatomical positioning model is constructed based on the standard anatomical structure model of the ureter, and the core location points are embedded in it to show the spatial location, hierarchical affiliation and relationship with the surrounding structures of the core location points.

[0148] For example, step S1561: Obtain three-dimensional anatomical structure data of the ureter. The three-dimensional anatomical structure data of the ureter is constructed based on human anatomy standards. The three-dimensional anatomical structure data of the ureter includes three-dimensional coordinate information of the entire ureter, thickness distribution of each anatomical level, and spatial positional relationship with surrounding organs.

[0149] In this embodiment, the three-dimensional anatomical structure data of the ureter comes from the standard model of the human anatomy database, which includes a three-dimensional mesh model of the entire ureter. The thickness distribution of each anatomical level is stored in the vertex data of the mesh in the form of attribute fields. The spatial positional relationship of the surrounding organs includes the relative position coordinates of the ureter with the kidney, bladder and blood vessels. The data is stored in STL format, which is convenient for the three-dimensional modeling software to read and process.

[0150] Step S1562: Extract the anatomical coordinate data of the core location point, wherein the anatomical coordinate data is the three-dimensional coordinate value in the three-dimensional anatomical structure data coordinate system of the ureter mentioned above.

[0151] In this embodiment, the anatomical coordinate data of the core location point is extracted from the output of step S154 and is a three-dimensional value of x, y, and z in the ureteral anatomical coordinate system. The x-axis corresponds to the length direction of the ureter, pointing from the renal pelvis junction to the bladder inlet; the y-axis corresponds to the circumference of the ureter, around the length direction of the ureter; and the z-axis corresponds to the radial direction of the ureter, pointing from the center of the lumen to the outer membrane layer.

[0152] Step S1563: Extract the anatomical parameters corresponding to the hierarchical affiliation of the core location point, including the thickness, tissue density, and distance from the center of the lumen of that layer.

[0153] In this embodiment, anatomical parameters of the core location point's layer are extracted from the three-dimensional anatomical data of the ureter: thickness is the radial dimension of that layer, the distance from the inner boundary to the outer boundary; tissue density is the average tissue density value of that layer, a standard parameter derived from an anatomical database; and the distance from the lumen center is the radial coordinate value of the core location point, i.e., the z-axis coordinate value. These parameters are stored numerically and include three fields: thickness, tissue density, and distance from the lumen center.

[0154] Step S1564: Associate and bind the three-dimensional coordinates of the core location point with the corresponding hierarchical anatomical structure parameters to form the three-dimensional feature data of the core location point. The three-dimensional feature data of the core location point contains both spatial location information and hierarchical structure information.

[0155] In this embodiment, the three-dimensional feature data of the core location points are stored in a dictionary structure. The keys of the dictionary are attribute names, such as x-coordinate, y-coordinate, z-coordinate, layer thickness, tissue density, and distance from the center of the lumen. The values ​​of the dictionary are the corresponding numerical values, which associates spatial location information with layer structure information, facilitating subsequent three-dimensional modeling and display.

[0156] Step S1565: Based on the three-dimensional feature data of the core location points, construct the basic model of the core location points using three-dimensional modeling operations. With the three-dimensional coordinates of the core location points as the center, construct the basic spatial geometric model in combination with the layer thickness parameters.

[0157] In this embodiment, the model is created using the API of a 3D modeling software. A sphere is constructed as the base model, centered on the 3D coordinates of the core location point. The radius of the sphere is half the thickness of the layer, as the core location point is located at the radial midpoint of the layer, and the size of the sphere corresponds to the radial dimension of that layer. The base model is stored as a polygonal mesh, containing vertex coordinates and face index data.

[0158] Step S1566: Integrate the three-dimensional anatomical structure data of the ureter with the basic model of the core location points, embed the basic model of the core location points into the three-dimensional anatomical structure data of the ureter, so that the spatial position of the basic model of the core location points completely corresponds to the actual anatomical position.

[0159] In this embodiment, a spatial registration algorithm is used to register the basic model of the core location points with the three-dimensional anatomical structure data of the ureter. The registration reference is the key anatomical points along the length of the ureter, such as the renal pelvis junction, the iliac vessel crossing, and the bladder inlet. The center coordinates of the basic model are precisely matched with the anatomical coordinates of the core location points to ensure that the basic model is located at the corresponding position of the ureteral anatomy. The registered model is stored in STL format and contains the three-dimensional mesh of the ureter and the spherical model of the core location points.

[0160] Step S1567: Add hierarchical classification information to the 3D model, and mark the anatomical level where the core location point is located by using different colors or textures.

[0161] In this embodiment, different color codes are used for different anatomical layers: the core lumen region is marked in blue, the mucosa layer in green, the muscle layer in yellow, and the adventitia layer in red. The base model of the core location points is set to the color of the corresponding layer, and a text label indicating the layer affiliation, such as "mucosa layer," is added to the model's attribute fields to facilitate subsequent model reading and identification.

[0162] Step S1568: Add the three-dimensional data of the ureteral lumen wall, adjacent blood vessels, and organ structures around the core location point to the above three-dimensional model, spatially register the three-dimensional model with the reference point of the known anatomical location, and adjust the coordinate accuracy and structural proportion of the three-dimensional model based on the spatial registration result.

[0163] In this embodiment, three-dimensional mesh data of the ureteral lumen wall, adjacent blood vessels, kidney, and bladder within a certain range around the core location point are extracted from the three-dimensional anatomical structure data of the ureter and added to the three-dimensional model, so that the model includes the surrounding structure of the core location point. Then, reference points with known anatomical locations, such as the coordinates of the renal pelvis junction and the coordinates of the bladder inlet, are selected, and the model's reference points are spatially registered with the anatomical standard coordinates. The scaling ratio and translation parameters of the model are adjusted to ensure that the model's coordinate accuracy and structural proportions are consistent with the actual anatomical structure.

[0164] Step S1569: Output a three-dimensional anatomical model of the core location point. The three-dimensional anatomical model of the core location point is used to show the spatial location, hierarchical affiliation, and relationship with surrounding structures of the core location point in the ureteral anatomy.

[0165] In this embodiment, the three-dimensional anatomical positioning model is stored in STL format, which includes a three-dimensional mesh of the ureter, a basic model of the core location points, and a three-dimensional mesh of the surrounding structures. It also stores the model's attribute information, including the anatomical coordinates of the core location points, their hierarchical affiliation, and the identification of the surrounding structures. This allows clinicians to view, rotate, and zoom the model using three-dimensional modeling software, and intuitively understand the relationship between the location of the stone and the surrounding structures.

[0166] Step S157: Combine the correlation strength values ​​between signal characteristics and stone imaging performance in the dynamic correlation link to evaluate the reliability of the location of the core location point, and retain the core location points whose reliability meets the set standards.

[0167] In this embodiment, the reliability of the core location point is determined by the average correlation strength between the corresponding cross-modal signal features and the stone imaging performance. The higher the average correlation strength, the higher the reliability. A reliability threshold is set. If the reliability of the core location point is higher than the threshold, the core location point is retained; if it is lower than the threshold, it is marked as a suspicious location point and needs to be further verified in conjunction with other signal features. The retained core location points are stored in the form of three-dimensional anatomical coordinates and reliability values.

[0168] Step S158: Based on the retained core location point, extract the signal feature change state within a set range around the retained core location point, analyze the signal feature distribution state around the core location point, and delineate the possible extension range and morphological boundary of the stone.

[0169] In this embodiment, the defined range is an area surrounding the core location point with a certain radial length, circumferential angle, and longitudinal distance. For all pixels within this range, the change state of cross-modal signal features is extracted, including the signal intensity attenuation rate, waveform distortion degree, frequency shift, and phase shift. The distribution of these features is analyzed: if the signal features of the surrounding area are similar to those of the core location point, it is defined as the extension range of the stone; if the features change from abnormal to normal, the boundary is the morphological boundary of the stone. The extension range of the stone is represented by the minimum bounding box in three-dimensional anatomical coordinates, and the morphological boundary is represented by a list of three-dimensional coordinates of a polygon.

[0170] Step S159: Integrate the core location points, hierarchical classification, three-dimensional anatomical positioning model and extended range information to construct a complete positioning description of the stone. The complete positioning description includes the central location of the stone, the anatomical segment in which it is located, the distance from the lumen wall, and the approximate morphological range.

[0171] In this embodiment, the complete location description is stored in structured text format. The center position is the three-dimensional anatomical coordinates of the core location point; the anatomical segment is the upper, middle, or lower segment of the ureter determined based on the length direction coordinates of the core location point; the distance to the lumen wall is the difference between the radial coordinate value of the core location point and the lumen radius; the approximate morphological range is a description of the dimensions of the smallest bounding box of the extended range, such as the length direction range, circumferential range, and radial range. Simultaneously, the storage path of the three-dimensional anatomical location model is added to the complete location description for easy reference and viewing.

[0172] Step S1510: Generate the location result of the stone in the ureter based on the complete location description. The location result adopts a standardized anatomical location representation method and includes a two-way mapping relationship between ultrasound image coordinates and anatomical structure coordinates.

[0173] In this embodiment, the standardized anatomical location description follows the norms of clinical anatomy, such as "within the lumen of the upper ureter, X cm from the renal pelvis junction, Y cm from the inner wall of the lumen, roughly circular in shape, with an approximate length of Z cm and a circumferential angle of W degrees." The bidirectional mapping relationship between ultrasound image coordinates and anatomical structure coordinates is stored in the form of a coordinate mapping table, which includes the two-dimensional pixel coordinates of the ultrasound image, the corresponding three-dimensional coordinates of the anatomical structure, and the two-dimensional pixel coordinates of the ultrasound image corresponding to the three-dimensional coordinates of the anatomical structure. This facilitates clinicians in quickly locating the stone in the ultrasound image and understanding its specific position within the anatomical structure. The location results are stored in the form of a structured document, including a complete location description text, a coordinate mapping table, and a three-dimensional anatomical location model path, facilitating reading and display by the clinical system.

[0174] Furthermore, Figure 2 A schematic diagram of the hardware structure of a multimodal ultrasound-based ureteral stone localization system 100 for implementing the method provided in the embodiments of this application is shown. Figure 2 As shown, the ureteral stone localization system 100 based on multimodal ultrasound may include at least one processor 102 (the processor 102 may be, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, a transmission device 106 for communication functions, and a controller 108. Those skilled in the art will understand that... Figure 2 The structure shown is for illustrative purposes only and does not limit the structure of the ureteral stone localization system 100 based on multimodal ultrasound. For example, the ureteral stone localization system 100 based on multimodal ultrasound may also include components such as... Figure 2 The more or fewer components shown, or having the same Figure 2 The different configurations shown.

[0175] The memory 104 can be used to store software programs and modules for application software, such as the program instructions corresponding to the method embodiments described above in this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-described method for locating ureteral stones based on multimodal ultrasound. The transmission device 106 is used to acquire or send data via a network.

[0176] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

Claims

1. A method for locating ureteral stones based on multimodal ultrasound, characterized in that, The method includes: The system acquires multimodal ultrasound imaging data based on multiple imaging principles. The multimodal ultrasound imaging data based on multiple imaging principles includes continuous imaging sequences at different scanning angles and ultrasound reflection information corresponding to different modes. The multiple imaging principles cover ultrasound imaging methods such as tissue morphology imaging and blood flow signal imaging. The continuous imaging sequences cover the entire ureter and adjacent organs. Based on the anatomical layering structure of the ureter, multimodal ultrasound imaging data of various imaging principles are combined into regions. According to the differences in the anatomical boundaries of the ureteral wall mucosa, muscle layer, adventitia and lumen, tissue identification related signal performance of different modal ultrasound data is fused to obtain combined ultrasound region data divided by anatomical level. The propagation behavior of different modal ultrasound signals corresponding to the combined ultrasound regions was analyzed, including the propagation speed, reflection path and attenuation state of different modal signals in different tissue layers, and the cross-modal signal feature set related to the presence of stones was extracted. A dynamic correlation link between cross-modal signal feature set and stone imaging performance is established. The dynamic correlation link is constructed by the correspondence between cross-modal signal features and stone morphology, internal echo performance, and posterior acoustic shadow performance. The dynamic correlation link adaptively adjusts the correlation logic as imaging conditions change. By combining the anatomical layer locations corresponding to the data from each joint ultrasound region through dynamic correlation links, the cross-modal signal feature distribution status and the spatial relationship of anatomical structures are integrated to generate the location result of the stone in the ureter. The location result includes the anatomical segment where the stone is located, its relative position to the lumen wall, and its distance relationship with surrounding key structures.

2. The method for locating ureteral stones based on multimodal ultrasound according to claim 1, characterized in that, The multimodal ultrasound imaging data based on the anatomical layering structure of the ureter is used for joint region division of multiple imaging principles to obtain joint ultrasound region data divided by anatomical layer, including: The hierarchical classification criteria for the anatomical layered structure of the ureter are obtained. The hierarchical classification criteria are determined based on the differences in the tissue composition of the ureteral wall in human anatomy, the physiological thickness range of each tissue layer, and the spatial morphological characteristics of the lumen. The hierarchical classification criteria cover the division of the mucosa, submucosa, muscularis propria, adventitia, and the core region of the lumen. Imaging region enhancement processing was performed on the multimodal ultrasound imaging data of each imaging principle. The differences in ultrasound reflection signals of tissues at different anatomical levels were highlighted by ultrasound signal amplitude amplification operation, and the interference of background noise on tissue boundaries was reduced by spatial domain filtering operation, resulting in enhanced multimodal ultrasound imaging data. Based on enhanced multimodal ultrasound imaging data, tissue boundary signals are extracted from each type of enhanced multimodal ultrasound imaging data. By analyzing the variation characteristics of the reflection coefficient of ultrasound signals at different tissue interfaces, the boundary positions between different anatomical levels are identified. Tissue boundary signals extracted from ultrasound data of different modalities were cross-compared, and the boundary position signals with the overlap degree meeting the set standard were marked as joint boundary signals; Based on the distribution range and amplitude variation of the joint boundary signal, the common distribution range of each anatomical level in the multimodal ultrasound imaging data of multiple imaging principles is preliminarily defined, and the starting and ending pixel coordinate intervals of each level are marked. Adjust the common distribution range of each anatomical level initially divided, extract the corresponding pixel data, and complete the processing, arrangement and integration of the hierarchical region data to generate joint ultrasound region data divided by anatomical level.

3. The method for locating ureteral stones based on multimodal ultrasound according to claim 2, characterized in that, The adjustment of the common distribution range of each anatomical level initially divided, extraction of corresponding pixel data, and processing, arrangement, and integration of the hierarchical region data generate joint ultrasound region data divided by anatomical level, including: Based on the tissue thickness range and spatial relationship in the hierarchical division standard, the common distribution range of each anatomical level in the initial division is adjusted according to the actual hierarchical distribution of the human ureteral anatomical structure. The range offset caused by imaging angle deviation or signal interference is corrected to obtain the adjusted anatomical level distribution range. Based on the adjusted anatomical hierarchy distribution range, all ultrasound pixel data within each adjusted anatomical hierarchy distribution range are extracted, including the gray value, signal intensity value and phase information of each pixel. The data are then classified and organized according to hierarchy and imaging modality to form initial joint hierarchical region data. The initial joint hierarchical region data is processed by edge processing. The signal aliasing region at the hierarchical boundary is processed by pixel neighborhood similarity analysis. The hierarchical assignment of the boundary pixels is adjusted by comparing the signal features of neighboring pixels, thus obtaining the initial joint hierarchical region data after edge processing. The initial combined hierarchical region data after edge processing are arranged in an orderly manner according to the physiological order of anatomical layering, and the data of mucosa, submucosa, muscle layer and adventitia are arranged outward from the core region of the lumen to form combined hierarchical sequence data; Based on the joint hierarchical sequence data, the relevant information of each imaging modality in the joint hierarchical sequence data is integrated, and the association index between each level of data and ultrasound signals of different modalities is established. Finally, joint ultrasound region data divided according to anatomical level is obtained. The joint ultrasound region data contains independent information of each level, and also retains the correlation between modalities and levels.

4. The method for locating ureteral stones based on multimodal ultrasound according to claim 1, characterized in that, The analysis of the propagation characteristics of different modal ultrasound signals corresponding to the combined ultrasound region data extracts a set of cross-modal signal features related to the presence of stones, including: The propagation path information of different modes of ultrasound corresponding to the data of each combined ultrasound region is obtained. By recording the scanning parameters of the ultrasound imaging equipment and analyzing the imaging geometric model, the propagation direction, path length and tissue medium type of each mode of ultrasound signal in the corresponding ultrasound region are determined, and the determined ultrasound propagation path information is obtained. Based on the determined ultrasonic propagation path information, the signal attenuation state on each mode of ultrasonic propagation path is analyzed. By recording the intensity change of the ultrasonic signal at each point during the propagation process, the signal intensity attenuation curve of each mode is plotted. The intensity loss of the ultrasonic signal from the transmitting end to the receiving end is recorded to obtain the signal intensity attenuation curve of each mode. Based on the signal intensity attenuation curves of each mode, abrupt segments in the signal attenuation state of each mode are extracted. By comparing the signal intensity difference between adjacent propagation path points, propagation path segments with signal intensity differences exceeding a set threshold are identified, thus obtaining the identified abrupt segments. Based on the identified abrupt change segments, signal waveform analysis is performed on the identified abrupt change segments. The ultrasound signal of the abrupt change segments is decomposed into waveforms, and the number of peaks, peak intervals, valley depths, and steepness of the rising and falling edges of the waveforms are extracted to obtain the signal waveform characteristics of the abrupt change segments. By comparing and analyzing the signal waveform performance of abrupt changes, cross-modal waveform features are extracted and key parameters are integrated to construct basic cross-modal signal features and expand them to form a set of cross-modal signal features related to the presence of stones.

5. The method for locating ureteral stones based on multimodal ultrasound according to claim 4, characterized in that, The comparative analysis of the signal waveform performance of the abrupt change segment extracts cross-modal waveform features and integrates key parameters to construct basic cross-modal signal features and expands them to form a set of cross-modal signal features related to the presence of stones, including: Based on the signal waveform performance of the abrupt change segment, the signal waveform performance of the same modal abrupt change segment in different combined ultrasound region data is compared. The abrupt change waveforms of each region are compared one by one, and the overlap and similarity of the waveform performance are statistically analyzed to obtain similar waveform data of the same modality across regions. Based on the signal waveform performance of abrupt transition segments, within the same combined ultrasound region data, the signal waveform performance of abrupt transition segments in different modes is compared, and common features of waveform performance between different modes are extracted to obtain cross-modal waveform common features. By combining similar waveform data across regions within the same mode with common features of cross-modal waveforms, a cross-modal common waveform pattern is formed. Based on this cross-modal common waveform pattern, key parameters are extracted. Waveform analysis tools are used to quantify the waveform's duration, peak interval, amplitude variation range, and symmetry, thus obtaining the key parameters of the cross-modal common waveform pattern. Extract the abrupt change amplitude data in the attenuation state of each modal signal, and integrate the abrupt change amplitude data with the key parameters of the cross-modal common waveform mode to construct the basic cross-modal signal features related to the presence of stones; The basic cross-modal signal features are expanded in terms of feature dimensions to include the frequency variation and phase shift data of each modality of ultrasound signal. The dominant frequency distribution and frequency component changes of the signal are obtained through frequency spectrum analysis, and the phase shift state of the signal is recorded through phase detection. The expanded basic cross-modal signal features are then classified and organized according to feature type, and the correspondence between features is established to form a set of cross-modal signal features related to the presence of stones.

6. The method for locating ureteral stones based on multimodal ultrasound according to claim 1, characterized in that, The establishment of a dynamic correlation link between cross-modal signal feature sets and stone imaging performance includes: Obtain typical data on the imaging manifestation of stones, including the contour morphology, internal echo intensity, internal echo uniformity, posterior acoustic shadow and lateral acoustic shadow of the stones in ultrasound imaging. The typical data of stone imaging are decomposed into features. The typical data are divided into independent feature units by feature decomposition algorithm to obtain stone morphological feature elements and echo feature elements. The morphological feature elements include smooth contour, regular shape, and clear boundary. The echo feature elements include echo intensity, echo uniformity, and echo attenuation rate. Based on the obtained stone morphological and echo characteristics, the correspondence between each signal feature and morphological feature in the cross-modal signal feature set is analyzed. By comparing the performance of each signal feature with the morphological feature, the characterization effect of the signal features on the morphology is defined. Based on the obtained stone morphology features and echo features, the matching status of each signal feature and echo feature in the cross-modal signal feature set is analyzed simultaneously. The matching performance of each signal feature and echo feature is quantified through feature matching analysis to obtain the correlation strength value between signal features and echo. Based on the defined representation function and the obtained correlation strength value, an initial correlation relationship is constructed, and each signal feature is bound to the corresponding morphological feature element and echo feature element to form a one-to-one or many-to-one correspondence. Verify and filter valid initial associations, complete association grouping, action order analysis, link structure construction and dynamic adjustment rule integration, and generate dynamic association links.

7. The method for locating ureteral stones based on multimodal ultrasound according to claim 6, characterized in that, The process of verifying and filtering valid initial associations, completing association grouping, action sequence analysis, link structure construction, and dynamic adjustment rule integration, and generating dynamic association links includes: Multimodal ultrasound imaging data of known stone locations, representing a clinical standard number, were used as validation samples. Initial correlations were applied to the analysis of the validation samples, and valid initial correlations that were stable and correlated in the validation samples were retained. The effective initial correlations are grouped according to the category of signal features. The corresponding effective initial correlations are classified according to the type of signal features to form feature correlation groups. The correlations within each group have similar feature bases. The interaction order between feature association groups was analyzed. By dividing the influence order and interaction mode of different types of signal features on the stone imaging performance, the order of action of each group of association relationships was determined. By connecting the feature association groups in series in the order of basic association first and supplementary association later, a link structure is constructed to form a preliminary dynamic association link; A dynamic correlation link adjustment rule is established using the imaging condition change factor. The adjustment rule is then integrated into the initial dynamic correlation link, allowing the initial dynamic correlation link to adjust the weight and order of action of the correlation relationship according to the changes in imaging conditions, thus obtaining the dynamic correlation link.

8. The method for locating ureteral stones based on multimodal ultrasound according to claim 1, characterized in that, The method involves dynamically linking data from various combined ultrasound regions to anatomical stratification locations, integrating cross-modal signal feature distribution and anatomical spatial relationships to generate the location of the stone in the ureter, including: The imaging manifestations of stones corresponding to each signal feature in the dynamic correlation link are analyzed. Based on the correlation relationship in the dynamic correlation link, the imaging manifestations of stone morphology and echo reflected by each signal feature are divided one by one, and a corresponding mapping table between signal features and imaging manifestations is established. Based on the pixel coordinate index of the joint ultrasound region data, the specific distribution location of each signal feature in the joint ultrasound region data is located, the pixel set corresponding to each signal feature is determined, and the spatial distribution range of the signal feature in the ultrasound image is delineated. By combining the anatomical layer locations corresponding to the data from each combined ultrasound region, and using a coordinate transformation model between anatomical layers and ultrasound images, the pixel coordinates of the signal feature distribution locations are mapped to the anatomical layer structure of the ureter, thus obtaining the anatomical location coordinates corresponding to the signal features. Based on the anatomical coordinates corresponding to the signal features, the distribution density and intensity concentration of the signal features are analyzed, and the anatomical location with the most concentrated signal features is extracted from the mapped location information as the core location point. Based on the core location point, we analyze the specific hierarchical affiliation of the core location point in the anatomical layered structure, and classify the core location point as being located in the lumen core region, mucosa layer or other layers of the ureter. Combined with the physiological location of the stone, we define the location range. A three-dimensional anatomical model of the core location point was established, and the credibility assessment of the core location point, the delineation of the stone extension range, the integration of the location description, and the generation of stone location results were completed.

9. The method for locating ureteral stones based on multimodal ultrasound according to claim 8, characterized in that, The process of establishing a three-dimensional anatomical model of the core location points, completing the reliability assessment of the core location points, defining the extent of stone extension, integrating location descriptions, and generating stone location results includes: By integrating the anatomical coordinates and hierarchical information of the core location points, a three-dimensional anatomical positioning model of the core location points is established. The three-dimensional anatomical positioning model reflects the spatial position of the core location points in the ureteral anatomy. By combining the correlation strength values ​​between signal characteristics and stone imaging performance in the dynamic correlation link, the reliability of the location of the core location point is evaluated, and the core location points that meet the set reliability standards are retained. Based on the retained core location point, extract the signal feature change state within a set range around the retained core location point, analyze the signal feature distribution state around the core location point, and delineate the possible extension range and morphological boundary of the stone. By comprehensively integrating the core location points, hierarchical classification, three-dimensional anatomical positioning model and extended range information, a complete positioning description of the stone is constructed. The complete positioning description includes the central location of the stone, the anatomical segment in which it is located, the distance from the lumen wall, and the approximate morphological range. The location of the stone in the ureter is generated based on a complete localization description. The localization result adopts a standardized anatomical localization representation method and includes a two-way mapping relationship between ultrasound image coordinates and anatomical structure coordinates.

10. A ureteral stone localization system based on multimodal ultrasound, characterized in that, The device includes a processor and a readable storage medium storing a program that, when executed by the processor, implements the method for locating ureteral stones based on multimodal ultrasound as described in any one of claims 1-9.

Images