Vascular ultrasound image recognition method based on inter-frame pose increment

Abstract

The invention provides a vascular ultrasound image recognition method based on inter-frame pose increment. According to the method, a target blood vessel area is scanned through an ultrasonic probe provided with an IMU sensor, a multi-frame continuous blood vessel ultrasonic image sequence is obtained, and motion data corresponding to each frame of image is synchronously collected. And estimating the pose increment between the adjacent image frames based on the acquired acceleration and angular velocity information. When a vascular ultrasound image classification model is constructed, inter-frame pose increment is introduced to guide calculation of prediction result consistency loss, so that a total loss function is constructed in combination with space-time continuity features, and the discrimination stability of the model in a continuous image sequence is improved. And through the classification model obtained through training, an image sequence of a new patient can be identified, and finally intelligent vascular disease identification under multi-frame information fusion is realized. According to the method, the understanding capability of the model on the ultrasonic image sequence space structure is improved, and the method is suitable for auxiliary diagnosis in a dynamic and multi-angle scanning scene in clinic.

Description

Technical Field

[0001] This invention belongs to the field of vascular ultrasound image recognition technology, specifically relating to a vascular ultrasound image recognition method based on inter-frame pose increment. Background Technology

[0002] Ultrasound imaging, as a non-invasive, real-time, and easy-to-operate medical imaging method, has been widely used in the clinical detection and early screening of vascular diseases. In practice, doctors typically use a handheld ultrasound probe to scan the target vascular area from multiple angles, combining the real-time imaging results to determine the morphology, structure, and lesion status of the blood vessels. However, the interpretation of vascular ultrasound images heavily relies on the doctor's subjective experience. Factors such as operator position, probe angle, and acquisition path can interfere with diagnostic results, often leading to significant discrepancies and poor repeatability. Therefore, how to achieve automatic identification and classification of vascular ultrasound images has become a current research hotspot.

[0003] In recent years, with the continuous development of deep learning technology, research has applied convolutional neural networks (CNN), recurrent neural networks (RNN), and attention mechanisms to the automatic recognition of ultrasound images to improve the accuracy and automation level of recognition. For example, Chinese patent CN113763309A discloses a method for target recognition and tracking of liver blood vessels in ultrasound images based on an improved U-Net network and an LSTM network. The method includes: Step 1: Preprocessing the ultrasound image sequence; Step 2: Training a ROI extraction model to segment regions from the ultrasound image in Step 1; Step 3: Accurately segmenting the target in the region in Step 2 based on the improved U-Net network; Step 4: Classifying the ultrasound image sequence in Step 1 using a CNN-LSTM network; Step 5: Based on the segmentation results in Step 3 and the classification results in Step 4, accurately predicting the target location in the ultrasound image sequence using an LSTM network.

[0004] Although existing technologies have made beneficial explorations in image sequence processing and multi-frame temporal modeling, the following limitations still exist: 1. Most methods classify and judge based on single-frame images, failing to effectively utilize the temporal and spatial correlations between image frames; 2. Single-frame images are often incomplete in information and unstable in features due to factors such as image blurring, probe angle changes, and tissue occlusion, which can easily lead to model misjudgment; 3. In actual ultrasound scanning, the ultrasound probe usually moves in a continuous posture in the vascular region, acquiring a series of continuous frame images. This temporal and spatial continuity has not been fully utilized by existing classification methods, resulting in insufficient robustness of the model when processing dynamic image sequences.

[0005] Therefore, how to effectively integrate the temporal continuity and spatial geometric relationship of image sequences in vascular ultrasound image recognition tasks to improve the stability and accuracy of image classification has become a technical challenge that urgently needs to be solved in the current field. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for vascular ultrasound image recognition based on inter-frame pose increment.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] This invention provides a method for vascular ultrasound image recognition based on inter-frame pose increment, comprising the following steps:

[0009] The target vascular region is scanned by an ultrasound probe to obtain a sequence of multiple consecutive vascular ultrasound images of the same patient, and motion data of each corresponding image is synchronously acquired by an IMU sensor set on the ultrasound probe.

[0010] Based on the motion data acquired by the IMU sensor, the pose increment of the ultrasonic probe between adjacent frame image acquisition times is estimated.

[0011] A vascular ultrasound image classification model is constructed for vascular ultrasound image classification. Based on the pose increment between adjacent image frames and the acquisition time of adjacent image frames in the multi-frame images, a loss function for the vascular ultrasound image classification model is constructed. The multi-frame continuous vascular ultrasound image sequence is used as training samples to input into the vascular ultrasound image classification model for training, and the trained vascular ultrasound image classification model is obtained.

[0012] The recognition of vascular ultrasound images is achieved based on a trained vascular ultrasound image classification model.

[0013] Furthermore, the step of scanning the target vascular region with an ultrasound probe to acquire a sequence of multiple consecutive vascular ultrasound images of the same patient specifically includes:

[0014] Medical staff use an ultrasound probe to scan the target blood vessel area from multiple angles. The ultrasound probe records the blood vessel ultrasound image frames during the scanning process at a first preset time interval, and adds timestamp information to each image frame to obtain a sequence of multiple consecutive blood vessel ultrasound images.

[0015] Furthermore, the motion data includes acceleration vector, angular velocity vector, and linear velocity vector.

[0016] Furthermore, estimating the pose increment of the ultrasonic probe between adjacent image acquisition times based on the motion data acquired by the IMU sensor specifically includes:

[0017] The IMU sensor at adjacent image frame acquisition times t i With t i+1 Acceleration data recorded between a k With angular velocity data ω k Denoising, gravity compensation, and temperature compensation are performed using a pre-integral model within the time interval [t] between two frames. i ,t i+1 ] Calculate the rotation increment ΔR between two frames of images. i With translation increment Δp i :

[0018]

[0019] Where, ΔR i Δp represents the rotation increment matrix between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence acquired by the ultrasound probe; i ω represents the translation increment vector between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence acquired by the ultrasound probe. k v represents the angular velocity vector recorded by the IMU sensor at time k. k Let Δt represent the linear velocity vector recorded by the IMU sensor at time k, where Δt = t i+1 -t i For the first preset time interval, exp represents the exponential mapping function on the Lie group SO(3);

[0020] The rotation increment matrix ΔR i With translation increment vector Δp i The combination constitutes the inter-frame pose increment.

[0021] Furthermore, the rotation increment matrix ΔR i With translation increment vector Δp i The combination constitutes the inter-frame pose increment, and the formula is:

[0022] ||ΔT i ||=||log(ΔR i )||+λ·||Δp i ||

[0023] Where, ||ΔT i || represents the pose increment between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence, ||log(ΔR) i )|| represents the rotation increment matrix ΔR i The Euclidean norm of the Lie algebra vector obtained after taking the logarithmic mapping, ||Δp i || represents the translation increment vector Δp iThe Euclidean norm, where λ is the preset weighting coefficient.

[0024] Furthermore, the vascular ultrasound image classification model is an image classification model built on a deep convolutional neural network, using an improved ResNet50 network structure. The improvements include replacing the standard convolution operation with a depthwise separable convolution and introducing an attention mechanism in the residual block.

[0025] Furthermore, the step of using the multi-frame continuous vascular ultrasound image sequence as training samples to train the vascular ultrasound image classification model specifically includes:

[0026] Multiple sets of continuous vascular ultrasound image sequences of target vascular regions are obtained from multiple different patients. Each set of image sequences includes multiple vascular ultrasound image frames acquired sequentially in time, and a uniform vascular disease category label is assigned to each set of image sequences.

[0027] Adjacent vascular ultrasound image frames belonging to the same patient in each image sequence are sequentially input into the vascular ultrasound image classification model in chronological order to form a training sample sequence, enabling the model to learn the spatial features of the image sequence as it evolves over time and its corresponding category label.

[0028] Using image sequences labeled with category tags as supervision information, the model's prediction results for each frame of the image are obtained through forward propagation;

[0029] The loss function value is calculated based on the prediction results of each frame of image, and the pose increment information between adjacent frames is combined to construct a loss function that includes classification error and prediction consistency constraints.

[0030] Based on the loss function value, the parameters of the vascular ultrasound image classification model are optimized using the backpropagation algorithm to complete model training.

[0031] Furthermore, the loss function of the vascular ultrasound image classification model is:

[0032]

[0033] in, Let N be the loss function for the vascular ultrasound image classification model, and N be the total number of images in the training samples. Let α be the classification loss for the i-th frame image, and α be a preset parameter, ||ΔT i || represents the pose increment between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence. The loss is the consistency loss of prediction results for adjacent frames, where ε>0 is a preset coefficient.

[0034] Furthermore, the classification loss for:

[0035]

[0036] Where C represents the total number of vascular disease categories in the vascular image classification task, and y i,c This represents the true label of the i-th frame image in category c; it is 1 if it belongs to that category, and 0 otherwise. Let be the predicted probability of the vascular ultrasound image classification model for the i-th frame image in category c.

[0037] The consistency loss of the prediction results for:

[0038]

[0039] in, This represents the predicted probability of the vascular ultrasound image classification model for the (i+1)th frame image in category c.

[0040] Furthermore, the recognition of vascular ultrasound images based on the trained vascular ultrasound image classification model specifically includes:

[0041] For patients to be tested, an ultrasound probe equipped with an IMU sensor is used to perform multi-angle continuous scanning of the vascular region to be tested, thereby acquiring a sequence of multi-frame continuous vascular ultrasound images of the patient and simultaneously recording the IMU motion data corresponding to each frame.

[0042] The IMU motion data is preprocessed, including denoising, gravity compensation and pre-integration, to estimate the inter-frame pose increment between any adjacent image frames in the multi-frame continuous vascular ultrasound image sequence to be detected.

[0043] The sequence of multiple consecutive vascular ultrasound images to be detected is sequentially input into the trained vascular ultrasound image classification model according to the acquisition order to obtain the predicted probability vector of each image for each vascular disease category. in, This represents the predicted probability of the i-th frame image in the c-th category, where C is the total number of vascular disease categories;

[0044] Calculate the difference in predicted probability vectors between adjacent frames:

[0045]

[0046] Where, d i This represents the Euclidean distance between the predicted probability vectors of frame i and frame i+1.

[0047] Calculate the weight of the i-th frame based on the difference in predicted probability vectors between adjacent frames and the pose increment:

[0048]

[0049] Where γ,ε>0 are preset coefficients, ω i Let be the weight of the i-th frame;

[0050] Based on the weights of each frame in the sequence of multiple consecutive vascular ultrasound images to be detected, a weighted score for each category is calculated:

[0051]

[0052] Among them, s c The weighted score for category c, where M is the total number of images in the multi-frame continuous vascular ultrasound image sequence to be detected;

[0053] The category with the highest weighted score is selected as the final identification result of the vascular ultrasound image sequence of the patient to be tested.

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] (1) The loss function constructed in this invention introduces a consistency constraint on prediction results based on inter-frame pose increments, on the basis of traditional image classification loss. The principle is to use the spatial continuity and pose change amplitude between image frames to reasonably guide the model to maintain the prediction stability of spatiotemporally adjacent images during training. Specifically, when the pose change between two ultrasound images is small, it means that they are spatially adjacent and the image content is highly similar. If the model's prediction results differ significantly, it should be considered unreasonable, and the consistency loss should be used to increase the penalty, forcing the model to learn more consistent and smooth discrimination boundaries in continuous images. When the pose change between frames is large, since the image content may change more drastically, a certain degree of difference in prediction results is allowed. Therefore, the influence of the consistency loss should be appropriately reduced to avoid the model overfitting local features. This invention uses the squared term of the prediction difference between adjacent image frames as the consistency loss and adds it to the total loss function inversely proportional to the norm of the inter-frame pose increment, so that the model automatically adjusts the constraint strength of prediction consistency during training, reflecting the spatial prior logic that similar poses should predict similar results. This design addresses the problem in existing deep learning models that lack spatiotemporal structure awareness of continuous ultrasound image sequences, avoiding prediction jitter or instability caused by training based solely on static frames. By introducing a loss regulation mechanism related to pose increments, this invention effectively improves the robustness and consistency of the model in continuous image discrimination, making it more suitable for dynamic image recognition tasks in actual clinical scanning processes, thereby improving recognition accuracy and the model's ability to understand the spatial structure of image sequences.

[0056] (2) This invention integrates an IMU sensor on an ultrasonic probe to acquire acceleration and angular velocity data in real time during the acquisition of each frame of image. Combined with pre-integration, noise reduction and gravity compensation processing methods, the rotation and translation changes between any two frames of image are estimated to construct the inter-frame pose increment. This pose increment not only describes the spatial motion relationship of the image during the acquisition process, but also serves as an objective and continuous geometric index to quantify the degree of spatial similarity between adjacent image frames.

[0057] (3) In the process of identifying patients to be tested, this invention effectively solves the key technical problems in the prior art, such as unstable identification results due to reliance on single-frame images, sensitivity to image noise or instantaneous image blur, and inability to utilize the spatiotemporal information contained in the image sequence, by combining multiple frames of vascular ultrasound images instead of relying on a single frame image for identification. In traditional methods, ultrasound image recognition is often based on single-frame images for classification and judgment, ignoring the correlation between consecutive frames in the image sequence. Especially during dynamic scanning, due to hand-held operation jitter or temporary changes in image content, the model output is prone to fluctuations, resulting in misjudgments. This invention performs joint processing of consecutive multi-frame images in the model and constructs a weighted fusion strategy based on the prediction results of adjacent frames, the pose increment between images, and prediction stability, thereby fully exploring the redundant information and structural consistency in the image sequence. Specifically, in the final identification stage, a frame weight mechanism based on prediction consistency and pose change adjustment is introduced, enabling the model to dynamically adjust the degree of attention to each frame, weaken the influence of abnormal frames or frames with uncertain predictions, and highlight the prediction contribution of stable and representative frames. This strategy effectively improves the robustness of identification decisions and avoids the problem of misleading judgments due to "isolated frames." It significantly enhances the stability and accuracy of identification results, enabling the model to better adapt to ultrasound image sequences acquired under unstructured and unstable conditions in clinical settings, thereby increasing the algorithm's practicality and promotional value.

[0058] (4) The weights constructed in this invention when calculating the weighted score solve the problem that traditional multi-frame ultrasound image recognition cannot effectively distinguish the image quality and information contribution of each frame. Since the ultrasound probe is handheld by medical staff for scanning, there are significant differences in pose changes and image quality between different frames during the actual acquisition process, which may lead to unstable prediction results or large errors in some frames. If the prediction results of all frames are simply averaged or equally weighted, noise interference is easily introduced, reducing the overall accuracy and robustness of recognition. By calculating the weights based on the difference in prediction probabilities between adjacent frames and the pose increment between frames, this invention can adaptively adjust the contribution of each frame image in the final category judgment. Specifically, when the prediction results of adjacent frames differ greatly and the pose increment is small, it indicates that the prediction of that frame is unstable and the image content changes little, and the weight of that frame should be reduced to reduce misleading; conversely, when the prediction results are consistent and the pose increment is large, a higher weight is given to enhance support for the true category. This weight design realizes dynamic weighted fusion of multi-frame continuous ultrasound image sequences, effectively suppressing the negative impact of abnormal frames and improving the model's ability to understand and utilize spatiotemporally continuous image information. The technical results are manifested in a significant improvement in overall recognition accuracy, and the model exhibits stronger stability and robustness in actual clinical dynamic scanning, thereby meeting the clinical demand for real-time and accurate vascular ultrasound image recognition. Attached Figure Description

[0059] Figure 1 This is a flowchart of the vascular ultrasound image recognition method according to an embodiment of the present invention;

[0060] Figure 2 This is a structural diagram of the vascular ultrasound image classification model according to an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0062] Example 1:

[0063] This embodiment provides a method for vascular ultrasound image recognition based on inter-frame pose increment, such as... Figure 1 As shown, it includes the following steps:

[0064] Step S1: Scan the target vascular region with an ultrasound probe to acquire a sequence of multiple consecutive vascular ultrasound images of the same patient, specifically including:

[0065] Medical staff use an ultrasound probe to scan the target blood vessel area from multiple angles. The ultrasound probe records the blood vessel ultrasound image frames during the scanning process at a first preset time interval, and adds timestamp information to each image frame to obtain a sequence of multiple consecutive blood vessel ultrasound images.

[0066] Step S2: Simultaneously acquire motion data for each frame of the image using an IMU sensor mounted on the ultrasound probe. The motion data includes acceleration vector, angular velocity vector, and linear velocity vector.

[0067] Step S3: Based on the motion data acquired by the IMU sensor, estimate the pose increment of the ultrasonic probe between adjacent image acquisition times, specifically including:

[0068] The IMU sensor at adjacent image frame acquisition times t i With t i+1 Acceleration data recorded between a k With angular velocity data ω k Denoising, gravity compensation, and temperature compensation are performed using a pre-integral model within the time interval [t] between two frames. i ,t i+1 ] Calculate the rotation increment ΔR between two frames of images. i With translation increment Δp i :

[0069]

[0070] Where, ΔR i Δp represents the rotation increment matrix between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence acquired by the ultrasound probe; i ω represents the translation increment vector between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence acquired by the ultrasound probe. k v represents the angular velocity vector recorded by the IMU sensor at time k. k Let Δt represent the linear velocity vector recorded by the IMU sensor at time k, where Δt = t i+1 -t i For the first preset time interval, exp represents the exponential mapping function on the Lie group SO(3);

[0071] The rotation increment matrix ΔR i With translation increment vector Δp i The combination constitutes the inter-frame pose increment, and the formula is:

[0072] ||ΔT i ||=||log(ΔR i )||+λ·||Δp i ||

[0073] Where, ||ΔT i || represents the pose increment between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence, ||log(ΔR) i )|| represents the rotation increment matrix ΔR i The Euclidean norm of the Lie algebra vector obtained after taking the logarithmic mapping, ||Δp i || represents the translation increment vector Δp i The Euclidean norm, where λ is the preset weighting coefficient.

[0074] Because medical staff typically hold the ultrasound probe during vascular ultrasound examinations and freely scan the target vascular region from multiple angles, even if image acquisition is performed within fixed time intervals, uncertainties in human operation, such as hand tremors, changes in rotation angles, and non-uniform motion, lead to significant differences in the actual spatial position and orientation between adjacent image frames at the acquisition time. Without external positioning information, it is difficult to accurately determine the spatial correlation and continuity of image content between frames solely based on the temporal order of the image frame sequence, thus affecting the model's learning effect on image temporal features and spatial structure. This invention integrates an IMU sensor and introduces a pre-integration method to recover the inter-frame pose changes during image acquisition from the acceleration and angular velocity data recorded by the IMU. This pose increment not only provides a quantitative description of the spatial transformation between adjacent image frames but also provides a reliable basis for the subsequent construction of prediction consistency constraints, dynamic weighting of image sequences, and model spatial structure perception. This design significantly enhances the model's ability to model the spatial relationships between image frames, effectively improving the accuracy and stability of vascular ultrasound image recognition, and is particularly suitable for dynamic, non-fixed-path actual scanning environments in clinical settings.

[0075] Step S4: Construct a vascular ultrasound image classification model for vascular ultrasound image classification, such as... Figure 2 As shown, the vascular ultrasound image classification model is an image classification model built on a deep convolutional neural network, employing an improved ResNet50 network structure. The improvements include replacing standard convolution operations with depthwise separable convolution (BTNK). BTNK decomposes standard convolution into depthwise convolution and pointwise convolution, reducing computational cost and the number of parameters, improving feature extraction efficiency, reducing computational resource consumption, and maintaining the model's expressive power. Attention mechanisms, such as channel attention and spatial attention, are introduced into the residual blocks, enhancing the focus on important features, improving feature extraction accuracy, enabling the model to better capture key features, and improving recognition performance.

[0076] Step S5: Based on the pose increment between adjacent image frames and the acquisition time of adjacent image frames in multiple frames, construct the loss function of the vascular ultrasound image classification model. The loss function of the vascular ultrasound image classification model is:

[0077]

[0078] in, Let N be the loss function for the vascular ultrasound image classification model, and N be the total number of images in the training samples. Let α be the classification loss for the i-th frame image, and α be a preset parameter, ||ΔT i || represents the pose increment between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence. The loss is the consistency loss of prediction results for adjacent frames, where ε>0 is a preset coefficient.

[0079] Classification loss for:

[0080]

[0081] Where C represents the total number of vascular disease categories in the vascular image classification task, and y i,c This represents the true label of the i-th frame image in category c; it is 1 if it belongs to that category, and 0 otherwise. Let be the predicted probability of the vascular ultrasound image classification model for the i-th frame image in category c.

[0082] Loss of consistency in prediction results for:

[0083]

[0084] in, This represents the predicted probability of the vascular ultrasound image classification model for the (i+1)th frame image in category c.

[0085] For adjacent frames, since they are typically acquired consecutively within a very short time interval, and if the pose increment is small, it indicates that the spatial position and viewpoint changes between the two frames are small, and the image content should be highly similar. Therefore, ideally, the model's prediction results for these two frames should be consistent or similar. If the model's predicted probabilities differ significantly, it indicates that its discrimination boundary in temporally continuous images is unstable, and there may be problems such as prediction jitter, overfitting, or insufficient training. Based on this, this invention introduces a prediction result consistency loss into the loss function. Dividing this by the norm of the inter-frame pose increment enables dynamic adjustment of the consistency error: when the inter-frame pose change is small, i.e., ||ΔT iIf the change in image content is small, the impact of consistency error is amplified, and the model will be forced to learn to output smoother and more consistent predictions. However, if the pose change is large, i.e., ||ΔT|| i The model allows for predictive discrepancies within a reasonable range, thus avoiding weakening the model's sensitivity to key changes due to forced smoothing.

[0086] This design effectively integrates spatiotemporal information from IMU perception with image prediction stability, addressing the issue of existing deep learning models neglecting inter-frame spatial continuity during training. It enhances the model's robustness in understanding and discriminating the spatial structure of continuous vascular ultrasound image sequences, ultimately improving its recognition stability and prediction consistency across consecutive image frames. This significantly reduces prediction jitter caused by minor changes in pose between frames, improving the model's clinical applicability and practical performance in dynamic ultrasound scanning. Through this design, the model can not only accurately classify single-frame images but also effectively utilize spatial prior information between adjacent frames to achieve collaborative modeling and robust discrimination of multiple frames, thus better serving the needs of automatic identification and assisted diagnosis of actual vascular diseases.

[0087] Step S6: Input a multi-frame continuous vascular ultrasound image sequence as training samples into the vascular ultrasound image classification model to obtain the trained vascular ultrasound image classification model, specifically including:

[0088] Multiple sets of continuous vascular ultrasound image sequences of the target vascular region were obtained from multiple different patients. Each set of image sequences included multiple vascular ultrasound image frames acquired in time and each set of image sequences was assigned a uniform vascular disease category label.

[0089] Adjacent vascular ultrasound image frames belonging to the same patient in each image sequence are sequentially input into the vascular ultrasound image classification model in chronological order to form a training sample sequence, enabling the model to learn the spatial features of the image sequence as it evolves over time and its corresponding category label.

[0090] Using image sequences labeled with category tags as supervision information, the model's prediction results for each frame of the image are obtained through forward propagation;

[0091] The loss function value is calculated based on the prediction results of each frame of image, and the pose increment information between adjacent frames is combined to construct a loss function that includes classification error and prediction consistency constraints.

[0092] Based on the loss function value, the parameters of the vascular ultrasound image classification model are optimized using the backpropagation algorithm to complete the model training.

[0093] Step S7: Recognize vascular ultrasound images based on the trained vascular ultrasound image classification model, specifically including:

[0094] For patients to be tested, an ultrasound probe equipped with an IMU sensor is used to perform multi-angle continuous scanning of the vascular region to be tested, thereby acquiring a sequence of multi-frame continuous vascular ultrasound images of the patient and simultaneously recording the IMU motion data corresponding to each frame.

[0095] The IMU motion data is preprocessed, including denoising, gravity compensation and pre-integration, to estimate the inter-frame pose increment between any adjacent image frames in the multi-frame continuous vascular ultrasound image sequence to be detected.

[0096] The sequence of multiple consecutive vascular ultrasound images to be detected is sequentially input into the trained vascular ultrasound image classification model according to the acquisition order to obtain the predicted probability vector of each image for each vascular disease category. in, This represents the predicted probability of the i-th frame image in the c-th category, where C is the total number of vascular disease categories;

[0097] Calculate the difference in predicted probability vectors between adjacent frames:

[0098]

[0099] Where, d i This represents the Euclidean distance between the predicted probability vectors of frame i and frame i+1.

[0100] Calculate the weight of the i-th frame based on the difference in predicted probability vectors between adjacent frames and the pose increment:

[0101]

[0102] Where γ,ε>0 are preset coefficients, ω i Let be the weight of the i-th frame;

[0103] Based on the weights of each frame in the sequence of multiple consecutive vascular ultrasound images to be detected, a weighted score for each category is calculated:

[0104]

[0105] Among them, s c The weighted score for category c, where M is the total number of images in the multi-frame continuous vascular ultrasound image sequence to be detected;

[0106] Select weighted score s c The highest category is used as the final identification result of the ultrasound image sequence of the patient's blood vessels.

[0107] This invention achieves disease identification of the target vascular region in a patient by combining multiple frames of images, rather than relying solely on a single frame, significantly improving the stability and accuracy of the identification. In actual clinical examinations, ultrasound images are easily affected by factors such as probe jitter, changes in scanning angle, and local noise interference. A single frame may not fully reflect the overall characteristics of the lesion area, easily leading to misdiagnosis. This invention dynamically adjusts the weight allocation of each frame by fusing the prediction results of multiple frames and combining the pose increment and prediction consistency between adjacent images. This suppresses the interference of unstable prediction images on the final identification result, enhancing the model's robustness and spatiotemporal structure modeling capabilities in dynamic scanning scenarios. In particular, by jointly considering prediction differences and pose increments for weighted discrimination, the diagnostic value of each frame in the entire sequence can be more reasonably reflected, achieving more reliable vascular disease identification and effectively meeting the actual clinical needs for a highly stable and accurate automatic identification system.

[0108] Example 2:

[0109] This embodiment provides a vascular ultrasound image recognition system based on inter-frame pose increment, including:

[0110] The image acquisition module is used to perform multi-angle scanning of the target vascular region using an ultrasound probe equipped with an IMU sensor, acquire multiple consecutive vascular ultrasound image sequences of the same patient, and synchronously record the IMU motion data corresponding to each frame, including acceleration and angular velocity information.

[0111] The motion estimation module is used to preprocess the raw motion data acquired by the IMU sensor, including denoising, gravity compensation and pre-integration, and then estimate the inter-frame pose increment between image frames, output the rotation increment matrix and translation increment vector between image frames, and combine them into a pose increment index.

[0112] The classification model building module is used to build a vascular ultrasound image classification model based on a deep neural network. It receives image sequences and their label information, and incorporates pose increments into the model training.

[0113] The loss function generation module constructs a loss function with consistency constraints based on the differences in prediction results between adjacent frames and the corresponding inter-frame pose increments in multi-frame images. This loss function combines the prediction consistency loss term with the classification cross-entropy loss and adjusts the weights according to the pose increments to ensure the model has prediction stability on spatially continuous images.

[0114] The model training module is used to input the image sequences and their labels provided by the image acquisition module as training samples into the classification model, train it using the loss function mentioned above, update the model parameters, and obtain the trained vascular ultrasound image classification model.

[0115] The image recognition module is used to identify image sequences of patients to be tested after the model has been trained. This module dynamically calculates frame weights and performs weighted discrimination based on the predicted probability of each frame for each disease category, combined with inter-frame prediction differences and pose increments, and outputs the final recognition result.

[0116] This system achieves deep integration of vascular ultrasound image sequences with pose information during training and inference, improving the stability of the recognition model and the accuracy of clinical applications. It is suitable for intelligent assisted diagnosis of ultrasound images in dynamic scenarios.

[0117] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for vascular ultrasound image recognition based on inter-frame pose increment, characterized in that, Includes the following steps: The target vascular region is scanned by an ultrasound probe to obtain a sequence of multiple consecutive vascular ultrasound images of the same patient, and motion data of each corresponding image is synchronously acquired by an IMU sensor set on the ultrasound probe. Based on the motion data acquired by the IMU sensor, the pose increment of the ultrasonic probe between adjacent frame image acquisition times is estimated. A vascular ultrasound image classification model is constructed for vascular ultrasound image classification. Based on the pose increment between adjacent image frames and the acquisition time of adjacent image frames in the multi-frame images, a loss function for the vascular ultrasound image classification model is constructed. The multi-frame continuous vascular ultrasound image sequence is used as training samples to input into the vascular ultrasound image classification model for training, and the trained vascular ultrasound image classification model is obtained. The recognition of vascular ultrasound images is achieved based on a trained vascular ultrasound image classification model.

2. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 1, characterized in that, The step of scanning the target vascular region with an ultrasound probe to acquire a sequence of multiple consecutive vascular ultrasound images of the same patient specifically includes: Medical staff use an ultrasound probe to scan the target blood vessel area from multiple angles. The ultrasound probe records the blood vessel ultrasound image frames during the scanning process at a first preset time interval, and adds timestamp information to each image frame to obtain a sequence of multiple consecutive blood vessel ultrasound images.

3. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 1, characterized in that, The motion data includes acceleration vector, angular velocity vector, and linear velocity vector.

4. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 1, characterized in that, The estimation of the pose increment of the ultrasonic probe between adjacent image acquisition times based on the motion data acquired by the IMU sensor specifically includes: The IMU sensor at adjacent image frame acquisition times t i With t i+1 Acceleration data recorded between a k With angular velocity data ω k Denoising, gravity compensation, and temperature compensation are performed using a pre-integral model within the time interval [t] between two frames. i ,t i+1 ] Calculate the rotation increment ΔR between two frames of images. i With translation increment Δp i : Where, ΔR i Δp represents the rotation increment matrix between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence acquired by the ultrasound probe; i ω represents the translation increment vector between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence acquired by the ultrasound probe. k v represents the angular velocity vector recorded by the IMU sensor at time k. k Let Δt represent the linear velocity vector recorded by the IMU sensor at time k, where Δt = t i+1 -t i For the first preset time interval, exp represents the exponential mapping function on the Lie group SO(3); The rotation increment matrix ΔR i With translation increment vector Δp i The combination constitutes the inter-frame pose increment.

5. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 4, characterized in that, The rotation increment matrix ΔR i With translation increment vector Δp i The combination constitutes the inter-frame pose increment, and the formula is: ||ΔT i ||=||log(ΔR i )||+λ·||Δp i || Where, ||ΔT i || represents the pose increment between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence, ||log(ΔR) i )|| represents the rotation increment matrix ΔR i The Euclidean norm of the Lie algebra vector obtained after taking the logarithmic mapping, ||Δp i || represents the translation increment vector Δp i The Euclidean norm, where λ is the preset weighting coefficient.

6. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 1, characterized in that, The vascular ultrasound image classification model is an image classification model based on a deep convolutional neural network, using an improved ResNet50 network structure. The improvements include replacing the standard convolution operation with a depthwise separable convolution and introducing an attention mechanism in the residual block.

7. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 1, characterized in that, The step of using the multi-frame continuous vascular ultrasound image sequence as training samples to train the vascular ultrasound image classification model specifically includes: Multiple sets of continuous vascular ultrasound image sequences of the target vascular region were obtained from multiple different patients. Each set of image sequences included multiple vascular ultrasound image frames acquired in time and each set of image sequences was assigned a uniform vascular disease category label. Adjacent vascular ultrasound image frames belonging to the same patient in each image sequence are sequentially input into the vascular ultrasound image classification model in chronological order to form a training sample sequence, enabling the model to learn the spatial features of the image sequence as it evolves over time and its corresponding category label. Using image sequences labeled with category tags as supervision information, the model's prediction results for each frame of the image are obtained through forward propagation; The loss function value is calculated based on the prediction results of each frame of image, and the pose increment information between adjacent frames is combined to construct a loss function that includes classification error and prediction consistency constraints. Based on the loss function value, the parameters of the vascular ultrasound image classification model are optimized using the backpropagation algorithm to complete model training.

8. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 1, characterized in that, The loss function of the vascular ultrasound image classification model is: in, Let N be the loss function for the vascular ultrasound image classification model, and N be the total number of images in the training samples. Let α be the classification loss for the i-th frame image, and α be a preset parameter, ||ΔT i || represents the pose increment between image frame i and image frame i+1 in a multi-frame continuous vascular ultrasound image sequence. The loss is the consistency loss of prediction results for adjacent frames, where ε>0 is a preset coefficient.

9. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 8, characterized in that, The classification loss for: Where C represents the total number of vascular disease categories in the vascular image classification task, and y i,c This represents the true label of the i-th frame image in category c; it is 1 if it belongs to that category, and 0 otherwise. Let be the predicted probability of the vascular ultrasound image classification model for the i-th frame image in category c. The consistency loss of the prediction results for: in, This represents the predicted probability of the vascular ultrasound image classification model for the (i+1)th frame image in category c.

10. The vascular ultrasound image recognition method based on inter-frame pose increment according to claim 1, characterized in that, The trained vascular ultrasound image classification model enables the recognition of vascular ultrasound images, specifically including: For patients to be tested, an ultrasound probe equipped with an IMU sensor is used to perform multi-angle continuous scanning of the vascular region to be tested, thereby acquiring a sequence of multi-frame continuous vascular ultrasound images of the patient and simultaneously recording the IMU motion data corresponding to each frame. The IMU motion data is preprocessed, including denoising, gravity compensation and pre-integration, to estimate the inter-frame pose increment between any adjacent image frames in the multi-frame continuous vascular ultrasound image sequence to be detected. The sequence of multiple consecutive vascular ultrasound images to be detected is sequentially input into the trained vascular ultrasound image classification model according to the acquisition order to obtain the predicted probability vector of each image for each vascular disease category. in, This represents the predicted probability of the i-th frame image in the c-th category, where C is the total number of vascular disease categories; Calculate the difference in predicted probability vectors between adjacent frames: Where, d i This represents the Euclidean distance between the predicted probability vectors of frame i and frame i+1. Calculate the weight of the i-th frame based on the difference in predicted probability vectors between adjacent frames and the pose increment: Where γ,ε>0 are preset coefficients, ω i Let be the weight of the i-th frame; Based on the weights of each frame in the sequence of multiple consecutive vascular ultrasound images to be detected, a weighted score for each category is calculated: Among them, s c The weighted score for category c, where M is the total number of images in the multi-frame continuous vascular ultrasound image sequence to be detected; Select weighted score s c The highest category is used as the final identification result of the ultrasound image sequence of the patient's blood vessels.

Images