Total carotid artery wall motion measurement method and device of layered state space model

Abstract

The invention provides a common carotid artery wall motion measurement method and device based on a hierarchical state space model. The method comprises the following steps: establishing a corresponding relation between common carotid artery wall ultrasonic data and a common carotid artery wall motion estimation result through an artificial intelligence model; specifically, a first corresponding sub-relation between common carotid artery wall ultrasonic data and a similarity graph is established through a first artificial intelligence sub-model; establishing a second corresponding sub-relationship between the common carotid artery wall ultrasonic data and the common carotid artery wall segmentation information through a second artificial intelligence sub-model; establishing a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through a third artificial intelligence sub-model; and obtaining target common carotid artery wall ultrasonic data of the target patient, and determining a target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasonic data through the corresponding relationship. The optimal estimation of the CCA wall motion can be obtained through the hierarchical state space model.

Description

Technical Field

[0001] The present application relates to the field of medical detection, and in particular to a method and device for measuring the motion of the common carotid artery wall using a hierarchical state space model. Background Art

[0002] Cardiovascular and cerebrovascular diseases (CVD) are a serious threat to human health, and one of its basic pathologies is atherosclerosis. The common carotid artery (CCA) of the human body continuously supplies blood to the brain. Once atherosclerosis occurs, it may cause thickening of the blood vessel wall and the formation of atherosclerotic plaques, which further develop into carotid stenosis or blockage, and eventually cause serious complications such as cerebral stroke. Therefore, accurate assessment of the health of the carotid artery in the early stage through medical imaging methods such as ultrasound before the patient shows obvious clinical symptoms can effectively reduce the risk of major diseases.

[0003] However, accurate measurement of CCA wall motion is challenging due to the instability of local image structure caused by the uniformity of CCA wall texture and the variability of tracking speckle patterns.

[0004] Most existing CCA wall motion measurement methods determine the evolution of internal hidden states through predefined transformation functions, which may not effectively constrain the complex nonlinear dynamic characteristics of the CCA wall. In addition, these methods only estimate CCA wall motion through the high score of the similarity map calculated by the block matching method. This may be problematic when tracking CCA wall tissue with a uniform appearance in the longitudinal direction. Existing CCA wall segmentation methods can be roughly divided into three categories: edge detection methods, active contour methods, and machine learning methods. Edge detection methods rely on traditional image processing techniques and mainly use thresholds to delineate the boundaries of the intima-media complex; active contour methods use contour-based procedures using parametric curves or geometric curves. However, both methods have difficulty coping with image noise and require manual adjustment of hyperparameters. Machine learning-based methods are greatly improved over the first two methods, but this method has tracking bias and the problem of trajectory divergence caused by the accumulation of errors in continuous tracking. Summary of the invention

[0005] In view of the above problems, the present application is proposed to provide a method and device for measuring the wall motion of the common carotid artery using a hierarchical state space model to overcome the above problems or at least partially solve the above problems, including:

[0006] A method for measuring the motion of the common carotid artery wall using a hierarchical state space model, the method involving an artificial intelligence model, the artificial intelligence model comprising a first artificial intelligence sub-model, a second artificial intelligence sub-model and a third artificial intelligence sub-model;

[0007] The method comprises:

[0008] A correspondence relationship between common carotid artery wall ultrasound data and common carotid artery wall motion estimation results is established through an artificial intelligence model; wherein the correspondence relationship includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, a first corresponding sub-relationship between common carotid artery wall ultrasound data and a similarity graph is established through the first artificial intelligence sub-model; a second corresponding sub-relationship between common carotid artery wall ultrasound data and common carotid artery wall segmentation information is established through the second artificial intelligence sub-model; a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation results is established through the third artificial intelligence sub-model;

[0009] The target common carotid artery wall ultrasound data of the target patient is acquired, and the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data is determined through the corresponding relationship.

[0010] Furthermore, the step of establishing a first corresponding sub-relationship between the common carotid artery wall ultrasound data and the similarity graph by using the first artificial intelligence sub-model includes:

[0011] Acquire the common carotid artery wall ultrasound data, and determine feature graphs of adjacent frames in the common carotid artery wall ultrasound data;

[0012] Determine a candidate region in the feature map that may contain a tracked target; wherein the tracked target is a target tracking tissue of a wall of a common carotid artery;

[0013] The features corresponding to the candidate region are Hungarian matched with the preset template features to obtain a corresponding similarity graph.

[0014] Furthermore, the step of obtaining feature maps of adjacent frames in the common carotid artery wall ultrasound data includes:

[0015] Extracting image features from the common carotid artery wall ultrasound data to obtain image features;

[0016] Extracting video segment features from the common carotid artery wall ultrasound data to obtain video segment features;

[0017] The feature map is generated according to the image features and the video clip features.

[0018] Furthermore, the step of establishing a second corresponding sub-relationship between the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information by using the second artificial intelligence sub-model includes:

[0019] Extracting and integrating features of the common carotid artery wall ultrasound data to generate a fusion feature map;

[0020] Determine the internal token sequence and the external token sequence according to the fused feature map; wherein each row of the external token sequence corresponds to a size d M ,j∈[1,2,…,n M ] memory slot m j , the memory slot m j Memory for storing images from the previous time step;

[0021] Based on a single-head self-attention operation, determining relevant information about pixels adjacent to the tracked object according to the internal token sequence;

[0022] Based on a single-head cross-attention operation, generating rich information across time frames according to the internal token sequence and the external token sequence;

[0023] generating an enhanced feature based on the internal token sequence, the relevant information and the enriched information;

[0024] The enhanced features are subjected to segmentation mask processing to obtain the common carotid artery wall segmentation information.

[0025] Furthermore, the step of generating rich information across time frames based on the internal token sequence and the external token sequence based on the single-head cross attention operation includes:

[0026] Performing a first projection operation on the internal token sequence and the external token sequence respectively to obtain intermediate features;

[0027] A second projection operation is performed on the internal token sequence and the intermediate features respectively to obtain rich information across time frames.

[0028] Furthermore, the third artificial intelligence sub-model includes a first agent and a second agent, each agent corresponds to a strategy network and a value network; the step of establishing a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through the third artificial intelligence sub-model includes:

[0029] Determine first local observation information based on the common carotid artery wall ultrasound data and the similarity map, and determine second local observation information based on the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information;

[0030] Determining a behavior strategy of the first agent based on the first local observation information, and determining a behavior strategy of the second agent based on the second local observation information;

[0031] Determining a first reward value for the behavior strategy of the first agent, and determining a second reward value for the behavior strategy of the second agent;

[0032] The behavior strategy of the first agent is updated according to the first reward value, and the behavior strategy of the second agent is updated according to the second reward value.

[0033] Furthermore, the step of determining a first reward value for the behavior strategy of the first agent and determining a second reward value for the behavior strategy of the second agent comprises:

[0034] The policy gradient of the expected benefit of each agent under the stochastic strategy is:

[0035]

[0036] In the formula, is the value function, s is the state of the environment perceived by the agent, and p μ is the state distribution, a=μ θ (s), Contains observations of all agents;

[0037] Consider now that there are N parameters θ i The continuous strategy μ i , we can get the gradient formula:

[0038]

[0039] In the formula, It is an experience replay pool that stores data. The data it stores is a tuple.

[0040] The value function is updated according to the following function:

[0041]

[0042] In the formula, The parameter is θ′ i The target strategy set.

[0043] A device for measuring the motion of the common carotid artery wall of a hierarchical state space model, the device relates to an artificial intelligence model, the artificial intelligence model comprises a first artificial intelligence sub-model, a second artificial intelligence sub-model and a third artificial intelligence sub-model;

[0044] The device comprises:

[0045] A correspondence establishing module, used to establish a correspondence between common carotid artery wall ultrasound data and common carotid artery wall motion estimation results through an artificial intelligence model; wherein the correspondence includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, a first corresponding sub-relationship between common carotid artery wall ultrasound data and a similarity graph is established through the first artificial intelligence sub-model; a second corresponding sub-relationship between common carotid artery wall ultrasound data and common carotid artery wall segmentation information is established through the second artificial intelligence sub-model; a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation results is established through the third artificial intelligence sub-model;

[0046] The data processing module is used to obtain the target common carotid artery wall ultrasound data of the target patient, and determine the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the corresponding relationship.

[0047] A device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method for measuring the wall motion of the common carotid artery using a hierarchical state-space model as described above are implemented.

[0048] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for measuring the wall motion of the common carotid artery using a hierarchical state-space model are implemented.

[0049] This application has the following advantages:

[0050] In an embodiment of the present application, with respect to the problem in the prior art that "existing CCA wall motion measurement methods cannot effectively constrain the complex nonlinear dynamic characteristics of the CCA wall", the present application provides a solution for stably and accurately measuring the common carotid artery wall motion through an artificial intelligence model, specifically: establishing a correspondence between common carotid artery wall ultrasound data and common carotid artery wall motion estimation results through an artificial intelligence model; wherein the correspondence includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, establishing a first corresponding sub-relationship between common carotid artery wall ultrasound data and a similarity graph through the first artificial intelligence sub-model; establishing a second corresponding sub-relationship between common carotid artery wall ultrasound data and common carotid artery wall segmentation information through the second artificial intelligence sub-model; establishing a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through the third artificial intelligence sub-model; acquiring the target common carotid artery wall ultrasound data of the target patient, and determining the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the corresponding relationship. The first artificial intelligence sub-model is used to process the hierarchical observation sequence in the common carotid artery wall ultrasound data to improve the accuracy of CCA wall motion measurement. The second artificial intelligence sub-model introduces the information of adjacent tissues to constrain the state of tracking tissues and obtain an accurate carotid artery wall segmentation mask. The third artificial intelligence sub-model is used to obtain the optimal estimate of CCA wall motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the description of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 It is a flowchart of the steps of a method for measuring the wall motion of the common carotid artery using a hierarchical state space model provided in one embodiment of the present application;

[0053] Figure 2 It is a structural block diagram of a device for measuring the motion of the common carotid artery wall of a hierarchical state space model provided in one embodiment of the present application;

[0054] Figure 3 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the objects, features and advantages of the present application more obvious and understandable, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0056] The inventors have found through analysis of the prior art that: there is one common carotid artery on each side, which rises along the outside of the trachea and larynx, and is relatively superficial, and its morphological structure can be easily observed during ultrasound examination. Compared with other medical imaging technologies, ultrasound examination is widely used in clinical examination of the carotid artery because of its convenience, economy, and non-invasiveness. Usually, after clinically using non-destructive ultrasound imaging technology to collect ultrasound images of the carotid artery, ultrasound doctors usually manually mark the vascular lumen-intima boundary (LII) and the vascular adventitia-media boundary (MAI) on the collected ultrasound images of the common carotid artery, and measure parameters such as the intima-media thickness (IMT), systolic inner diameter (Ds), and diastolic inner diameter (Dd) of the common carotid artery, and calculate information such as the wall mobility and stiffness coefficient.

[0057] Manual segmentation of LII and MAI and analysis of carotid wall motion are very time-consuming and laborious tasks, and are completely dependent on the doctor's personal experience and proficiency, which is somewhat subjective. In order to reduce the pressure on medical staff and improve the objective reliability of carotid wall motion estimation results, it is of great significance to perform automated motion estimation of carotid wall ultrasound image sequences. However, this is a challenging task. In addition to the low echo, speckle noise and motion artifact interference of the ultrasound image itself, the longitudinally coherent anatomical characteristics of the carotid wall itself also increase the difficulty of accurately measuring carotid wall motion (especially longitudinal motion).

[0058] There are also related research and exploration methods based on the measurement of common carotid artery (CCA) wall motion: (1) The original block matching algorithm is a direct motion tracking method, also known as speckle tracking in ultrasound imaging, which is a method widely used to estimate two-dimensional tissue motion in B-mode ultrasound image sequences. The basic idea of ​​the block matching algorithm is block division and motion estimation. The reference block of the current frame is used to search the search window of the next frame, and the mean square error (MSE), mean absolute difference (MAD) and normalized cross correlation (NCC) of the pixel grayscale between the reference block and the search block are calculated to find the best matching block. The matching block is considered as the reference block of the next frame and propagated frame by frame in the image sequence. However, this is a model-free method. Since it does not consider model assumptions and prior information, it is easily affected by the local instability of the ultrasound image, resulting in a large deviation in the estimation of the motion trajectory of the reference block on the carotid artery wall. (2) Most of the methods proposed recently for estimating carotid artery wall motion are based on block matching algorithms, combined with Kalman filter algorithms and state space models to model and analyze carotid artery wall dynamics. Gastounioti, Golemati, Stoitsis, and Nikita established a state space model as an additional constraint for CCA wall motion estimation. Based on the Kalman filter method, the matching blocks found by the block matching algorithm are used as observations, and the reference blocks output by the model are used as predictions. The two are weighed to update the optimal estimate of the reference block position and motion state. In order to better describe CCA wall motion, the state space method has also been updated. Gao, Xiong et al. proposed a linear state space method based on an elastic model controlled by an H∞ filter; Gao, Li et al. further introduced a nonlinear state space method and used an unscented Kalman filter to solve the nonlinear state transfer function. The above methods generally adopt the idea of ​​combining the state space model with the optimal state estimation, but different attempts have been made locally.

[0059] Existing state-space model methods have achieved certain results, mainly focusing on the state of tracked tissues in consecutive frames and using Kalman filters or other recursive algorithms to predict and update tissue positions. However, they only focus on estimating the state of tracked tissues in consecutive frames, lack a comprehensive understanding of the overall dynamic characteristics of the CCA wall, and do not make full use of information from adjacent tissues to improve prediction accuracy.

[0060] In response to the above problems, the inventors proposed the present application through extensive experimental analysis of hundreds of carotid artery ultrasound imaging sequences. The present application regards the segmented adjacent tissues as an additional state variable, transfers appearance information between adjacent tissues to simulate the spatial evolution constraints between them, and solves the limitations of relatively simple state transition structures. At the same time, HCSS introduces additional latent variables to model the evolution of each variable over time, realizes information transfer across time frames, and models the time evolution constraints between adjacent tissues. In addition, HCSS uses segmented CCA walls to constrain the position of the tracked tissue, ensuring that the tracked tissue remains within the boundaries of the CCA wall during motion tracking.

[0061] Reference Figure 1 , showing a method for measuring the motion of the common carotid artery wall using a hierarchical state-space model provided by an embodiment of the present application; the method involves an artificial intelligence model, the artificial intelligence model comprising a first artificial intelligence sub-model, a second artificial intelligence sub-model and a third artificial intelligence sub-model;

[0062] The method comprises:

[0063] S1. Establishing a correspondence between common carotid artery wall ultrasound data and common carotid artery wall motion estimation results through an artificial intelligence model; wherein the correspondence includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, establishing a first corresponding sub-relationship between common carotid artery wall ultrasound data and a similarity graph through the first artificial intelligence sub-model; establishing a second corresponding sub-relationship between common carotid artery wall ultrasound data and common carotid artery wall segmentation information through the second artificial intelligence sub-model; establishing a third corresponding sub-relationship between the similarity graph and common carotid artery wall segmentation information and common carotid artery wall motion estimation results through the third artificial intelligence sub-model;

[0064] S2. Acquire the target common carotid artery wall ultrasound data of the target patient, and determine the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the corresponding relationship.

[0065] In an embodiment of the present application, with respect to the problem in the prior art that "existing CCA wall motion measurement methods cannot effectively constrain the complex nonlinear dynamic characteristics of the CCA wall", the present application provides a solution for stably and accurately measuring the common carotid artery wall motion through an artificial intelligence model, specifically: establishing a correspondence between common carotid artery wall ultrasound data and common carotid artery wall motion estimation results through an artificial intelligence model; wherein the correspondence includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, establishing a first corresponding sub-relationship between common carotid artery wall ultrasound data and a similarity graph through the first artificial intelligence sub-model; establishing a second corresponding sub-relationship between common carotid artery wall ultrasound data and common carotid artery wall segmentation information through the second artificial intelligence sub-model; establishing a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through the third artificial intelligence sub-model; acquiring the target common carotid artery wall ultrasound data of the target patient, and determining the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the corresponding relationship. The first artificial intelligence sub-model is used to process the hierarchical observation sequence in the common carotid artery wall ultrasound data to improve the accuracy of CCA wall motion measurement. The second artificial intelligence sub-model introduces the information of adjacent tissues to constrain the state of tracking tissues and obtain an accurate carotid artery wall segmentation mask. The third artificial intelligence sub-model is used to obtain the optimal estimate of CCA wall motion.

[0066] Next, a method for measuring the common carotid artery wall motion using a hierarchical state-space model in this exemplary embodiment will be further described.

[0067] As described in step S1, a correspondence between the common carotid artery wall ultrasound data and the common carotid artery wall motion estimation result is established through an artificial intelligence model; wherein the correspondence includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, a first corresponding sub-relationship between the common carotid artery wall ultrasound data and the similarity graph is established through the first artificial intelligence sub-model; a second corresponding sub-relationship between the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information is established through the second artificial intelligence sub-model; and a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result is established through the third artificial intelligence sub-model.

[0068] It should be noted that the artificial intelligence model is a segmentation-assisted hierarchical state space model. The hierarchical state space model includes a first artificial intelligence sub-model, a second artificial intelligence sub-model and a third artificial intelligence sub-model. The first artificial intelligence sub-model is a multi-layer state space module, the second artificial intelligence sub-model is a segmentation-assisted module, and the third artificial intelligence sub-model is a multi-level segmentation-assisted state space model.

[0069] Multi-layer state space module: The multi-layer state space module includes a Transformer-XL encoder and a similarity graph propagation module. The Transformer-XL encoder is used to capture spatiotemporal evolution constraints and process hierarchical observation sequences; its output features are fused and used for subsequent processing to generate a similarity graph, which reflects the matching degree of the target area in the search area, and is propagated using ConvLSTM (Convolutional Long Short-term Memory Network).

[0070] Segmentation Assist Module: A context mining method, including an inner context mining module to effectively reintegrate relevant high-quality information from neighboring pixels, and an outer context mining module to enhance features by transferring rich information across time frames. The final accurate carotid artery wall segmentation mask is obtained.

[0071] Multi-level segmentation-assisted state-space model: The solution of this model uses a Transformer-assisted multi-agent reinforcement learning method, which accepts the similarity graph output by the multi-level state-space module and the CCA wall segmentation information output by the segmentation-assisted module as input. The reinforcement learning method is used to enable the segmentation agent and the motion analysis agent to execute the optimal strategy and obtain the optimal estimate of the CCA wall motion.

[0072] In one embodiment of the present invention, the specific process of "establishing the first corresponding sub-relationship between the common carotid artery wall ultrasound data and the similarity graph through the first artificial intelligence sub-model" in step S1 can be further explained in combination with the following description.

[0073] S111, acquiring the common carotid artery wall ultrasound data, and determining feature graphs of adjacent frames in the common carotid artery wall ultrasound data;

[0074] It should be noted that the common carotid artery wall ultrasound data is a carotid artery ultrasound sequence that is scanned and recorded in a certain order and steps during the carotid artery ultrasound examination. The historical common carotid artery wall ultrasound sequence is obtained as sample data, and the artificial intelligence model is trained through the sample data.

[0075] In one embodiment of the present invention, the specific process of step S111 "determining the feature graph of adjacent frames in the common carotid artery wall ultrasound data" can be further explained in combination with the following description.

[0076] As described in the following steps, image features are extracted from the common carotid artery wall ultrasound data to obtain image features; video segment features are extracted from the common carotid artery wall ultrasound data to obtain video segment features; and the feature map is generated based on the image features and the video segment features.

[0077] It should be noted that the spatial features of adjacent frames are aggregated across time through the Time Aware Network (TAN) to capture the dynamic changes of the carotid artery wall at different time points. ResNet is used to extract image features of the tracked target to obtain c i , use I3D to extract video clip features to get k i The features of the video clips are compressed by CNN (ConvLSTM) and combined with the image features to obtain a feature map

[0078] S112, determining a candidate region in the feature map that may contain a tracked target; wherein the tracked target is a target tracking tissue of a wall of a common carotid artery;

[0079] It should be noted that, based on the feature map containing spatiotemporal information obtained by TAN, the Region Proposal Network (RPN) is applied to select the region that may contain the target object. Specifically, RPN usually includes two branches: one branch is used for classification, that is, to determine whether the target object is contained in the anchor box, and the other branch is used for regression, that is, to adjust the position and size of the anchor box so that it can more accurately frame the target object. RPN first generates a series of predefined anchor boxes at each position of the feature map. These anchor boxes have different sizes and proportions to cover target objects of different sizes and shapes. For each anchor box, RPN outputs two scores: one represents the probability that the target object is contained in the anchor box (i.e., foreground probability), and the other represents the probability that the target object is not contained in the anchor box (i.e., background probability). At the same time, RPN also outputs four regression parameters for adjusting the position and size of the anchor box to make it closer to the actual target object. According to the output of RPN, the anchor box with a higher foreground probability is selected as the candidate region.

[0080] S113, performing Hungarian matching on the features corresponding to the candidate region and the preset template features to obtain a corresponding similarity graph.

[0081] It should be noted that for the area in the current frame that may contain the target object, the features in the template library are used to perform Hungarian matching with its features. Initially, the template features are only the features of the true value object in the first frame. After matching, the area with the highest confidence is selected and passed to the Mask Head for final segmentation generation. At the same time, if the target object in this area has a large appearance change, its features will be added to the template library.

[0082] Frame the picture i≤n continuous segments S=[s1,…,s n-1 ] and the tracked frame As input, where H×W represents the spatial size and C represents the channel.

[0083] represents the bounding box, where represents the center coordinates of the tracked target in the tth frame, Indicates its width and height. Indicates that at the tth frame, the confidence score is greater than σ det The detection box is recorded as In the initial stage of the tracking algorithm, at t = 0, the tracker initializes the bounding box to This bounding box is used to represent the approximate position and range of the target in the current frame. In the tth frame, the algorithm calculates the similarity between all possible targets in the current frame and the target to be tracked, and constructs a bipartite graph based on the position similarity IoU (Intersection over Union). IoU is an indicator to measure the degree of overlap between two regions. Its value is the ratio of the intersection area of ​​the two regions to the union area. The higher the IoU value, the higher the overlap between the two regions, that is, the higher the similarity. Define the weight between the i-th target and the j-th detected target in the current frame t i and After the Hungarian matching is completed, the target t i Updated to And retain its feature map f i As input to the reinforcement learning module.

[0084] In one embodiment of the present invention, the specific process of "establishing a second corresponding sub-relationship between the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information through the second artificial intelligence sub-model" in step S1 can be further explained in combination with the following description.

[0085] S121, extracting and integrating features of the common carotid artery wall ultrasound data to generate a fusion feature map;

[0086] It should be noted that in order to accurately segment the carotid intima-media, the inventors proposed an integrated context mining method based on the Mix Transformer encoder (MiT) of SegFormer. At time step t, the input image of size H×W is processed. The image is segmented into small blocks and then fed into the MiT encoder to generate a multi-scale feature map i∈1,2,3,4. In the formula, is a multi-scale feature map, C i ×H i ×W i dimensional real number space, C iis the channel dimension when i takes different values, H i is the height of the image block when i takes different values, W i is the image patch width for different values ​​of i. These feature maps correspond to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 resolutions of the original feature maps. After SegFormer, MLP layers and upsampling operations are applied to standardize the channel size and feature scale of all stages. By integrating multi-scale features, the fused feature map is obtained

[0087] S122, determining an internal token sequence and an external token sequence according to the fused feature graph; wherein each row of the external token sequence corresponds to a size d M ,j∈[1,2,…,n M ] memory slot m j , the memory slot m j Memory for storing images from the previous time step;

[0088] It should be noted that, subsequently, F t All its elements are expanded sequentially into an internal token sequence of size N×C Where N = H1 × W1. Internal context mining and external context mining are performed as follows:

[0089]

[0090] where F int and F ext denote the internal and external context mining modules respectively, and is their output. is an outer token sequence, where each row corresponds to a token of size d M ,j∈[1,2,…,n M ] memory slot m j , stores the image memory of the previous time step.

[0091] External memory sequence is a sequence of multiple memory slots, representing the memory state stored by the model at a certain time step t. Each memory slot can be regarded as a vector used to store specific state or feature information. At each time step t+1, the external memory discards the oldest memory slot. New memory slot m t+1 By transforming From the output Calculated. Among them: represents the output at time step t; tanh represents the hyperbolic tangent activation function, which limits the output to the range of (-1,1)(-1,1)(-1,1); WM Represents a weight matrix for linear transformation; M b Represents a bias vector. The new external memory sequence Update by:

[0092] S123, based on a single-head self-attention operation, determining relevant information about pixels adjacent to the tracked target according to the internal token sequence;

[0093] It should be noted that the multi-stage feature unification operation in SegFormer will lead to feature confusion, so the inventors proposed using the internal context mining module F int It is used to reintegrate the relevant high-quality information from adjacent pixels. In order to solve the feature confusion problem, a single-head self-attention operation is adopted. First, Project to Then pass Calculate Among them, Q int ,K int ,V int To project the input features The generated query, key, and value vectors. N is the sequence length of the input feature, and C is F t The feature dimension corresponding to each element in is K int The transpose of .

[0094] S124, based on a single-head cross-attention operation, generating rich information across time frames according to the internal token sequence and the external token sequence;

[0095] In one embodiment of the present invention, the specific process of step S124 "generating rich information across time frames according to the internal token sequence and the external token sequence based on a single-head cross-attention operation" can be further explained in combination with the following description.

[0096] As described in the following steps, a first projection operation is performed on the internal token sequence and the external token sequence respectively to obtain intermediate features; and a second projection operation is performed on the internal token sequence and the intermediate features respectively to obtain rich information across time frames.

[0097] It should be noted that the external context mining module F ext , enriching features by transferring a large amount of information across time frames. ext right and Perform two single-head cross-attention operations. The first projection operation: Projection to Q mid ,Will Projection to K mid ,V mid , get the intermediate features Second projection operation: Projection to Q ext , X mid Projection to K ext ,V ext ,pass Get Features feature That is the rich information across time frames.

[0098] In the formula, Q mid To project the input features The generated query vector, K mid ,V mid To project the input features The generated key and value vectors, K mid The transpose of Q ext To project the input features The generated query vector, K ext ,V ext is the input feature X through projection mid The resulting key and value vectors. K ext The transpose of .

[0099] Then the outer token sequence Discard memory slot Merge the latest output from the Transformer update. The new memory slot is calculated by It is obtained that, where tanh represents the hyperbolic tangent activation function, W M is a weight matrix, b m is a bias vector. The updated memory was changed to

[0100] S125, generating an enhanced feature according to the internal token sequence, the relevant information and the enriched information;

[0101] It should be noted that the enhanced features pass Get, where It is element-by-element addition.

[0102] S126. Perform segmentation mask processing on the enhanced features to obtain the common carotid artery wall segmentation information.

[0103] It should be noted that, finally, the predicted segmentation mask M mask Export As in Represents a classification header.

[0104] In one embodiment of the present invention, the specific process of "establishing the third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through the third artificial intelligence sub-model" described in step S1 can be further explained in combination with the following description.

[0105] It should be noted that the HCSS of the present application adopts a multi-agent DDPG (muli-agent DDPG, MADDPG) framework. The third artificial intelligence sub-model includes a first agent and a second agent, each of which corresponds to a policy network and a value network. The first agent is a motion measurement agent, and the second agent is a segmentation agent. There are N agents in the HCSS, including N-1 segmentation agents and one motion measurement agent. Each type of agent has different observation, action and reward equations.

[0106] S131, determining first local observation information according to the common carotid artery wall ultrasound data and the similarity map, and determining second local observation information according to the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information;

[0107] It should be noted that at time step t, the local observation of the motion measurement agent is

[0108] The local observation of each segmentation agent is Specifically, the local observation of the motion measurement agent It consists of the following three parts:

[0109] (1) It is a picture frame The cropped tiles within the range, where represents the center coordinates of the target being pursued, w and h represent its width and height respectively.

[0110] (2) is through The generated optical flow map. The generation of the optical flow map is obtained by the existing technology, such as the method of the literature (Jonschkowski, R., Stone, A., Barron, JT, Gordon, A., Konolige, K., & Angelova, A. (2020). What matters in unsupervised optical flow. In Proceedings of the European conference on computer vision (pp. 557–572).), which will not be repeated here.

[0111] (3) It is the similarity graph obtained by the first artificial intelligence sub-model.

[0112] It should be noted that the local observation of the segmentation agent It consists of the following three parts:

[0113] (1) It is a picture frame in represents the center coordinates of N-1 tiles, w and h represent their width and height. These points are evenly distributed in the center of the inner membrane-middle membrane complex.

[0114] (2) is through Resulting optical flow map. The generation of the optical flow map is consistent with the generation of the optical flow map of the motion measurement agent.

[0115] (3) is obtained from the second artificial intelligence submodel The cropping mask M of the segmentation mask within mask .

[0116] S132, determining a behavior strategy of the first agent based on the first local observation information, and determining a behavior strategy of the second agent based on the second local observation information;

[0117] S133, determining a first reward value for the behavior strategy of the first agent, and determining a second reward value for the behavior strategy of the second agent;

[0118] S134. Update the behavior strategy of the first agent according to the first reward value, and update the behavior strategy of the second agent according to the second reward value.

[0119] For the above S132-S134, it should be noted that at time step t, the behavior strategy of the motion measurement agent is The action of the i-th segmentation agent is Motion measurement agent uses action a t =[Δx t ,Δy t ] is used to represent the relative motion of the tracked target. t ,Δy t Represents the horizontal and vertical movement respectively. Therefore, by The updated bounding box is obtained through this action in is the center coordinate of the tracked target. For each segmented agent, by defining the action To map the binary mask to a continuous latent space, the action space is reduced in dimensionality. Here, the dimension l is much smaller than w·h. The latent code is then expanded using the decoder to obtain the full-resolution mask

[0120] The policy gradient of the expected return of each agent under the stochastic policy:

[0121]

[0122] in, is the value function, s is the state of the environment perceived by the agent, and p μ is the state distribution, a=μ θ (s), Contains observations of all agents.

[0123] Consider now that there are N parameters θ i Continuous strategy (Abbreviated as μ i ), we can get the gradient formula:

[0124]

[0125] in, It is an experience replay pool that stores data. The data it stores is a tuple.

[0126] The value function can be updated as follows:

[0127]

[0128] in The parameter is θ i ′ The target strategy set.

[0129] In general, the process of this module is as follows: First, initialize the behavior strategy of each agent (βi Can be used with μ i Different), deterministic target strategy Value Function At each time step t, according to get Thus updating:

[0130]

[0131] In the formula, is the observation of agent i at time step t, is the agent i at time step t based on the observation The behavioral strategies adopted, is the action of agent i at time step t, is the reward of agent i at time step t.

[0132] As described in step S2, the target common carotid artery wall ultrasound data of the target patient is acquired, and the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data is determined through the corresponding relationship.

[0133] It should be noted that, by using a medical ultrasound imaging device, an ultrasound probe is used to find and locate the carotid artery on the patient's neck, and then the ultrasound probe is placed close to the patient's skin and scanned along the carotid artery to collect a target ultrasound image sequence of the target patient's carotid artery. The target ultrasound image sequence is input into the trained artificial intelligence model, and the model is used to analyze the target ultrasound image sequence of the target patient, and output the target common carotid artery wall motion estimation result.

[0134] The beneficial effects achieved by the present invention include:

[0135] (1) Segmentation-assisted hierarchical constrained state space method:

[0136] The method of the present invention constrains the state of the tracking tissue by introducing information of adjacent tissues, which helps to solve the instability problem in traditional methods caused by the uniformity of the texture and the change of the spot pattern of the common carotid artery (CCA) wall, so as to better simulate the dynamic behavior of the entire CCA wall and obtain more reliable results.

[0137] The method of the present invention not only considers the state of the tracked tissue, but also utilizes the information of the adjacent tissues, which helps to solve the instability problem caused by the changes in the uniformity of the texture and the speckle pattern of the common carotid artery wall.

[0138] (2) Spatial and temporal evolution constraints:

[0139] This method enhances the tracking accuracy and robustness through spatial evolution constraint (SEC) and temporal evolution constraint (TEC). The spatial evolution constraint emphasizes the physical relationship between adjacent tissues and helps to reduce the interference caused by the uniformity of CCA wall texture. The temporal evolution constraint overcomes the influence of speckle pattern variation by considering the change of speckle pattern over time.

[0140] (3) Multi-layer state space method:

[0141] The method of the present invention processes long sequences through a multi-layer state space method, and uses the potential state of short sequences to capture the changes of observed variables in long sequences. Among them, the Transformer-XL encoder module and the similarity map propagation module respectively establish effective models for dynamic constraints and dependencies, significantly improving the accuracy of CCA wall motion measurement.

[0142] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0143] Reference Figure 2 , showing a common carotid artery wall motion measurement device of a hierarchical state space model provided by an embodiment of the present application; the device relates to an artificial intelligence model, and the artificial intelligence model includes a first artificial intelligence sub-model, a second artificial intelligence sub-model and a third artificial intelligence sub-model;

[0144] Specifically include:

[0145] The corresponding relationship establishing module 210 is used to establish the corresponding relationship between the common carotid artery wall ultrasound data and the common carotid artery wall motion estimation result through the artificial intelligence model; wherein the corresponding relationship includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, the first corresponding sub-relationship between the common carotid artery wall ultrasound data and the similarity map is established through the first artificial intelligence sub-model; the second corresponding sub-relationship between the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information is established through the second artificial intelligence sub-model; the third corresponding sub-relationship between the similarity map and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result is established through the third artificial intelligence sub-model;

[0146] The data processing module 220 is used to obtain the target common carotid artery wall ultrasound data of the target patient, and determine the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the corresponding relationship.

[0147] In one embodiment of the present invention, the corresponding relationship establishing module 210 includes:

[0148] A data acquisition submodule, used to acquire the common carotid artery wall ultrasound data and determine a feature map of adjacent frames in the common carotid artery wall ultrasound data;

[0149] A candidate region determination submodule, used to determine a candidate region in the feature map that may contain a tracked target; wherein the tracked target is a target tracking tissue of the wall of the common carotid artery;

[0150] The matching submodule is used to perform Hungarian matching on the features corresponding to the candidate region and the preset template features to obtain a corresponding similarity graph.

[0151] In one embodiment of the present invention, the data acquisition submodule includes:

[0152] An image feature extraction unit, used for extracting image features from the common carotid artery wall ultrasound data to obtain image features;

[0153] A video feature extraction unit, used for extracting video segment features from the common carotid artery wall ultrasound data to obtain video segment features;

[0154] A feature generation unit is used to generate the feature map according to the image feature and the video clip feature.

[0155] In one embodiment of the present invention, the corresponding relationship establishing module 210 includes:

[0156] A feature fusion submodule, used for extracting and integrating features of the common carotid artery wall ultrasound data to generate a fusion feature map;

[0157] The token sequence submodule is used to determine the internal token sequence and the external token sequence according to the fused feature map; wherein each row of the external token sequence corresponds to a size d M ,j∈[1,2,…,n M ] memory slot m j , the memory slot m j Memory for storing images from the previous time step;

[0158] A first self-attention operation submodule, configured to determine relevant information about pixels adjacent to the tracked target according to the internal token sequence based on a single-head self-attention operation;

[0159] A second self-attention operation submodule, configured to generate rich information across time frames according to the internal token sequence and the external token sequence based on a single-head cross-attention operation;

[0160] an enhanced feature generation submodule, configured to generate an enhanced feature based on the internal token sequence, the relevant information and the enriched information;

[0161] The segmentation submodule is used to perform segmentation mask processing on the enhanced features to obtain the common carotid artery wall segmentation information.

[0162] In one embodiment of the present invention, the second self-attention operation submodule includes:

[0163] A first projection unit, configured to perform a first projection operation on the internal token sequence and the external token sequence respectively to obtain intermediate features;

[0164] The second projection unit is used to perform a second projection operation on the internal token sequence and the intermediate feature respectively to obtain rich information across time frames.

[0165] In one embodiment of the present invention, the third artificial intelligence sub-model includes a first agent and a second agent, each agent corresponds to a strategy network and a value network; the corresponding relationship establishment module 210 includes:

[0166] A local observation submodule, configured to determine first local observation information based on the common carotid artery wall ultrasound data and the similarity map, and to determine second local observation information based on the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information;

[0167] a behavior strategy determination submodule, configured to determine the behavior strategy of the first agent based on the first local observation information, and to determine the behavior strategy of the second agent based on the second local observation information;

[0168] a reward submodule, configured to determine a first reward value for the behavior strategy of the first agent, and to determine a second reward value for the behavior strategy of the second agent;

[0169] An updating submodule is used to update the behavior strategy of the first agent according to the first reward value, and to update the behavior strategy of the second agent according to the second reward value.

[0170] In one embodiment of the present invention, the reward submodule includes:

[0171] The policy gradient calculation unit is used to calculate the policy gradient of the expected benefit of each agent under the stochastic policy:

[0172]

[0173] In the formula, is the value function, p μ is the state distribution, a=μ θ (s), Contains observations of all agents;

[0174] Gradient calculation unit, used to consider that there are now N parameters θi The continuous strategy μ i , we can get the gradient formula:

[0175]

[0176] In the formula, It is an experience replay pool that stores data. The data it stores is a tuple.

[0177] The value function updating unit is used to update the value function according to the following function:

[0178]

[0179] In the formula, The parameter is θ i ′ The target strategy set.

[0180] Reference Figure 3 , showing a computer device of a method for measuring the motion of the common carotid artery wall based on a hierarchical state space model of the present invention, which may specifically include the following:

[0181] The computer device 12 is in the form of a general-purpose computing device, and the components of the computer device 12 may include but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0182] The bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor or a local bus 18 using any of a variety of bus 18 architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, an Audio Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.

[0183] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0184] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). Although Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical medium) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42, which are configured to perform the functions of various embodiments of the present invention.

[0185] A program / utility 40 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0186] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, cameras, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs)), wide area networks (WANs), and / or public networks (e.g., the Internet) via a network adapter 20. Figure 3 As shown, the network adapter 20 communicates with other modules of the computer device 12 via the bus 18. It should be understood that although Figure 3 Not shown, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.

[0187] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing a method for measuring the common carotid artery wall motion based on a hierarchical state-space model provided in an embodiment of the present invention.

[0188] That is, when the processing unit 16 executes the program, the following is achieved: establishing a correspondence between the common carotid artery wall ultrasound data and the common carotid artery wall motion estimation result through an artificial intelligence model; wherein the correspondence includes a first correspondence sub-relationship, a second correspondence sub-relationship and a third correspondence sub-relationship; specifically, establishing a first correspondence sub-relationship between the common carotid artery wall ultrasound data and the similarity graph through the first artificial intelligence sub-model; establishing a second correspondence sub-relationship between the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information through the second artificial intelligence sub-model; establishing a third correspondence sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through the third artificial intelligence sub-model; acquiring the target common carotid artery wall ultrasound data of the target patient, and determining the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the correspondence.

[0189] In an embodiment of the present invention, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for measuring the common carotid artery wall motion using a hierarchical state-space model as provided in all embodiments of the present application:

[0190] That is, when the program is executed by the processor, it is achieved: establishing a correspondence between the common carotid artery wall ultrasound data and the common carotid artery wall motion estimation result through the artificial intelligence model; wherein the correspondence includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, establishing a first corresponding sub-relationship between the common carotid artery wall ultrasound data and the similarity graph through the first artificial intelligence sub-model; establishing a second corresponding sub-relationship between the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information through the second artificial intelligence sub-model; establishing a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through the third artificial intelligence sub-model; acquiring the target common carotid artery wall ultrasound data of the target patient, and determining the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the corresponding relationship.

[0191] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0192] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0193] The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.

[0194] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0195] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0196] The above is a detailed introduction to the method and device for measuring the common carotid artery wall motion of a hierarchical state-space model provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for measuring the motion of the common carotid artery wall using a hierarchical state space model, characterized in that: The method involves an artificial intelligence model, the artificial intelligence model comprising a first artificial intelligence sub-model, a second artificial intelligence sub-model, and a third artificial intelligence sub-model; The method comprises: A correspondence relationship between common carotid artery wall ultrasound data and common carotid artery wall motion estimation results is established through an artificial intelligence model; wherein the correspondence relationship includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, a first corresponding sub-relationship between common carotid artery wall ultrasound data and a similarity graph is established through the first artificial intelligence sub-model; a second corresponding sub-relationship between common carotid artery wall ultrasound data and common carotid artery wall segmentation information is established through the second artificial intelligence sub-model; a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation results is established through the third artificial intelligence sub-model; The target common carotid artery wall ultrasound data of the target patient is acquired, and the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data is determined through the corresponding relationship.

2. The method according to claim 1, characterized in that The step of establishing a first corresponding sub-relationship between the common carotid artery wall ultrasound data and the similarity graph by using the first artificial intelligence sub-model comprises: Acquire the common carotid artery wall ultrasound data, and determine feature graphs of adjacent frames in the common carotid artery wall ultrasound data; Determine a candidate region in the feature map that may contain a tracked target; wherein the tracked target is a target tracking tissue of a wall of a common carotid artery; The features corresponding to the candidate region are Hungarian matched with the preset template features to obtain a corresponding similarity graph.

3. The method according to claim 2, characterized in that The step of obtaining feature maps of adjacent frames in the common carotid artery wall ultrasound data comprises: Extracting image features from the common carotid artery wall ultrasound data to obtain image features; Extracting video segment features from the common carotid artery wall ultrasound data to obtain video segment features; The feature map is generated according to the image features and the video clip features.

4. The method according to claim 2, characterized in that: The step of establishing a second corresponding sub-relationship between the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information by using the second artificial intelligence sub-model comprises: Extracting and integrating features of the common carotid artery wall ultrasound data to generate a fusion feature map; Determine the internal token sequence and the external token sequence according to the fused feature map; wherein each row of the external token sequence corresponds to a size d M ,j∈[1,2,…,n M ] memory slot m j , the memory slot m j Memory for storing images from the previous time step; Based on a single-head self-attention operation, determining relevant information about pixels adjacent to the tracked object according to the internal token sequence; Based on a single-head cross-attention operation, generating rich information across time frames according to the internal token sequence and the external token sequence; generating an enhanced feature based on the internal token sequence, the relevant information and the enriched information; The enhanced features are subjected to segmentation mask processing to obtain the common carotid artery wall segmentation information.

5. The method according to claim 4, characterized in that The step of generating rich information across time frames based on the internal token sequence and the external token sequence based on the single-head cross attention operation comprises: Performing a first projection operation on the internal token sequence and the external token sequence respectively to obtain intermediate features; A second projection operation is performed on the internal token sequence and the intermediate features respectively to obtain rich information across time frames.

6. The method according to claim 1, characterized in that The third artificial intelligence sub-model includes a first agent and a second agent, each agent corresponds to a strategy network and a value network; the step of establishing a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation result through the third artificial intelligence sub-model includes: Determine first local observation information based on the common carotid artery wall ultrasound data and the similarity map, and determine second local observation information based on the common carotid artery wall ultrasound data and the common carotid artery wall segmentation information; Determining a behavior strategy of the first agent based on the first local observation information, and determining a behavior strategy of the second agent based on the second local observation information; Determining a first reward value for the behavior strategy of the first agent, and determining a second reward value for the behavior strategy of the second agent; The behavior strategy of the first agent is updated according to the first reward value, and the behavior strategy of the second agent is updated according to the second reward value.

7. The method according to claim 1, characterized in that The step of determining a first reward value for the behavior strategy of the first agent and determining a second reward value for the behavior strategy of the second agent comprises: The policy gradient of the expected benefit of each agent under the stochastic strategy is: In the formula, is the value function, s is the state of the environment perceived by the agent, and p μ is the state distribution, a=μ θ (s), Contains observations of all agents; Consider now that there are N parameters θ i The continuous strategy μ i , we can get the gradient formula: In the formula, It is an experience replay pool that stores data. The data it stores is a tuple. The value function is updated according to the following function: In the formula, The parameter is θ′ i The target strategy set.

8. A device for measuring the motion of the common carotid artery wall based on a hierarchical state space model, characterized in that: The apparatus relates to an artificial intelligence model, the artificial intelligence model comprising a first artificial intelligence sub-model, a second artificial intelligence sub-model, and a third artificial intelligence sub-model; The device comprises: A correspondence establishing module, used to establish a correspondence between common carotid artery wall ultrasound data and common carotid artery wall motion estimation results through an artificial intelligence model; wherein the correspondence includes a first corresponding sub-relationship, a second corresponding sub-relationship and a third corresponding sub-relationship; specifically, a first corresponding sub-relationship between common carotid artery wall ultrasound data and a similarity graph is established through the first artificial intelligence sub-model; a second corresponding sub-relationship between common carotid artery wall ultrasound data and common carotid artery wall segmentation information is established through the second artificial intelligence sub-model; a third corresponding sub-relationship between the similarity graph and the common carotid artery wall segmentation information and the common carotid artery wall motion estimation results is established through the third artificial intelligence sub-model; The data processing module is used to obtain the target common carotid artery wall ultrasound data of the target patient, and determine the target common carotid artery wall motion estimation result corresponding to the target common carotid artery wall ultrasound data through the corresponding relationship.

9. A device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images