Ultrasonic image registration method and device, storage medium and electronic equipment

Abstract

The invention relates to an ultrasonic image registration method and device, a storage medium and electronic equipment. The method comprises the steps that a preoperative CT image and an anatomical structure label graph based on the preoperative CT image are acquired; on the basis of the anatomical structure label graph and the preoperative CT image, generating a voxel-level continuous regular weight graph representing deformation constraint strength of different anatomical regions; determining a reachable scanning space of an ultrasonic probe in a space where the preoperative CT image is located based on the anatomical structure label map and / or the voxel-level continuous regular weight map; in the reachable scanning space, according to a preset action output by a reinforcement learning strategy network, updating an acquisition pose of the ultrasonic probe, and acquiring a corresponding ultrasonic image frame sequence based on the updated acquisition pose; and inputting the ultrasonic image frame sequence and the preoperative CT image into a pre-constructed registration network for non-rigid registration to obtain a deformation field. The accuracy of the acquired ultrasonic image can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an ultrasound image registration method, apparatus, storage medium, and electronic device. Background Technology

[0002] In clinical procedures such as liver tumor resection, radiofrequency ablation, and intraoperative navigation, preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images are typically required as anatomical references to guide intraoperative path planning and target identification. However, due to significant differences between preoperative images and actual intraoperative conditions, especially under the influence of tissue deformation, respiratory motion, and surgical manipulation, precise localization cannot be achieved solely based on preoperative images.

[0003] Ultrasound, as an important intraoperative imaging tool, has advantages such as being radiation-free, highly real-time, and low-cost, and is widely used in abdominal surgeries, especially in liver surgery. Intraoperative ultrasound images can be used to observe lesion boundaries, blood vessel positions, and tissue deformation in real time. Therefore, they can be combined with preoperative images using a pre-set registration algorithm to align the ultrasound images with the preoperative images, thereby determining the path planning and target points corresponding to the preoperative images within the ultrasound images. However, due to the limitations of ultrasound images, such as strong speckle noise, low signal-to-noise ratio, and imaging viewpoint dependence, they differ significantly from CT and MRI images in terms of resolution, grayscale distribution, and spatial consistency. Consequently, registration accuracy when using registration algorithms to align ultrasound images with preoperative images is relatively low, thus affecting the precise localization of target points. Summary of the Invention

[0004] In view of this, the present invention provides an ultrasound image registration method, apparatus, storage medium, and electronic device.

[0005] Specifically, the present invention is achieved through the following technical solution: According to a first aspect of the present invention, an ultrasound image registration method is provided, the ultrasound image registration method comprising: Acquire preoperative CT images and anatomical structure labeling maps based on the preoperative CT images; Based on the anatomical structure label map and the preoperative CT image, a voxel-level continuous regularized weight map is generated to characterize the deformation constraint intensity of different anatomical regions. In the space of the preoperative CT image, the reachable scanning space of the ultrasound probe is determined based on the anatomical structure label map and / or the voxel-level continuous regularized weight map; Within the reachable scanning space, the acquisition pose of the ultrasound probe is updated according to the actions output by the pre-set reinforcement learning strategy network, and the corresponding ultrasound image frame sequence is acquired based on the updated acquisition pose. The ultrasound image frame sequence and the preoperative CT image are input into a pre-constructed registration network for non-rigid registration to obtain the deformation field; The loss function of the registration network includes a regularization constraint term constructed based on the voxel-level continuous regularized weight map. The regularization constraint term is used to apply differentiated deformation constraints to different anatomical regions so that the deformation intensity of structurally sensitive regions is more restricted, while flexible tissue regions are allowed to have a larger degree of deformation freedom.

[0006] Optionally, the method further includes: Gradient features are obtained from the preoperative CT images; Based on the anatomical structure label map, the preoperative CT image, and the gradient features, a voxel-level anatomical label map is obtained to determine the initial pose information of the probe used to capture ultrasound images.

[0007] Optionally, obtaining a voxel-level anatomical label map based on the anatomical structure label map, the preoperative CT image, and the gradient features includes: The preoperative CT images were converted to grayscale to obtain the original CT volume data; Based on a pre-constructed prior space model, the anatomical structure label map corresponding to the original CT body data is obtained; The anatomical structure label map is cropped and interpolated and resampled to obtain a continuous probability label map whose voxel spacing is consistent with the voxel spacing in the preoperative CT image. Extract the texture histogram from the continuous probabilistic label map, and obtain voxel features based on the preoperative CT image, the gradient features, and the texture histogram; Based on a pre-set mapping function, the anatomical structure label map, the preoperative CT image, and the gradient features are mapped to obtain a voxel-level anatomical label map. Based on the voxel features and a pre-built lightweight classification model, the classification of the voxel-level anatomical label map is obtained.

[0008] Optionally, the generation of a voxel-level continuous regularized weighted map based on the anatomical structure label map and the preoperative CT image includes: Based on the category information of different anatomical structures in the anatomical structure label map, initial discrete regular weights are assigned to each anatomical structure according to the preset category-deformation rigidity mapping relationship; Spatial filtering and smoothing are performed on the initial discrete regularized weights to obtain a regularized control continuous distribution function that transitions continuously at the boundaries of the anatomical structure. Based on the regularized control continuous distribution function, the voxel-level continuous regularized weight map is generated. In the voxel-level continuous regularized weight map, the regularized weight of voxels corresponding to structurally sensitive regions including the main blood vessel trunk and tumor boundary is higher, while the regularized weight of voxels corresponding to flexible tissue regions including liver parenchyma is lower. The voxel-level continuous regularized weight map is used to apply differentiated deformation constraints to different anatomical regions in the regularization constraint term of the registration network.

[0009] Optionally, determining the initial pose information of the probe used to capture ultrasound images includes: Based on the anatomical structure label map, mark the classification regions corresponding to the preset classifications; Expand the predefined voxel range outward from the classification region to obtain the reachable space, and preprocess the reachable space; In the preprocessed reachable space, imageable points on the liver surface are selected to obtain the initial pose information of the probe.

[0010] Optionally, the step of expanding the classification region outward by a preset voxel range to obtain the reachable space includes: Obtain the starting mask of the classification region, and use the starting mask to perform masking processing on the classification region to obtain the masked classification region; 3D morphological dilation is performed on the mask classification region to obtain the dilated mask classification region; Based on the liver surface in the dilation mask classification region, obtain the extrusion classification region containing the extrusion depth; Obtain the union of the dilation mask classification region and the extrusion classification region to obtain the preliminary reachable space; Based on the pre-constructed exclusion mask, invalid voxels in the initial reachable space are excluded to obtain the reachable space.

[0011] Optionally, the method further includes: Centered on the pose point of the current ultrasound probe mapped to the CT coordinate system, a three-dimensional sub-block matching the field of view of the ultrasound image is extracted from the CT volume data corresponding to the preoperative CT image. The extracted 3D sub-blocks are interpolated and resampled to generate CT structural features corresponding to the spatial dimensions of the ultrasound image.

[0012] According to a second aspect of the present invention, an ultrasound image registration apparatus is provided, the ultrasound image registration apparatus comprising: The tag acquisition module is used to acquire preoperative CT images and anatomical structure tag maps based on the preoperative CT images; The pose determination module is used to generate a voxel-level continuous regularized weighted map characterizing the deformation constraint intensity of different anatomical regions based on the anatomical structure label map and the preoperative CT image. The image transformation module is used to determine the reachable scanning space of the ultrasound probe based on the anatomical structure label map and / or the voxel-level continuous regularized weight map in the space where the preoperative CT image is located. The pose update module is used to update the acquisition pose of the ultrasound probe within the reachable scanning space according to the action output by the pre-set reinforcement learning strategy network, and acquire the corresponding ultrasound image frame sequence based on the updated acquisition pose. The ultrasound image update module is used to perform non-rigid registration of the ultrasound image frame sequence with the preoperative CT image input to a pre-constructed registration network to obtain a deformation field. The loss function of the registration network includes a regularization constraint term constructed based on the voxel-level continuous regularized weight map. The regularization constraint term is used to apply differentiated deformation constraints to different anatomical regions so that the deformation intensity of structurally sensitive regions is more restricted, while flexible tissue regions are allowed to have a larger degree of deformation freedom.

[0013] According to a third aspect of the invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the ultrasound image registration method in any possible implementation of the first aspect.

[0014] According to a fourth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the ultrasound image registration method in any possible implementation of the first aspect. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0017] Figure 1 A schematic flowchart of an ultrasound image registration method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a closed-loop path modeling and acquisition strategy based on reinforcement learning in an ultrasound image registration method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a closed-loop path modeling and acquisition strategy based on reinforcement learning in an ultrasound image registration method provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an ultrasound image registration device provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In related technologies, methods that use pre-set registration algorithms to register and align ultrasound images with preoperative images suffer from low registration accuracy. This is because ultrasound images are limited by strong speckle noise, low signal-to-noise ratio, and imaging viewpoint dependence, resulting in significant differences from CT and MRI images in terms of resolution, grayscale distribution, and spatial consistency. Consequently, registration based on these algorithms results in low accuracy, affecting the precise localization of target points. Furthermore, since ultrasound image acquisition paths rely on operator experience, issues such as repeated acquisitions, missed areas, or insufficient exposure of key structures often arise. This leads to uncertainties in the obtained ultrasound images regarding viewpoint, area coverage, and structural representation, increasing the complexity and uncertainty of non-rigid registration tasks. Moreover, in terms of deformation control, non-rigid registration algorithms employ a space-invariant regularization strategy, which applies a fixed degree of smoothness constraint to all regions globally. This makes it difficult to adapt to the complexity of the anatomical structure of the liver region. For example, excessive deformation should be avoided at vascular bifurcation points, while the middle of the liver parenchyma can tolerate more flexible registration. This makes methods with fixed regularization strategies prone to problems such as "vascular misalignment" or "tumor boundary alignment distortion," which is not conducive to accurate intraoperative judgment and also reduces the accuracy of target localization.

[0020] In recent years, with the development of deep learning technology, registration methods based on learning models, such as the VoxelMorph model and RegNet model, have been proposed. These methods learn mapping relationships through end-to-end neural networks and have shown good results in rigid or slightly non-rigid scenarios, effectively improving registration accuracy and robustness. However, these methods require ultrasound images input to the learning model to be full-coverage, high-quality, and orientation-standardized images. In contrast, the intraoperative ultrasound images actually captured often exhibit significant unstructured characteristics. For example, ultrasound image frames may come from different directions, have large-scale occlusion or distortion, and the acquisition path is determined based on the operator's experience, making the path uncontrollable. This makes it difficult to achieve stable accuracy in the absence of regional priors and path guidance.

[0021] Furthermore, current learning models lack the ability to jointly model the "acquisition path" and "structural distribution." Specifically, learning models typically treat registration as a static problem of "given image pairs, output deformation fields," neglecting the dynamic and strategic nature of the image acquisition process. For example, when the registration error in a certain region is high, the model should be able to actively identify and guide the ultrasound probe to perform supplementary scanning in that region, thereby achieving closed-loop optimization through "correction as it goes." However, because related techniques lack path memory mechanisms and the ability to globally model historical frame information, and because regularization strategies are mostly fixed parameters or have weak spatial adaptability, it is difficult to dynamically adjust the degree of constraint on registration deformation based on anatomical priors.

[0022] Therefore, the ultrasound-CT image registration methods of related technologies suffer from key bottlenecks in real clinical applications, such as "difficulty in optimizing blind scanning of the acquisition path," "inability to finely control global fixed regularization," and "lack of dynamic feedback mechanisms," as detailed below: Blind acquisition paths lead to insufficient structural coverage and redundancy: The lack of a path planning mechanism in ultrasound image acquisition makes it difficult to effectively guide the ultrasound probe to cover key anatomical areas, resulting in limited registration quality.

[0023] Fixed regularization strategies fail to adapt to the differentiated deformation requirements of liver structures: registration algorithms of related technologies use globally consistent deformation constraints, which make it difficult to achieve accurate registration around fine structures such as blood vessels.

[0024] The lack of a real-time feedback mechanism prevents dynamic optimization of registration results: the registration process is static and cannot be adjusted or optimized based on real-time processing results; it also lacks data acquisition capabilities. Evaluate The closed-loop capability of adjustment makes it difficult to achieve adaptive accuracy improvement.

[0025] Lacking path memory capabilities, the data collection strategy lacks global awareness and optimization: it cannot make decisions based on historical scanning trajectories to obtain the optimal data collection path, resulting in blind and inefficient data collection behavior.

[0026] In this embodiment, to achieve efficient and accurate intraoperative image registration in complex anatomical structures (such as the liver), an ultrasound image registration method combining acquisition path modeling (reinforcement learning path modeling) and spatial prior difference constraints (spatial prior regularization) is proposed. This method constructs an intraoperative fused image by registering real-time intraoperative images, such as ultrasound images, with non-rigid images. This allows the intraoperative fused image to overlay real-time ultrasound image information (ultrasound image) onto familiar CT anatomical structures (preoperative CT images), taking into account both spatial perception and soft tissue dynamic changes. This provides a "strong planning, more accurate alignment, and faster feedback" end-to-end capability, significantly improving registration accuracy and target point localization accuracy, thereby enhancing surgical safety and precision, and ultimately promoting the clinical application of intelligent image fusion technology. In this embodiment, the inference process begins with the path acquisition action of the initial frame. The structural prior map and λ-map constructed based on the preoperative CT image and its anatomical labels serve as guiding information, directing the scanning focus area of ​​the initial path. The first ultrasound image frame is input as the initial state. A state vector is constructed by combining image content, current pose encoding, historical path mask, and λ-map. The enhancement strategy network then generates the optimal action for the next scan, specifying the direction and magnitude of displacement. This action is represented as a displacement vector Δp ​​in a three-dimensional coordinate system, indicating the direction and number of millimeters to move in the acquisition space. After acquiring the next frame, a registration network is invoked in real-time to perform non-rigid registration between the new ultrasound and CT images, generating the deformation field at the current moment. The image is t and the registered and fused image. The registration result is scored using an image similarity metric function, commonly including mutual information (MI), normalized cross-correlation (NCC), or the structure label Dice coefficient. Based on the change in score Δs, i.e., the improvement in registration quality, the immediate reward rt for the current action can be calculated, which is used to reinforce policy optimization.

[0027] See Figure 1 This invention provides an ultrasound image registration method, which may include the following steps: S101. Acquire preoperative CT images and anatomical structure labeling based on the preoperative CT images; S102. Based on the anatomical structure label map and the preoperative CT image, generate a voxel-level continuous regularized weight map that characterizes the deformation constraint intensity of different anatomical regions. S103. In the space where the preoperative CT image is located, the reachable scanning space of the ultrasound probe is determined based on the anatomical structure label map and / or the voxel-level continuous regularized weight map. S104. Within the reachable scanning space, the acquisition pose of the ultrasound probe is updated according to the action output by the pre-set reinforcement learning strategy network, and the corresponding ultrasound image frame sequence is acquired based on the updated acquisition pose. S105. Perform non-rigid registration between the ultrasound image frame sequence and the preoperative CT image input pre-constructed registration network to obtain the deformation field; The loss function of the registration network includes a regularization constraint term constructed based on the voxel-level continuous regularized weight map. The regularization constraint term is used to apply differentiated deformation constraints to different anatomical regions so that the deformation intensity of structurally sensitive regions is more restricted, while flexible tissue regions are allowed to have a larger degree of deformation freedom.

[0028] In this embodiment, a preoperative CT image is acquired. Based on the preoperative CT image and a pre-constructed prior space model, an anatomical structure label map corresponding to the preoperative CT image is acquired. A voxel-level continuous regularized weight map is acquired based on the anatomical structure label map and the preoperative CT image. Based on the anatomical structure label map, the initial pose information of the probe used to capture ultrasound images is determined, and the current pose information of the probe is obtained. Ultrasound images are acquired based on the current pose information of the probe.

[0029] In this embodiment, as an optional embodiment, the preoperative CT images include, but are not limited to, preoperative three-dimensional industrial computed tomography (ICT).

[0030] In this embodiment, the preoperative CT image is used as the registration reference image, and the data is presented in the form of three-dimensional volume data, denoted as... Where H, W, and D correspond to the number of voxels in the longitudinal (height), transverse (width), and axial (depth) dimensions of the data in the preoperative CT image, respectively.

[0031] In this embodiment, as an optional implementation, the preoperative CT image is input into a pre-constructed prior space model to obtain an anatomical structure label map corresponding to the preoperative CT image. The anatomical structure label map is marked with labels indicating the included categories, which include, but are not limited to: liver parenchyma, main portal vein, branch vessels, and tumor. The anatomical structure label map contains structurally sensitive information from the preoperative CT image, represented as follows: Where C is the number of tissue categories, corresponding to the classification, and the constructed anatomical structure label map can be used for subsequent regularization guidance and registration accuracy evaluation.

[0032] In this embodiment, as an optional embodiment, determining the initial pose information of the probe used to capture ultrasound images based on the anatomical structure label map includes: Gradient features are obtained from the preoperative CT images; Based on the anatomical structure label map, the preoperative CT image, and the gradient features, a voxel-level anatomical label map is obtained to determine the initial pose information of the probe used to capture ultrasound images.

[0033] In this embodiment, the anatomical structure label diagram ( (This can be based on structural-level annotations of CT images, segmented using manual or semi-automatic prior spatial models, resulting in images containing the main classifications. These images define the spatial extent of the overall anatomical region, providing macroscopic structural partitioning.) Voxel-level anatomical labeling ( Based on the anatomical structure label map, by assigning a unique classification (anatomical category) index or attribute weight to each voxel in the anatomical structure label map, a one-to-one anatomical semantics is established for each pixel / voxel. This can serve as a supervision signal for subsequent λ-map network training and improve the accuracy of the determined initial pose information of the probe.

[0034] In this embodiment, as an optional implementation, the voxel-level anatomical label map is obtained using the following formula:

[0035] in, These are preoperative CT images, represented as CT grayscale images, i.e., raw CT volume data. Gradient features; · This represents a mapping function that generates a voxel-level anatomical label map by fusing anatomical structure label maps with CT grayscale and gradient features to produce voxel-by-voxel classification results.

[0036] Figure 2 This is a schematic diagram of a closed-loop path modeling and acquisition strategy based on reinforcement learning in an ultrasound image registration method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a closed-loop path modeling and acquisition strategy based on reinforcement learning in an ultrasound image registration method provided in an embodiment of the present invention.

[0037] The following combination Figure 2 and Figure 3 Please provide an explanation.

[0038] In this embodiment, as an optional embodiment, a voxel-level anatomical label map is obtained based on the anatomical structure label map, the preoperative CT image, and the gradient features, including: A01, perform grayscale conversion on the preoperative CT image to obtain the original CT volume data; In this embodiment, the pixels of the preoperative CT image are converted to grayscale to obtain the raw CT data, denoted as: .

[0039] A02, Based on a pre-constructed prior space model, obtain the anatomical structure label map corresponding to the original CT body data; A03, perform region cropping and interpolation resampling on the anatomical structure label map to obtain a continuous probability label map whose voxel spacing is consistent with the voxel spacing in the preoperative CT image. In this embodiment, interpolation resampling is performed on each structural mask to make the voxel spacing consistent with the original CT voxels, and Gaussian smoothing is performed at the boundaries to obtain a continuous probability label map.

[0040] A04, extract the texture histogram from the continuous probability type label map, and obtain voxel features based on the preoperative CT image, the gradient features and the texture histogram; In this embodiment, low-level features such as local CT grayscale mean, gradient features, and texture histogram are extracted at each voxel location to constitute voxel features. As an optional embodiment, voxel features are obtained using the following formula:

[0041] In the formula, Voxel characteristics These are preoperative CT images. Gradient features This is a texture histogram.

[0042] A05. Based on a pre-set mapping function, the anatomical structure label map, the preoperative CT image, and the gradient features are mapped to obtain a voxel-level anatomical label map. Based on the voxel features and a pre-built lightweight classification model, the classification of the voxel-level anatomical label map is obtained.

[0043] In this embodiment, voxel features are classified and mapped to obtain a voxel-level anatomical label map containing classifications.

[0044] In this embodiment, labeled structural regions can be used as supervised samples to train a lightweight classification model, such as the RandomForest model or a small CNN model, to map continuous voxel features to discrete anatomical categories or voxel-level anatomical label maps. As an optional embodiment, the voxel-level anatomical label maps are classified using the following formula:

[0045] In the formula, This is a voxel-level anatomical labeling image. For categories or classifications.

[0046] In this embodiment, as an optional embodiment, the method further includes: Spatial consistency correction is performed on the voxel-level anatomical label map using conditional random fields or 3D morphological operations.

[0047] In this embodiment, the voxel-level anatomical labeling image undergoes smoothing and consistency correction. For example, spatial consistency correction is performed on the voxel-level anatomical labeling image using Conditional Random Field (CRF) or 3D morphological operations (opening / closing operations, boundary repair) to remove isolated noise voxels in the voxel-level anatomical labeling image, further improving the data quality of the voxel-level anatomical labeling image and resulting in a more accurate final voxel-level anatomical labeling image. ∈{0,1,…,C} H×W×D Each voxel carries an anatomical category index, which can be directly used for training supervision or regular mask generation of λ-map networks.

[0048] In this embodiment, as an optional implementation, generating a voxel-level continuous regularized weighted map based on the anatomical structure label map and the preoperative CT image includes: Based on the category information of different anatomical structures in the anatomical structure label map, initial discrete regular weights are assigned to each anatomical structure according to the preset category-deformation rigidity mapping relationship; Spatial filtering and smoothing are performed on the initial discrete regularized weights to obtain a regularized control continuous distribution function that transitions continuously at the boundaries of the anatomical structure. Based on the regularized control continuous distribution function, the voxel-level continuous regularized weight map is generated. In the voxel-level continuous regularized weight map, the regularized weight of voxels corresponding to structurally sensitive regions including the main blood vessel trunk and tumor boundary is higher, while the regularized weight of voxels corresponding to flexible tissue regions including liver parenchyma is lower. The voxel-level continuous regularized weight map is used to apply differentiated deformation constraints to different anatomical regions in the regularization constraint term of the registration network.

[0049] In this embodiment, as another optional embodiment, obtaining a voxel-level continuous regularized weighted map based on the anatomical structure label map and the preoperative CT image includes: A051, Extract the classification information of each voxel in the voxel-level anatomical label map corresponding to the anatomical structure label map; In this embodiment, the voxel-level anatomical label map corresponding to the preoperative CT image is used. In the process, the classification information of each voxel is read, such as liver parenchyma, blood vessels, tumors, etc.

[0050] A052 sets initial deformation constraint weights that reflect the deformability of different categories, and constructs a mapping function between the initial deformation constraint weights and the voxel-level anatomical label map; In this embodiment, as an optional embodiment, the initial deformation constraint weight is set to a low constraint (high flexibility) for the voxel region corresponding to the liver parenchyma, the initial deformation constraint weight is set to a high constraint (strong rigidity) for the voxel boundary region corresponding to the blood vessels and tumors, and the initial deformation constraint weight is set to a medium constraint for the cavity region.

[0051] In this embodiment, as an optional implementation, the mapping function between the initial deformation constraint weights and the voxel-level anatomical label map is as follows:

[0052] In the formula, For mapping relationship functions, The initial deformation constraint weights.

[0053] A053, using spatial filtering to smooth the mapping function, obtains a regularized control continuous distribution function; In this embodiment, as an optional embodiment, spatial filtering includes, but is not limited to, 3D Gaussian convolution filtering and local mean filtering, thereby smoothing discrete classification (category) weights into a continuous distribution.

[0054] In this embodiment, the regularized control continuous distribution function is expressed as follows:

[0055] In the formula, This is a continuous distribution function for regular control. Through spatial filtering, it is possible to ensure that the transition region between different structures has continuously varying regular control. λ This value helps avoid abrupt changes in the deformation field.

[0056] A054. Based on the regularized control continuous distribution function, obtain a voxel-level continuous regularized weight map characterizing the deformation constraint strength.

[0057] In this embodiment, the regularized control continuous distribution function is optimized and normalized at the boundary: by adding a gradient penalty term in the boundary region, the regularized control continuous distribution function, i.e., the regularized control graph ( λ-map In high gradient regions (organ boundaries), the gradient is smoother, and it is eventually normalized to [0,1] or [...]. λmin,λmax ] interval.

[0058] In this embodiment, the voxel-level continuous regularized weight graph is represented as follows:

[0059] In this embodiment, in the voxel-level continuous regularized weighted graph, the λ value of each voxel represents the deformation constraint strength of that voxel point, which is used for the weighting of the regularization term of the subsequent registration network, reflecting the difference in the deformation tolerance of each region, and is used to constrain the non-rigid deformation capability of the registration network, so as to realize adaptive constraint on the deformability of different tissues.

[0060] In this embodiment, in non-rigid registration tasks, different anatomical regions exhibit significantly different tolerances to deformation. For example, liver parenchyma allows for a certain range of slow deformation, while structures such as vascular trunks and tumor boundaries require stronger geometric stability and boundary alignment capabilities. Related technologies using a uniform deformation regularization strategy struggle to maintain overall flexible deformation capability while ensuring alignment of key structures. This embodiment proposes a spatially prior-guided continuous regularized weight control strategy. By constructing a voxel-level continuous regularized weight map λ-map, differentiated deformation constraints are provided for the registration process to achieve adaptive deformation regulation of anatomical structures.

[0061] In this embodiment, λ-map is a voxel-level continuous regularized weighted map, whose domain is consistent with the preoperative CT image, and is denoted as λ-map. Its value range is [0,1]. The higher the value, the more sensitive the current region is to deformation, and the stronger the regularization constraint should be applied during the registration process. By assigning a regularization weight based on factors such as structural semantics, boundary sensitivity and topological connectivity to each voxel position, a voxel-by-voxel differentiated regularization term modulation parameter is provided for the subsequent registration model in the loss function.

[0062] Anatomical structure label diagram L anat First, voxel masks are obtained through one-hot encoding. Then, the category → weight mapping function f c (Assign high weights to blood vessel / tumor boundaries and low weights to the solid area) Generate an initial weight map. Then, voxel-level Gaussian smoothing and boundary fidelity tuning are applied to obtain a continuous λ-map (0–1 normalization), which serves as the voxel-by-voxel regularization term in the subsequent registration loss.

[0063] In this embodiment, each category label corresponds to a specific anatomical region, such as parenchyma, blood vessels, tumor, cavity, etc. This embodiment constructs a category (classification)-regular weight mapping function. In this process, high regularization weights are assigned to the main blood vessel trunk and lesion boundaries, while low regularization weights are assigned to regions with high deformation tolerance. Subsequently, a voxel-level convolution smoothing operation is used to apply Gaussian filtering to the initial λ-map, eliminating gradient discontinuities at class boundaries and obtaining a structurally constrained, continuous prior map, i.e., a voxel-level continuous regularized weight map. .

[0064] In this embodiment, a λ-map prediction network can also be designed based on a voxel-level continuous regularized weight map to achieve the ability to automatically generate a continuous regularized weighted λ-map from preoperative CT images. As an optional embodiment, the λ-map prediction network structure adopts an encoder-decoder architecture, with the input being the preoperative CT image I. CT The output is a continuous λ-plot. , and the target (voxel-level continuous regularized weighted graph) Alignment is maintained at each voxel location. The encoder employs a layer-by-layer shrinking convolutional structure to extract multi-scale spatial structural semantics. The decoder restores spatial resolution through deconvolutional layers and uses skip connections to fuse shallow detail information to maintain clear structural edges. The output layer uses a sigmoid activation function to map the result to the 0–1 range, ensuring that the λ value conforms to physical constraints.

[0065] In this embodiment, to optimize the prediction performance of the λ-map prediction network, multiple loss functions are used for joint supervision during training. The first term is the mean squared error loss, which is used to constrain the regression accuracy of the prediction results and the target λ-map at the voxel level.

[0066] Where N = H × W × D represents the total number of prime numbers. and These represent the predicted and target values, i.e., the predicted classification and the actual classification. H, W, and D represent the height, width, and number of slices of the CT volume data in voxel coordinates, respectively; the total prime number corresponds to the physical dimension Δ. x ,Δ y ,Δ z (Unit: mm), voxel spacing is s x , s y , s z Voxel index ( i , j , k ) and physical coordinates ( x , y , z The conditions between ) are met.

[0067] The second term is the boundary sensitivity loss, used to enhance the response capability of the predicted λ-map at structural boundaries. As an optional implementation, a gradient-enhanced boundary map G(Lanat) is constructed using voxel-level anatomical label maps, and the gradient-aware weighted loss function is defined as follows:

[0068] By using a gradient-aware weighted loss function, we can encourage more accurate prediction of λ values ​​at the boundaries of anatomical structures, which helps to improve the conservatism of subsequent deformations on the structural boundaries.

[0069] The third term is the smoothing regularization term, which controls the local continuity of the λ-map and avoids deformation instability caused by excessive oscillations. The smoothing regularization term function is as follows:

[0070] in Voxel representation The neighborhood set of the constraint ensures smooth local λ value changes, suppressing the network from generating non-anatomically reasonable local anomalous responses.

[0071] In this embodiment, the total loss of the λ-map prediction network is the weighted sum of the above three terms:

[0072] Where α, β, and γ are loss weight coefficients, which are optimized based on training stability and generalization performance. Through training, a convergent λ-map prediction network is obtained, which can be used to obtain a voxel-level continuous regularized weight map.

[0073] In this embodiment, the trained λ-map prediction network is used to guide non-rigid registration. In the registration model, the deformation field generation network typically optimizes the deformation vector field. To achieve optimal alignment between the source and target images.

[0074] In related technologies, the regularization term in the registration loss takes the following form:

[0075] This embodiment extends it to a weighted form:

[0076] in, This represents the predicted value of the λ-map at voxel x.

[0077] In this embodiment, a weighted mechanism is used to make deformation more restricted in important structural regions (λ high), thereby preserving the accuracy of alignment of key anatomical structures; while in ordinary regions (λ low), flexible deformation is allowed, thereby improving the overall registration adaptability and stability.

[0078] In this embodiment, as an optional implementation, a voxel-level continuous regularized weight map can be used as an auxiliary input, along with the source image (ultrasound image) and the target image, into the attention module of the registration network. This allows the registration network to focus more on regions with strong structural responses when learning deformation, thereby improving learning efficiency.

[0079] In this embodiment, as an optional embodiment, determining the initial pose information of the probe used to capture ultrasound images based on the anatomical structure label map includes: A11, based on the anatomical structure label map, mark the classification regions corresponding to the preset classification; In this embodiment, as an optional embodiment, the corresponding classification regions include, but are not limited to: liver parenchyma region, vascular region, and tumor region.

[0080] A12, expand the preset voxel range outward from the classification region to obtain the reachable space, and preprocess the reachable space; In this embodiment, as an optional embodiment, taking the liver region as a preset area as an example, the pre-set voxel range is extended outward from the CT liver region as the center, for example, 10mm–30mm, to generate an accessible space.

[0081] In this embodiment, as an optional embodiment, expanding the classification region outwards by a preset voxel range to obtain an accessible space includes: A121, obtain the starting mask of the classification region, and use the starting mask to perform masking processing on the classification region to obtain the masked classification region; In this embodiment, as an optional implementation, a voxel-level mask of the liver structure is used. (Align with CT voxels).

[0082] A122, perform 3D morphological dilation on the mask classification region to obtain the dilated mask classification region; In this embodiment, the Euclidean distance d(x) (in mm) from the voxel point within the dilated mask classification region to the liver boundary is calculated. Assuming the safety boundary m∈[5,20]mm, 3D morphological dilation is performed on the voxel-level mask Lliver to ensure that each voxel point within the dilated mask classification region satisfies:

[0083] A123, based on the liver surface in the dilation mask classification region, obtain the extrusion classification region containing the extrusion depth; In this embodiment, the extrusion depth is obtained based on the surface normal of the dilation mask classification region, that is, the extrusion depth is obtained by the normal n of the liver surface S extruding dmin, dmax (e.g., 0–30 mm) inside and outside the body, thus obtaining the extrusion classification region:

[0084] A124, obtain the union of the expansion mask classification region and the extrusion classification region to obtain the preliminary reachable space; In this embodiment, the union of the dilated mask classification region and the extruded classification region is as follows:

[0085] A125, based on the pre-constructed exclusion mask, excludes invalid voxels in the initial reachable space to obtain the reachable space.

[0086] In this embodiment, a binary exclusion mask Minvalid (bone / air / lung air chambers / external body, etc.) is constructed, and morphological closure operation is performed to remove isolated points, resulting in an effective region mask, which can be obtained as follows:

[0087] In this embodiment, the reachable space is obtained by performing 3D expansion / surface normal shell on the liver segmentation voxel-level mask Lliver. .

[0088] In this embodiment, the exclusion mask is generated class-by-class based on each category: External / Air: CT Intensity Threshold < 700HU (air) + extracellular area (obtained by skin / body surface segmentation); Bone / rib: Bone mask obtained by threshold > +200~300HU, connectivity + morphological expansion of 1–2mm to form a "no-entry buffer zone"; Lung air chambers / gastrointestinal gas: [ 900,

[500] HU connected domain; can intersect with anatomical segments (lung / stomach) to enhance robustness; Deep regions far from the target: voxels with a distance to the liver surface > dmax (e.g., 30 mm); Field of view geometry limitation: Based on the probe sector / cone FoV model (start point = probe position, opening angle = equipment calibration), voxels outside the FoV are marked as invalid; Acoustic window difference: Voxels traversing bone / gas paths are quickly identified as invalid using ray projection.

[0089] In this embodiment, the reachable space provides policy updates and sampling constraints, ensuring that subsequent path searches are performed only within the voxel neighborhood of Mvalid=1.

[0090] In this embodiment, 3D closing operations (filling holes) and opening operations (removing small artifacts) are performed on Minvalid, and then the complement of the largest connected component is retained as the candidate valid region to obtain the valid region within the reachable space:

[0091] In this embodiment, path search and policy update are performed only in the neighborhood of Mvalid=1 to prevent running out of the target or falling into the bone / gas occlusion area.

[0092] In this embodiment, the reachable space is preprocessed, i.e., through a mask. By excluding invalid areas within the reachable space, the acquisition space range is obtained. This acquisition space range can restrict path updates to only be performed within the effective dissection neighborhood, preventing acquisition from deviating from the target area and ensuring that the acquisition path is within a safe and effective range.

[0093] In conclusion,

[0094] in To obtain the clinically accessible region based on the 3D expansion or surface normal extrusion shell of the target organ. The field of view is obtained based on the probe pose and the opening angle of the device. This excludes invalid areas formed by combining masking and line-of-sight rules for bone / air / external environments. Subsequent path search and strategy updates only occur within these areas. It will be carried out internally.

[0095] In this embodiment, as an optional implementation, the reachable space is preprocessed, including: Use masks to exclude ribs, lung cavities, and external regions from the reachable space.

[0096] A13. In the preprocessed reachable space, select imageable points on the liver surface to obtain the initial pose information of the probe.

[0097] In this embodiment, in the reachable space (CT space), the imageable points on the liver surface are selected as the initial pose information of the probe.

[0098] In this embodiment, as an optional implementation, the initial pose information of the probe is represented as follows:

[0099] in, This is the initial pose information of the probe. The initial spatial coordinates of the probe; These are the probe's initial pitch angle, representing the probe's vertical tilt angle relative to the horizontal plane; yaw angle, representing the probe's rotation angle around its vertical axis, used to describe the left and right rotation direction; and roll angle, representing the probe's rotation angle around its own axis, reflecting the probe's lateral tilt.

[0100] In this embodiment, the probe acquires ultrasound images based on the probe's initial pose information.

[0101] In this embodiment, the prior space model can be constructed by acquiring preoperative CT images and based on the preoperative CT images, the corresponding anatomical structure label map, and the deformation stiffness weights of the corresponding categories.

[0102] In this embodiment, based on the coordinate system of the ultrasound image and the coordinate system of the preoperative CT image, the ultrasound image is transformed into an ultrasound transformed image corresponding to the coordinate system of the preoperative CT image. As an optional embodiment, transforming the ultrasound image into an ultrasound transformed image corresponding to the coordinate system of the preoperative CT image, based on the coordinate system of the ultrasound image and the coordinate system of the preoperative CT image, includes: Based on the coordinate system of the ultrasound image and the coordinate system of the preoperative CT image, a coordinate system transformation matrix is ​​obtained to transform the coordinate system of the ultrasound image to the coordinate system of the preoperative CT image. The product of the ultrasound image and the coordinate system transformation matrix is ​​calculated to obtain the ultrasound transformed image.

[0103] In this embodiment, based on the current acquisition pose of the ultrasonic probe... The acquired pose is then mapped to the CT coordinate system using a pre-acquired affine or rigid matrix.

[0104] In this embodiment, the next probe pose information is obtained based on the historical path information composed of the voxel-level continuous regularized weight map, anatomical structure label map, ultrasound transformed image, preoperative CT image, and probe pose information, as well as the pre-trained path model.

[0105] In this embodiment, as an optional implementation, a path model is pre-trained to predict the path, enabling the probe to acquire ultrasound images according to the predicted path. The path model (reinforcement learning policy network) outputs an action vector. This is used to determine the translation and rotation operations that the probe should perform in the next moment.

[0106] In this embodiment, as an optional implementation, pre-training the path model may include the following steps: B11, based on a pre-set feature extraction network, performs feature extraction on the ultrasound transformation image to obtain ultrasound texture features, and extracts CT structural features corresponding to the ultrasound transformation image location from the preoperative CT image, and calculates cross-modal similarity index based on ultrasound texture features and CT structural features. In this embodiment, the current frame ultrasound image (Ultrasonic transformed image) Feature extraction is performed through a feature extraction network to obtain ultrasonic texture features. Meanwhile, CT structural features corresponding to the ultrasound image locations are extracted from preoperative CT images, and cross-modal similarity indices are calculated based on ultrasound texture features and CT structural features.

[0107] In this embodiment, as an optional embodiment, CT structural features corresponding to the ultrasound transformed image location are extracted from the preoperative CT image, including: Centered on the pose point of the current ultrasound probe mapped to the CT coordinate system, a three-dimensional sub-block matching the field of view of the ultrasound image is extracted from the CT volume data corresponding to the preoperative CT image. The extracted 3D sub-blocks are interpolated and resampled to generate CT structural features corresponding to the spatial dimensions of the ultrasound image.

[0108] In this embodiment, CT structural features are also referred to as local CT images. As an optional embodiment, the local CT image can also be preprocessed, such as by grayscale normalization and edge enhancement, so that the cropped local CT image is comparable to the ultrasound image at the texture level, and can be used for similarity measurement and reinforcement learning state input.

[0109] In this embodiment, as an optional embodiment, cross-modal similarity metrics include, but are not limited to: mutual information (MI) and structural similarity index measurement (SSIM).

[0110] B12, based on the cross-modal similarity index of adjacent acquisition cycles corresponding to the path model, obtains similarity gain; In this embodiment, as an optional implementation, the similarity gain, i.e., the image similarity enhancement, is calculated using the following formula:

[0111] For the ultrasound frame image of the current acquisition period, in this embodiment, the calculated similarity gain can be used as the reward signal for subsequent reinforcement learning.

[0112] B13, calculates the immediate reward based on the calculated similarity gain and the pre-acquired deformation cost term; In this embodiment, considering the improvement in image similarity between the current image frame and the target image (such as mutual information and structural similarity) and the deformation energy cost, a reward function is used. Define an immediate reward as the objective function optimization signal for path policy learning.

[0113] In this embodiment, The immediate reward is used to measure the overall benefit of the current action in terms of registration accuracy and cost, and serves as a direct feedback signal for strategy optimization.

[0114] Mutual Information Gain represents the increase in the registration similarity between the current frame ultrasound image and the CT target image after this step, reflecting the degree of improvement in cross-modal alignment.

[0115] The Deformation Energy Cost describes the non-rigidity and smoothness loss of the current deformation field. A larger value indicates more severe or unreasonable deformation.

[0116] The trade-off coefficient controls the balance between similarity enhancement and deformation cost, and is generally taken as 0.1–0.3.

[0117] In this embodiment, the current frame image is the currently acquired ultrasound image frame. The target image corresponds to the current acquisition pose (pt). The target image is either a preoperative CT reference image or a target image corrected by spatial prior, serving as the "reference alignment target" in the registration process.

[0118] In this embodiment, reinforcement learning is used to adjust the current frame at each step. The image similarity enhancement (mutual information MI, structural similarity SSIM, etc.) is calculated between the target image and the target image. The similarity enhancement ΔMIt in the reward function represents the improvement in the matching degree between the new ultrasound frame and the preoperative CT image after performing one action.

[0119] B14: Based on the immediate reward, the registration score is obtained. If the improved registration score is greater than the preset score threshold, ultrasound image sampling is performed according to the current acquisition path. If the registration score is less than the score threshold, the strategy parameters of the path model are adjusted according to the preset reward function.

[0120] In this embodiment, as an optional implementation, the path model can also be optimized based on immediate rewards.

[0121] In this embodiment, the policy network parameters of the path model are updated based on immediate rewards, thereby achieving optimal path learning through gradient ascent or Actor-Critic mechanisms.

[0122] In this embodiment, for each update of the acquisition path, an immediate reward is calculated and accumulated based on the similarity improvement after registration between the current frame and the previous frame, as well as the corresponding deformation energy cost, to obtain the current cumulative reward. If, in several consecutive iterations, the increase in registration score and the increase in cumulative reward are both lower than a preset threshold, the path strategy network is determined to have converged, and the strategy parameters are no longer updated; only the current path direction and acquisition strategy are maintained. Simultaneously, the registration transformation field obtained at this time is used as the final deformation field Φ*. If, during the iteration process, the registration error in a local region exceeds the error threshold, the current action is penalized by giving a negative reward to guide the strategy network to adjust the next acquisition pose, avoiding repeated coverage or deviation from the target region. When the convergence condition is met (e.g., the registration similarity improvement |ΔMI| is continuously lower than the threshold ε), the path update and iterative calculation are stopped, and the final ultrasound-CT fused image and the corresponding optimal acquisition path are output.

[0123] In this embodiment, the optimized registration transformation field Φ is obtained by performing the optimal action. Based on the registration transformation field, registration is performed to obtain ultrasound-CT fused images, thereby providing a precise spatial alignment basis for intraoperative navigation or subsequent AI diagnosis.

[0124] In this embodiment, action vectors are obtained through training a reinforcement learning policy network. The reinforcement learning policy network then selects the action that maximizes the reward based on the current state vector, which becomes the action vector. As an optional embodiment, the reward function described above is reorganized into the following formula:

[0125] in: To enhance the mutual information between the current frame image and the preoperative CT image; This represents a change in local structural similarity. This is the cost of path length and attitude change.

[0126] in, This represents the change in the corresponding index before and after two consecutive data acquisition actions, used to characterize the improvement in registration quality brought about by a single-step action; for example... This characterizes the increase in mutual information between the ultrasound image and the preoperative CT image after this step is performed. Characterizes the increase in similarity to the corresponding anatomical structure. Path cost term. This is used to penalize cases with excessively long paths, repeated sampling, or drastic pose changes; its value increases with the magnitude of probe displacement and the change in pose. Coefficients α, β, and γ are weight parameters greater than zero, used to balance the relative importance of mutual information enhancement, structural similarity enhancement, and path cost in reward calculation.

[0127] In this embodiment, a new ultrasound frame is acquired after each action is performed. The results are then input into the registration network for matching, forming a closed-loop feedback.

[0128] In this embodiment, the reward function Rt comprehensively considers the following indicators: 1) Registration similarity improvement 1) Mutual information change between the current frame image and the preoperative CT image; 2) Registration region overlap index, such as ΔDice, used to reflect the structural alignment accuracy; 3) Path cost. That is, the cost of path length and attitude change, including path length, penalty for repeated regions, and penalty for illegal region crossing; 4) Structural response weights This encourages the acquisition path to be close to areas of important anatomical structures. Therefore, as another optional embodiment, the reward function takes the following form:

[0129] Among them, each weight Experience can be set or adjusted through meta-learning based on task trade-offs.

[0130] Based on the current state vector and the previous state vector, as well as the preoperative CT and ultrasound images before and after registration, a registration score is obtained. If the improved registration score is greater than a preset score threshold, ultrasound image sampling is performed according to the current acquisition path. If the registration score is less than the score threshold, the strategy parameters are adjusted according to a preset reward function.

[0131] In this embodiment, if the registration score improves significantly, the current path is retained and sampling continues in that direction; if the registration score decreases, the strategy parameters are adjusted according to the reward gradient, and the path distribution is optimized through the experience replay mechanism, ultimately forming a scanning path with a dynamic optimal strategy automatically generated by the algorithm based on the trade-off between registration accuracy and acquisition efficiency.

[0132] In this embodiment, the source image input is a two-dimensional ultrasound frame sequence. This corresponds to the image observation of each frame along the path. Each frame contains the image's acquisition pose in three-dimensional space. These constitute the state variables in the registration process. The starting point of the path is set by the initialization strategy, and the selection of subsequent actions is generated by a pre-set strategy network to control the direction and amplitude of probe movement.

[0133] In this embodiment, a reinforcement learning policy network (PolicyNetwork), i.e., a pre-trained path model, is used to dynamically generate acquisition paths, which correspond to a series of probe pose information. At each time t, an optimal action at is predicted based on the current state st (including the current ultrasound frame, probe pose, historical registration path, and CT prior) to determine the acquisition point or scanning direction for the next frame.

[0134] In this embodiment, the current frame ultrasound image , Registered reference image partial view, current probe pose The path heatmap and historical paths are generated by overlaying the λ-map and constructing the current state vector.

[0135] In this embodiment, the action space is defined as a fixed discrete set. This represents the step size transformation of translation (along the XYZ axes) or rotation (around the XYZ axes) based on the current position, and the next probe pose information, covering translation and rotation operations within a finite step size. Preferred discrete set. ,in For example, the translation step size is 2–5 mm, and the rotation step size is 2–5°; the actual deployment can be set according to the equipment accuracy. The current frame ultrasound image reflects the imaging angle of the probe at this time; historical path information (historical path trajectory) is also available. The system is used to characterize the scanned area and infer the probe's motion trend; local image features are feature information extracted from the current frame image, such as edge texture and grayscale gradient; anatomical context is derived from preoperative CT images and λ-map priors, providing structural constraint information for the ultrasound frame. The historical trajectory encodes the path coverage area in the form of a three-dimensional sparse voxel map, and combines it with the anatomical λ-map overlay to generate a path heatmap, used to explicitly model the spatial difference between the visited area and the target uncovered area. Furthermore, structural label maps can also be used... L anat The defined spatial prior mask is projected onto the current acquisition pose direction to form a local structure graph guiding state. The state vector is composed as follows:

[0136]

[0137] in, To be in the current pose from Reference sub-blocks obtained by cropping and resampling volume data; These are local texture / edge features extracted from the current frame; for k Step history trajectory raster (including translation / rotation encoding); This is the normalized λ-map.

[0138] In this embodiment, the current position will be updated after the action is executed. Based on this, the next frame image and the new state S are generated. t+1 ; The step length for historical retrospection indicates looking back at the past. Records of each acquisition and movement; This represents a trajectory grid formed from several past frames, storing the probe's path sampling sequence in space, including translation and rotation information; local image features can be extracted from the current frame of the ultrasound image using a convolutional neural network. Multi-scale grayscale and texture features are extracted to describe local tissue morphology and boundary changes.

[0139] In this embodiment, the optimal action is predicted based on the state variables according to the pre-set path model. The optimal action includes a small-range translation or rotation operation. The path model (policy network) is a network based on state variables and actions. In this embodiment, as an optional implementation, the policy network is: The optimal action is .in, For optimal action, It is a state variable.

[0140] In this embodiment, the next probe pose information can be obtained by executing the predicted optimal action.

[0141] In this embodiment, the probe pose is updated based on the optimal action, and the probe is moved to the position corresponding to the next probe pose information, so as to obtain a new ultrasound image based on the updated probe pose (next probe pose information).

[0142] In this embodiment, after performing the optimal action, the probe pose is updated as follows: And acquire new ultrasound frames in real time. This is to form new state variable inputs.

[0143] In this embodiment, the probe is adjusted to the pose corresponding to the pose information of the next probe in order to acquire ultrasound images.

[0144] In this embodiment, after acquiring ultrasound images, registration is performed based on the acquired ultrasound images until all ultrasound images are acquired.

[0145] In this embodiment, to address the non-rigid registration between preoperative CT images and intraoperative ultrasound images during liver surgery navigation, a registration method combining reinforcement learning path modeling and spatial prior regularization is proposed. Through strategic image acquisition and region-differentiated registration control, the fusion accuracy of key structures (such as the portal vein and tumor margins) is improved, while reducing reliance on human experience. The registration is based on preoperative CT images, combined with real-time ultrasound image input, and sequentially goes through multiple stages including path modeling, regularization prediction, registration execution, and strategy feedback, forming a closed-loop mechanism of "acquisition-registration-guidance-optimization". Thus, by introducing a path memory mechanism to construct a reinforcement learning policy network and generating a spatially variable regularized graph (λ-map) from anatomical atlas, efficient scanning and high-precision registration are synergistically optimized. For example, by introducing a reinforcement learning-based path modeling strategy, the system can learn how to select the optimal acquisition direction and position in non-rigid registration scenarios, thereby efficiently obtaining ultrasound images of structurally identifiable regions and improving the registration accuracy with the CT target image. In this way, by simulating the closed-loop process of perception-feedback-decision in the acquisition path, the movement behavior of the probe at each step can be dynamically controlled, so that the path not only meets the anatomical accessibility constraint, but also achieves an optimal balance between maximizing information and deformation constraint.

[0146] In this embodiment, the policy network structure is based on a dual-branch encoder-fusion architecture, with the input being the feature vector encoded by each submodule in the state vector. Ultrasound image frames and CT images are processed through a shared convolutional network to extract image features. The path trajectory heatmap is compressed using a 3D-CNN to obtain spatial coverage features. The pose vector and structural mask are mapped to a unified dimension using a multilayer perceptron (MLP) and then concatenated with the image features. The fused state representation is fed into a three-layer fully connected network for policy output, ultimately generating an action distribution π(a|st;θ), where θ represents the policy parameters.

[0147] During the training phase, the Proximal Policy Optimization (PPO) algorithm is used, along with entropy regularization and a value function baseline to improve convergence stability.

[0148] In this embodiment, a simulated path acquisition and image registration linkage iterative mechanism is adopted during policy training. After each policy network output action, the registration similarity is evaluated and the reward is calculated through simulated registration to update the policy parameters. Sample acquisition employs a parallel trajectory exploration method to avoid getting trapped in local optima. An experience replay caching mechanism is used to strengthen the memory retention of rare high-reward states, improving the policy's generalization ability. The training convergence criterion is based on the long-term cumulative reward average improvement rate; training is terminated if there is no significant improvement or the maximum number of training rounds is reached after several consecutive rounds.

[0149] In this embodiment, the path update phase employs greedy sampling or noisy strategy sampling to perform policy inference. After each acquisition action, a new state is generated based on real-time images and state information, and recursively updated until the registration score reaches a set threshold or the maximum number of steps is exhausted. Path visualization evaluation shows that the generated path has significant regional guidance characteristics, actively avoiding low signal-to-noise ratio regions and tending towards structurally significant regions, achieving adaptive policy perception control and dynamic path adjustment.

[0150] In this embodiment, during the non-rigid registration process between preoperative CT images and intraoperative ultrasound images, the registration network, as the core component of deformation field prediction, directly determines the resolution, continuity, and structural preservation capability of the deformation representation through its structural design. Based on path modeling and regularized weight prediction, this embodiment constructs a multi-scale structure-aware registration sub-network and combines it with a λ-map-guided regularized control mechanism to achieve precise alignment of key anatomical structures and high-quality deformation field generation.

[0151] In this embodiment, the input to the registration network is a US-CT image pair (IUS, ICT), where the US image comes from the current frame image generated by the enhancement strategy, and the CT image is a local cropped region or the overall image view of the registration target image. The two images exhibit significant heterogeneity in morphology, intensity distribution, and texture patterns. Directly stitching them together before input may lead to feature alignment failure. To mitigate modal differences, this embodiment employs a dual-encoder structure to extract intra-modal features from both the ultrasound and CT images. Furthermore, during the decoding and fusion stage, a cross-modal feature alignment mechanism is constructed to effectively improve the structural recognizability in the joint feature space.

[0152] In this embodiment, the specific network structure adopts a symmetrical multi-scale UNet architecture. Each modal encoder contains four levels of convolutional modules, and each level of convolutional module consists of two ConvBlocks (convolution + BN + ReLU). The sampling method is 2×2 max pooling, which progressively compresses the spatial resolution while extracting deep semantic features. After the two modal encoders concatenate and fuse the bottom-level features, they complete the upsampling restoration operation through a shared decoder to reconstruct the deformation field output with the same size as the input. As an optional embodiment, to enhance the preservation of high-level structural information, a multi-scale feature skip connection mechanism is adopted to guide shallow spatial structural information to participate in the deep inference process, and the output is a three-dimensional vector field. , which represents the deformation mapping from the source image to the target image.

[0153] In this embodiment, to explicitly introduce differentiated regularization control capabilities into the network structure, a λ-map is embedded as a conditional prior into multiple network locations. First, during the encoder feature extraction stage, the λ-map is downsampled to the same resolution as the encoded features of each layer and, after convolutional encoding, is multiplied with the main branch feature map as a channel enhancement factor, thus playing a role in region attention regulation. Second, in the final loss function, the λ-map participates in the weight modulation of the deformation regularization term, resulting in a smoother and more limited deformation field in structurally sensitive regions, while non-critical regions can deform freely to compensate for alignment errors. Therefore, this dual fusion mechanism balances forward inference efficiency and backward constraint stability.

[0154] In this embodiment, a loss function with dual objectives of deformation field supervision and structure preservation is also utilized. The first term is an image alignment loss based on mutual information (MI) or local normalized cross-correlation (LNCC), used to measure the registered source image. Similarity between the target image ICT and the target image:

[0155] In this embodiment, considering the strong speckle noise and weak structural boundaries in ultrasound images, local block-level LNCC is adopted as a more robust similarity metric that can adapt to intermodal gray-level nonlinearity.

[0156] The second term of the loss function is a deformation field smoothing regularization term, used to limit the risks of discontinuous deformation, structural tearing, and irreversible mapping. After fusing the λ-map, its expression is:

[0157] Where Λ(x) is the value of λ-map at point x. This term enhances the continuity of deformation at anatomical boundaries and lesion areas, preventing severe local folding.

[0158] The third term of the loss function is the structure preservation loss Lstruct. Using the known structure label map Lanat, important structural regions are re-segmented after deformation, and the Dice overlap ratio is calculated with the target structure to measure whether the key structures remain consistent after registration.

[0159] Where f( ) indicates the selection of the structural re-identification network or the maximum response channel, which further enhances the ability of the structurally identifiable region to guide the training of the registration model.

[0160] The total loss function is expressed as follows:

[0161] The coefficients α, β, and γ are dynamically adjusted according to the training stage or structural priority to balance registration accuracy and deformation controllability.

[0162] In this embodiment, the registration network can be initialized and pre-trained using simulated ultrasound images and registration labels during the training phase, and then fine-tuned on real acquired data. During the inference phase, the trained model is used to quickly generate deformation fields on each frame and align with the preoperative CT image. Combined with the aforementioned path strategy and regularization prediction module, this forms a complete image fusion process. Thus, through the collaborative design of the registration network structure and regularization mechanism, structure-aware registration of multimodal heterogeneous images is achieved, effectively solving the key problems of unstable registration accuracy and structural misalignment in irregular regions in related technologies. The inference process begins with the path acquisition action of the initial frame. The structural prior map and λ-map constructed based on the preoperative CT image and its anatomical labels serve as guidance information, directing the scanning focus area of ​​the initial path. The first ultrasound image frame is input as the starting state. A state vector is jointly constructed using image content, current pose encoding, historical path mask, and λ-map. The enhancement strategy network generates the current optimal action, i.e., the direction and magnitude of the next scan displacement. This action is represented as a Δp displacement vector in a three-dimensional coordinate system, indicating the direction and number of millimeters to move in the acquisition space.

[0163] After acquiring the next frame of image, the registration network is invoked in real time to perform non-rigid registration between the new frame ultrasound image and the CT image, generating the deformation field at the current moment. The image is t and the registered and fused image. The registration result is scored using an image similarity metric function, commonly including mutual information (MI), normalized cross-correlation (NCC), or the structure label Dice coefficient. Based on the change in score Δs, i.e., the improvement in registration quality, the immediate reward rt for the current action can be calculated, which is used to reinforce policy optimization.

[0164] In summary, the method of this invention is based on preoperative CT images and their anatomical priors. It constructs spatially differentiated deformation constraints through a voxel-level continuous regularized weight map (λ-map), and then combines it with a path modeling strategy based on reinforcement learning to achieve closed-loop collaboration between ultrasound image acquisition, spatial registration, and strategy optimization. Specifically, in each frame acquisition, the path model first predicts the next probe pose based on the current state to acquire a new ultrasound image frame. Then, the registration network performs non-rigid registration between the ultrasound image frame and the preoperative CT image under λ-map constraints to obtain the corresponding deformation field and registration score. Changes in the registration score are fed back to the path model as an immediate reward to update the acquisition strategy. By repeatedly executing the iterative cycle of "acquisition-registration-scoring-feedback", when the registration gain of several consecutive frames is lower than a preset threshold or the scan space traversal is completed, the system stops path updating and outputs the final deformation field and ultrasound-CT fused image. Thus, while ensuring controlled deformation of the anatomical structure, it achieves high-precision and high-efficiency cross-modal registration.

[0165] Throughout the inference process, historical scan trajectories are encoded into two-dimensional path mask maps and three-dimensional voxel scan maps to record visited areas and coverage density. Combined with high-structural-value regions not yet covered in the λ-map, the policy network can adjust action biases accordingly, avoiding repeated paths and improving the structural information acquisition rate. This ensures stable improvement in registration quality while reducing invalid data collection.

[0166] In this embodiment, action selection is performed by randomly sampling the probability distribution output by the policy network, combined with an ε-greedy exploration mechanism. Initially, the exploration weights are high, encouraging the policy to fully explore possible paths in the high-dimensional action space. As the registration score increases, the sampling policy gradually converges to deterministic actions, improving policy stability. Policy updates do not rely on environmental feedback but are explicitly evaluated based on the functional relationship between the acquired image and the registration score, forming a closed-loop reward structure in reinforcement learning.

[0167] This embodiment achieves a systematic improvement in registration accuracy, efficiency, and controllability by providing an ultrasound image registration method that integrates path modeling and spatial prior regularization. The reinforcement learning-driven path strategy endows the acquisition behavior with structure awareness, effectively increasing the coverage density of key regions. Meanwhile, the λ-map-dominated regularization mechanism implements differentiated constraints on structurally sensitive regions, effectively suppressing mismatches and deformation drift during registration. Specifically, λ-map assigns differentiated deformation constraint weights to each voxel, maintaining high rigidity constraints on key anatomical regions such as vascular trunks and tumor boundaries during registration, while granting greater deformation freedom to flexible tissue regions such as liver parenchyma. This significantly enhances the stability and interpretability of the deformation field while improving cross-modal registration accuracy. The two work together to form a closed-loop optimization mechanism of registration-feedback-reacquisition, achieving fine structural alignment without relying on external systems. This method framework possesses good versatility and expansion potential, and is expected to serve a wider range of medical image processing tasks such as multimodal fusion, image navigation, and self-supervised registration, possessing broad value for algorithm transfer and engineering transformation. It has the following beneficial technical effects: Significantly improves registration accuracy: By combining path memory with regional differential regularization, it effectively improves the alignment of key areas such as blood vessel bifurcation and tumor boundaries.

[0168] Improve acquisition efficiency: Reinforcement learning strategies avoid repetitive and ineffective scanning, enabling targeted probe movement and reducing scanning time and operational burden.

[0169] Enhanced model interpretability and controllability: The regularized weight graph (λ-map) has a clear anatomical structure correspondence, which facilitates manual review and parameter adjustment.

[0170] Path memory-based reinforcement learning acquisition strategy: Introducing a historical scan trajectory modeling mechanism to guide the ultrasound probe to efficiently cover key structural regions and optimize the image acquisition path.

[0171] Anatomy-prior-driven spatial variable regularization method: By generating voxel-level λ-maps from anatomical atlases, differential deformation constraints are applied to different regions during non-rigid registration.

[0172] collection Registration Feedback-based closed-loop optimization process: Construct a real-time acquisition and registration closed loop to enable the system to self-adjust and converge registration accuracy, thereby improving its practicality.

[0173] Registration gain Reward function design for joint modeling of "path cost": A reward function is proposed that takes into account both registration effect improvement and action cost control, balancing accuracy and efficiency, and supporting stable training of RL decision-making strategies.

[0174] Based on the same inventive concept, such as Figure 4 As shown, this embodiment of the invention also provides an ultrasound image registration device, the device comprising: The tag acquisition module 401 is used to acquire preoperative CT images and anatomical structure tag maps based on the preoperative CT images. In this embodiment, as an optional embodiment, the preoperative CT images include, but are not limited to, preoperative three-dimensional volumetric computed tomography (CT).

[0175] The pose determination module 402 is used to generate a voxel-level continuous regularized weight map characterizing the deformation constraint intensity of different anatomical regions based on the anatomical structure label map and the preoperative CT image. In this embodiment, as an optional embodiment, the pose determination module 402 is specifically used for: Based on the category information of different anatomical structures in the anatomical structure label map, initial discrete regular weights are assigned to each anatomical structure according to the preset category-deformation rigidity mapping relationship; Spatial filtering and smoothing are performed on the initial discrete regularized weights to obtain a regularized control continuous distribution function that transitions continuously at the boundaries of the anatomical structure. Based on the regularized control continuous distribution function, the voxel-level continuous regularized weight map is generated. In the voxel-level continuous regularized weight map, the regularized weight of voxels corresponding to structurally sensitive regions including the main blood vessel trunk and tumor boundary is higher, while the regularized weight of voxels corresponding to flexible tissue regions including liver parenchyma is lower. The voxel-level continuous regularized weight map is used to apply differentiated deformation constraints to different anatomical regions in the regularization constraint term of the registration network.

[0176] In this embodiment, as another optional embodiment, the pose determination module 402 is specifically used for: Gradient features are obtained from the preoperative CT images; Based on the anatomical structure label map, the preoperative CT image, and the gradient features, a voxel-level anatomical label map is obtained to determine the initial pose information of the probe used to capture ultrasound images.

[0177] In this embodiment, as an optional embodiment, a voxel-level anatomical label map is obtained based on the anatomical structure label map, the preoperative CT image, and the gradient features, including: The preoperative CT images were converted to grayscale to obtain the original CT volume data; Based on a pre-constructed prior space model, the anatomical structure label map corresponding to the original CT body data is obtained; The anatomical structure label map is cropped and interpolated and resampled to obtain a continuous probability label map whose voxel spacing is consistent with the voxel spacing in the preoperative CT image. Extract the texture histogram from the continuous probabilistic label map, and obtain voxel features based on the preoperative CT image, the gradient features, and the texture histogram; Based on a pre-set mapping function, the anatomical structure label map, the preoperative CT image, and the gradient features are mapped to obtain a voxel-level anatomical label map. Based on the voxel features and a pre-built lightweight classification model, the classification of the voxel-level anatomical label map is obtained.

[0178] In this embodiment, as an optional embodiment, the pose determination module 402 is further used for: Extract the classification information of each voxel in the voxel-level anatomical label map corresponding to the anatomical structure label map; For different categories, set initial deformation constraint weights that reflect the deformability of the category, and construct a mapping function between the initial deformation constraint weights and the voxel-level anatomical label map; Spatial filtering is used to smooth the mapping function to obtain a regularized control continuous distribution function; In this embodiment, as another optional embodiment, the pose determination module 402 is further used for: Based on the anatomical structure label map, mark the classification regions corresponding to the preset classifications; Expand the predefined voxel range outward from the classification region to obtain the reachable space, and preprocess the reachable space; In the preprocessed reachable space, imageable points on the liver surface are selected to obtain the initial pose information of the probe.

[0179] In this embodiment, the step of expanding the classification region outwards to a preset voxel range to obtain an accessible space includes: Obtain the starting mask of the classification region, and use the starting mask to perform masking processing on the classification region to obtain the masked classification region; 3D morphological dilation is performed on the mask classification region to obtain the dilated mask classification region; Based on the liver surface in the dilation mask classification region, obtain the extrusion classification region containing the extrusion depth; Obtain the union of the dilation mask classification region and the extrusion classification region to obtain the preliminary reachable space; Based on the pre-constructed exclusion mask, invalid voxels in the initial reachable space are excluded to obtain the reachable space.

[0180] Based on the regularized control continuous distribution function, a voxel-level continuous regularized weight map characterizing the deformation constraint strength is obtained.

[0181] The image transformation module 403 is used to determine the reachable scanning space of the ultrasound probe in the space where the preoperative CT image is located, based on the anatomical structure label map and / or the voxel-level continuous regularized weight map. In this embodiment, as an optional embodiment, the image transformation module 403 is further configured to: Centered on the pose point of the current ultrasound probe mapped to the CT coordinate system, a three-dimensional sub-block matching the field of view of the ultrasound image is extracted from the CT volume data corresponding to the preoperative CT image. The extracted 3D sub-blocks are interpolated and resampled to generate CT structural features corresponding to the spatial dimensions of the ultrasound image.

[0182] The pose update module 404 is used to update the acquisition pose of the ultrasound probe within the reachable scanning space according to the action output by the pre-set reinforcement learning strategy network, and acquire the corresponding ultrasound image frame sequence based on the updated acquisition pose. The ultrasound image update module 405 is used to perform non-rigid registration of the ultrasound image frame sequence with the preoperative CT image input to a pre-constructed registration network to obtain a deformation field. The loss function of the registration network includes a regularization constraint term constructed based on the voxel-level continuous regularized weight map. The regularization constraint term is used to apply differentiated deformation constraints to different anatomical regions so that the deformation intensity of structurally sensitive regions is more restricted, while flexible tissue regions are allowed to have a larger degree of deformation freedom.

[0183] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the ultrasound image registration method in any of the above possible implementations.

[0184] Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0185] Based on the same inventive concept, see [link to inventive concept] Figure 5 This invention also provides an electronic device, including a memory 101 (e.g., non-volatile memory), a processor 102, and a computer program stored on the memory 101 and executable on the processor 102. When the processor 102 executes the program, it implements the steps of the ultrasound image registration method in any of the above possible implementations, which can be equivalent to the ultrasound image registration device described above. Of course, the processor can also be used to process other data or perform calculations. This electronic device can be a PC, server, terminal, or other similar device.

[0186] like Figure 5As shown, the electronic device may also include: memory 103, network interface 104, and internal bus 105. In addition to these components, other hardware may also be included, which will not be described in detail here.

[0187] It should be noted that the aforementioned ultrasound image registration device can be implemented by software. As a device in a logical sense, it is formed by the processor 102 of the electronic device in which it is located reading the computer program instructions stored in the non-volatile memory into the memory 103 for execution.

[0188] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0189] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by special-purpose logic circuitry—such as FPGA (Field Programmable Gate Array) or ASIC (Application-Specific Integrated Circuit), and the device can also be implemented as special-purpose logic circuitry.

[0190] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0191] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0192] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0193] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0194] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0195] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0196] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An ultrasound image registration method, characterized in that, include: Acquire preoperative CT images and anatomical structure labeling maps based on the preoperative CT images; Based on the anatomical structure label map and the preoperative CT image, a voxel-level continuous regularized weight map is generated to characterize the deformation constraint intensity of different anatomical regions. In the space of the preoperative CT image, the reachable scanning space of the ultrasound probe is determined based on the anatomical structure label map and / or the voxel-level continuous regularized weight map; Within the reachable scanning space, the acquisition pose of the ultrasound probe is updated according to the actions output by the pre-set reinforcement learning strategy network, and the corresponding ultrasound image frame sequence is acquired based on the updated acquisition pose. The ultrasound image frame sequence and the preoperative CT image are input into a pre-constructed registration network for non-rigid registration to obtain the deformation field; The loss function of the registration network includes a regularization constraint term constructed based on the voxel-level continuous regularized weight map. The regularization constraint term is used to apply differentiated deformation constraints to different anatomical regions so that the deformation intensity of structurally sensitive regions is more restricted, while flexible tissue regions are allowed to have a larger degree of deformation freedom.

2. The ultrasound image registration method according to claim 1, characterized in that, The method further includes: Gradient features are obtained from the preoperative CT images; Based on the anatomical structure label map, the preoperative CT image, and the gradient features, a voxel-level anatomical label map is obtained to determine the initial pose information of the probe used to capture ultrasound images.

3. The ultrasound image registration method according to claim 2, characterized in that, The process of obtaining a voxel-level anatomical label map based on the anatomical structure label map, the preoperative CT image, and the gradient features includes: The preoperative CT images were converted to grayscale to obtain the original CT volume data; Based on a pre-constructed prior space model, the anatomical structure label map corresponding to the original CT body data is obtained; The anatomical structure label map is cropped and interpolated and resampled to obtain a continuous probability label map whose voxel spacing is consistent with the voxel spacing in the preoperative CT image. Extract the texture histogram from the continuous probabilistic label map, and obtain voxel features based on the preoperative CT image, the gradient features, and the texture histogram; Based on a pre-set mapping function, the anatomical structure label map, the preoperative CT image, and the gradient features are mapped to obtain a voxel-level anatomical label map. Based on the voxel features and a pre-built lightweight classification model, the classification of the voxel-level anatomical label map is obtained.

4. The ultrasound image registration method according to any one of claims 1 to 3, characterized in that, The generation of a voxel-level continuous regularized weighted map based on the anatomical structure label map and the preoperative CT image includes: Based on the category information of different anatomical structures in the anatomical structure label map, initial discrete regular weights are assigned to each anatomical structure according to the preset category-deformation rigidity mapping relationship; Spatial filtering and smoothing are performed on the initial discrete regularized weights to obtain a regularized control continuous distribution function that transitions continuously at the boundaries of the anatomical structure. Based on the regularized control continuous distribution function, the voxel-level continuous regularized weight map is generated. In the voxel-level continuous regularized weight map, the regularized weight of voxels corresponding to structurally sensitive regions including the main blood vessel trunk and tumor boundary is higher, while the regularized weight of voxels corresponding to flexible tissue regions including liver parenchyma is lower. The voxel-level continuous regularized weight map is used to apply differentiated deformation constraints to different anatomical regions in the regularization constraint term of the registration network.

5. The ultrasound image registration method according to claim 2, characterized in that, The determination of the initial pose information of the probe used to capture ultrasound images includes: Based on the anatomical structure label map, mark the classification regions corresponding to the preset classifications; Expand the predefined voxel range outward from the classification region to obtain the reachable space, and preprocess the reachable space; In the preprocessed reachable space, imageable points on the liver surface are selected to obtain the initial pose information of the probe.

6. The ultrasound image registration method according to claim 5, characterized in that, The step of expanding a preset voxel range outward from the classification region to obtain an accessible space includes: Obtain the starting mask of the classification region, and use the starting mask to perform masking processing on the classification region to obtain the masked classification region; 3D morphological dilation is performed on the mask classification region to obtain the dilated mask classification region; Based on the liver surface in the dilation mask classification region, obtain the extrusion classification region containing the extrusion depth; Obtain the union of the dilation mask classification region and the extrusion classification region to obtain the preliminary reachable space; Based on the pre-constructed exclusion mask, invalid voxels in the initial reachable space are excluded to obtain the reachable space.

7. The ultrasound image registration method according to any one of claims 1 to 4, characterized in that, The method further includes: Centered on the pose point of the current ultrasound probe mapped to the CT coordinate system, a three-dimensional sub-block matching the field of view of the ultrasound image is extracted from the CT volume data corresponding to the preoperative CT image. The extracted 3D sub-blocks are interpolated and resampled to generate CT structural features corresponding to the spatial dimensions of the ultrasound image.

8. An ultrasound image registration device, characterized in that, The ultrasound image registration device includes: The tag acquisition module is used to acquire preoperative CT images and anatomical structure tag maps based on the preoperative CT images; The pose determination module is used to generate a voxel-level continuous regularized weighted map characterizing the deformation constraint intensity of different anatomical regions based on the anatomical structure label map and the preoperative CT image. The image transformation module is used to determine the reachable scanning space of the ultrasound probe in the space of the preoperative CT image, based on the anatomical structure label map and / or the voxel-level continuous regularized weight map. The pose update module is used to update the acquisition pose of the ultrasound probe within the reachable scanning space according to the action output by the pre-set reinforcement learning strategy network, and acquire the corresponding ultrasound image frame sequence based on the updated acquisition pose. The ultrasound image update module is used to perform non-rigid registration of the ultrasound image frame sequence with the preoperative CT image input to a pre-constructed registration network to obtain a deformation field. The loss function of the registration network includes a regularization constraint term constructed based on the voxel-level continuous regularized weight map. The regularization constraint term is used to apply differentiated deformation constraints to different anatomical regions so that the deformation intensity of structurally sensitive regions is more restricted, while flexible tissue regions are allowed to have a larger degree of deformation freedom.

9. A storage medium, characterized in that, A program or instruction is stored on a storage medium, and the program or instruction is executed by a processor to implement the steps of the ultrasound image registration method as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the ultrasound image registration method according to any one of claims 1 to 7.

Images