Tumor segmentation method based on multi-modal image dynamic feature fusion

Abstract

The invention relates to a tumor segmentation method based on multi-modal image dynamic feature fusion, and relates to the technical field of medical image segmentation, and the method comprises the steps: obtaining multi-modal medical image data; and inputting the multi-modal medical image data into the constructed tumor boundary manifold model, and extracting multi-modal tumor boundary features in the multi-modal medical image data by using the dynamically adjusted convolution kernel parameters of the tumor boundary manifold model, the tumor boundary manifold model is used for generating a heat distribution probability graph according to the multi-modal medical image data and dynamically adjusting convolution kernel parameters according to the heat distribution probability graph; and performing nonlinear feature weight adjustment on the multi-modal tumor boundary features, and fusing the adjusted feature weights with the multi-modal tumor boundary features to generate multi-modal tumor boundary enhancement features, so as to generate a tumor segmentation result based on the multi-modal tumor boundary enhancement features. By applying the scheme of the invention, the accuracy and reliability of image segmentation can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of medical image segmentation, and in particular to a tumor segmentation method based on multimodal image dynamic feature fusion. Background Art

[0002] Early and accurate diagnosis and treatment of tumors are crucial for improving patient survival. Currently, medical imaging technologies such as computed tomography (CT) and magnetic resonance imaging (MRI) have become the mainstays of tumor diagnosis. These imaging techniques can provide crucial information about a tumor's location, size, morphology, and its relationship to surrounding tissues, providing strong support for clinicians in formulating treatment plans.

[0003] Image segmentation techniques based on a single modality have been widely used in clinical practice. For example, segmentation techniques based on CT images can provide high-resolution anatomical information, while segmentation techniques based on MRI images can provide detailed information about tumor tissue characteristics and function. However, due to the fuzzy boundaries and complex morphology of tumors, as well as the unique characteristics of different modal images, single-modality segmentation methods often fail to achieve ideal segmentation results, thereby reducing the accuracy and reliability of image segmentation. Summary of the Invention

[0004] In view of this, the present disclosure provides a tumor segmentation method based on the fusion of dynamic features of multimodal images. The main purpose is to solve the current technical problem that due to the blurred tumor boundaries, complex morphology, and the characteristics of different modal images, single-modality segmentation methods often find it difficult to obtain ideal segmentation effects, thereby reducing the accuracy and reliability of image segmentation.

[0005] According to a first aspect of the present disclosure, a tumor segmentation method based on multimodal image dynamic feature fusion is provided, the method comprising:

[0006] Acquire multimodal medical imaging data;

[0007] Inputting the multimodal medical imaging data into the constructed tumor boundary manifold model to extract multimodal tumor boundary features in the multimodal medical imaging data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model, wherein the tumor boundary manifold model is used to generate a heat distribution probability map based on the multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map;

[0008] Nonlinear feature weight adjustment is performed on the multimodal tumor boundary feature, and the adjusted feature weight is fused with the multimodal tumor boundary feature to generate a multimodal tumor boundary enhancement feature, so as to generate a tumor segmentation result based on the multimodal tumor boundary enhancement feature.

[0009] According to a second aspect of the present disclosure, a tumor segmentation device based on multimodal image dynamic feature fusion is provided, the device comprising:

[0010] An acquisition module, used to acquire multimodal medical imaging data;

[0011] an extraction module, configured to input the multimodal medical imaging data into a constructed tumor boundary manifold model, and extract multimodal tumor boundary features from the multimodal medical imaging data using convolution kernel parameters dynamically adjusted by the tumor boundary manifold model, wherein the tumor boundary manifold model is configured to generate a heat distribution probability map based on the multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map;

[0012] A generation module is used to perform nonlinear feature weight adjustment on the multimodal tumor boundary feature, and fuse the adjusted feature weight with the multimodal tumor boundary feature to generate a multimodal tumor boundary enhancement feature, so as to generate a tumor segmentation result based on the multimodal tumor boundary enhancement feature.

[0013] According to the third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of the aforementioned first aspect.

[0014] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method of the aforementioned first aspect.

[0015] The present disclosure provides a tumor segmentation method based on dynamic feature fusion of multimodal images. Compared with the prior art, the present disclosure obtains multimodal medical imaging data; inputs the multimodal medical imaging data into a constructed tumor boundary manifold model to extract multimodal tumor boundary features in the multimodal medical imaging data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model, wherein the tumor boundary manifold model is used to generate a heat distribution probability map based on the multimodal medical imaging data and dynamically adjust the convolution kernel parameters based on the heat distribution probability map; nonlinear feature weight adjustment is performed on the multimodal tumor boundary features, and the adjusted feature weights are fused with the multimodal tumor boundary features to generate multimodal tumor boundary enhancement features, so as to generate tumor segmentation results based on the multimodal tumor boundary enhancement features. By applying the scheme of the present disclosure, by obtaining multimodal medical imaging data and comprehensively utilizing the advantages of different modal images, the data of multiple modalities are fused to more comprehensively capture tumor features; the multimodal medical imaging data is input into the constructed tumor boundary manifold model. The model can generate a heat distribution probability map based on multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map. This dynamic adjustment mechanism can better adapt to the tumor boundary features in different imaging data, thereby improving the accuracy of boundary recognition; the present application scheme performs nonlinear feature weight adjustment on the multimodal tumor boundary features, and adopts the feature stability control mechanism of nonlinear dynamic system theory, which can dynamically allocate weights according to the importance and stability of the features, suppress the interference of noise features, and enhance the influence of effective features. The adjusted feature weights are then fused with the multimodal tumor boundary features to generate multimodal tumor boundary enhancement features. This fusion method comprehensively considers the importance and stability of different modal features, making the generated enhanced features more accurate and reliable, further improving the accuracy and reliability of image segmentation, and maintaining good segmentation effects even in low-quality images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0017] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a tumor segmentation method based on multimodal image dynamic feature fusion provided by an embodiment of the present disclosure;

[0019] Figure 2A schematic flow chart of another tumor segmentation method based on multimodal image dynamic feature fusion provided by an embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of the structure of a tumor boundary manifold model construction module provided by an embodiment of the present disclosure;

[0021] Figure 4 A schematic structural diagram of a heat nuclear diffusion map generation module provided in an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram of the structure of a dynamic convolution kernel adjustment module provided in an embodiment of the present disclosure;

[0023] Figure 6 A schematic structural diagram of a nonlinear weight control module provided in an embodiment of the present disclosure;

[0024] Figure 7 A schematic diagram of the structure of a multimodal feature fusion module provided in an embodiment of the present disclosure;

[0025] Figure 8 This is a structural block diagram of a renal tumor intelligent segmentation system based on multimodal image dynamic feature fusion provided by an embodiment of the present disclosure;

[0026] Figure 9 A schematic diagram of the structure of a tumor segmentation device based on multimodal image dynamic feature fusion provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other unless there is a conflict.

[0028] The following describes a tumor segmentation method based on multimodal image dynamic feature fusion according to an embodiment of the present disclosure with reference to the accompanying drawings.

[0029] In order to solve the technical problem that the current single-modality segmentation method often fails to achieve ideal segmentation results due to the fuzzy tumor boundaries, complex morphology, and the characteristics of different modal images, thereby reducing the accuracy and reliability of image segmentation, the embodiments of the present disclosure provide a tumor segmentation method based on multi-modal image dynamic feature fusion, such as Figure 1 As shown, the method includes:

[0030] Step 101: Acquire multimodal medical imaging data.

[0031] Among them, multimodal medical imaging data can be a collection of a series of medical imaging information obtained by imaging the human body from different angles and levels using a variety of different imaging principles, technical means and equipment.

[0032] According to the embodiment of the present disclosure, the multimodal medical imaging data may be a set of medical imaging information for patients with renal tumors.

[0033] Multimodal medical imaging data, including CT and MRI images, is acquired from patients with renal tumors. For CT images, enhanced CT is preferred, with scanning parameters set to: slice thickness 1-3 mm, tube voltage 120 kV, tube current 250-300 mA, and field of view (FOV) 35-40 cm. For MRI images, T1-weighted and T2-weighted sequences are preferred, with scanning parameters set to: slice thickness 3-5 mm, repetition time (TR) 500-700 ms (T1) / 3000-5000 ms (T2), echo time (TE) 10-20 ms (T1) / 80-120 ms (T2), and field of view (FOV) 35-40 cm.

[0034] After acquisition, the raw images (i.e., the acquired multimodal medical image data) can be preprocessed, including denoising, normalization, and registration. Denoising can be performed using an adaptive median filter, with the window size automatically adjusted between 3×3 and 7×7 based on the noise level. Normalization adjusts the Hounsfield unit range of CT images to [-100, 400] and normalizes the MRI signal intensity to the range [0, 1]. Spatial registration uses the mutual information maximization criterion, with registration accuracy controlled to within 1 mm.

[0035] For the embodiments of the present disclosure, Figure 2 As shown, the tumor segmentation method based on multimodal image dynamic feature fusion provided by the present disclosure mainly includes the following steps:

[0036] First, multimodal medical imaging data of renal tumor patients, including CT images and MRI images, are collected. Then, a tumor boundary manifold model is constructed based on these multimodal medical imaging data. Next, a heat kernel diffusion map is generated based on the tumor boundary manifold model. After that, the convolution kernel parameters are dynamically adjusted according to the heat distribution probability map. Subsequently, the dynamically adjusted convolution kernel is used to extract features from the multimodal medical imaging data. Then, the extracted features are subjected to nonlinear weight control. Based on the feature weights, the features of the multimodal medical imaging data are fused to obtain tumor boundary enhancement features. Finally, based on the tumor boundary enhancement features, the renal tumor segmentation result is generated.

[0037] Step 102: Input the multimodal medical imaging data into the constructed tumor boundary manifold model to extract multimodal tumor boundary features in the multimodal medical imaging data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model. The tumor boundary manifold model is used to generate a heat distribution probability map based on the multimodal medical imaging data and dynamically adjust the convolution kernel parameters based on the heat distribution probability map.

[0038] The tumor boundary manifold model is a model based on manifold learning theory and deep learning. Manifold learning aims to discover low-dimensional structures in high-dimensional data. In the context of tumor boundary recognition, this model treats multimodal medical imaging data as data points in a high-dimensional space and attempts to discover low-dimensional manifold structures related to tumor boundaries.

[0039] Multimodal tumor boundary features can be feature information related to tumor boundaries extracted from multimodal medical imaging data including CT images, MRI images, etc.

[0040] Heat distribution probability maps are a visualization tool that graphically displays the likelihood that each pixel or voxel in multimodal medical imaging data belongs to the tumor boundary. By assigning a heat value to each point in the image, information about the tumor boundary is presented in an intuitive manner.

[0041] For the embodiment of the present disclosure, before inputting the multimodal medical image data into the constructed tumor boundary manifold model, it is first necessary to construct the tumor boundary manifold model to realize the function of modeling the tumor boundary in the multimodal medical image as a topological manifold. The construction process of the tumor boundary manifold model is as follows: Figure 3 Specifically, it may include:

[0042] Step 301: extracting a tumor boundary point set from multimodal medical imaging data, and constructing a boundary topological relationship of the tumor boundary point set using a local neighborhood connection strategy;

[0043] In this step, first, a set of tumor boundary points can be extracted from the preprocessed multimodal medical imaging data. Boundary point extraction can utilize an improved gradient vector flow (GVF) algorithm, which incorporates an adaptive parameter adjustment mechanism to address the fuzzy nature of tumor boundaries. The GVF vector field is calculated as follows:

[0044]

[0045] Where E(v) is the energy function; v is the gradient vector field, which has the same dimension as the image space and is usually a three-dimensional vector field; is the image gradient, which represents the rate of change of the image intensity in three directions; μ is the regularization parameter, which is a scalar that is automatically adjusted between 0.05 and 0.2 according to the local gradient strength; Represents the square norm of the gradient vector field, which measures the smoothness of the vector field; represents the weighted deviation of the vector field from the original gradient, where the weight is the square norm of the original gradient; dxdydz is the spatial volume element. The threshold for boundary point extraction is set at 40% of the gradient amplitude. This threshold is the optimal value determined through extensive experimentation and effectively suppresses noise while preserving boundary details.

[0046] Next, the boundary topology relationship can be constructed based on the extracted tumor boundary point set. This disclosure adopts a local neighborhood connection strategy, where each boundary point can be connected to its K nearest neighbors, where the K value is dynamically adjusted according to the local curvature:

[0047] K i =K base +α·κ i

[0048] Among them, K i is the number of neighbors of the i-th point, which is a positive integer; K base is the basic number of neighbors, set to 8, indicating the minimum number of connections; α is the adjustment coefficient, set to 5, to control the degree of influence of curvature on the number of neighbors; κ i is the local curvature of the i-th point, normalized to a scalar value in the range [0, 1]. This dynamic adjustment strategy preserves more adjacency relationships in areas with large boundary curvature (such as sharp corners), thereby more accurately describing complex boundaries.

[0049] Step 302: performing weighted processing on the neighborhood connectivity relationships between the tumor boundary points in the tumor boundary point set according to the boundary topological relationship, and calculating the curvature tensor field of the weighted tumor boundary points;

[0050] Calculating the curvature tensor field of the tumor boundary point after weighted processing may specifically include:

[0051] Locally fitting the tumor boundary points to generate a quadratic surface of the tumor boundary points;

[0052] Based on the quadratic surface, the principal curvature and the direction of the tumor boundary point are calculated to form a curvature tensor field of the tumor boundary point based on the principal curvature and the direction.

[0053] In this step, the established neighborhood connections can be weighted, and the weight values ​​can reflect the local geometric similarity:

[0054]

[0055] Among them, W ijis the connection weight between points i and j, a scalar with a value range of (0,1); d ij is the Euclidean distance between two points, in millimeters; I i and I j are the image intensity values ​​of two points, the unit for CT images is Hounsfield unit, and the unit for MRI images is the normalized signal intensity; σ d and σ l are the standard deviation parameters of distance and intensity, which are set to 2 mm and 20 (CT) / 0.1 (MRI) respectively, to control the influence of distance and intensity differences on the weight; exp() is an exponential function to ensure that the weight is positive and its maximum value is 1.

[0056] Then, an orthogonal coordinate system is constructed on the boundary manifold, and the principal curvature and its direction are calculated based on the local fitting quadratic surface. For each boundary point, a quadratic surface is fitted in its local neighborhood:

[0057] z=ax 2 +by 2 +cxy+dx+ey+f

[0058] Where z, x, and y are the coordinate values ​​in the local coordinate system, in millimeters; a, b, c, d, e, and f are coefficient parameters determined from the coordinates of the local neighborhood points by the least squares method. Based on this surface, the principal curvatures κ1 and κ2 and the corresponding principal directions are calculated. and Forming the curvature tensor field:

[0059]

[0060] Where K is the curvature tensor, which is a 3×3 symmetric matrix; κ1 and κ2 are the principal curvatures, in mm-1, indicating the curvature of the surface in the principal direction; and is the principal direction, which is a unit vector indicating the direction of maximum and minimum curvature; the superscript T indicates the transpose of the vector; Represents the outer product of vectors, resulting in a 3×3 matrix. This curvature tensor field describes the local geometric characteristics of the boundary and provides an important basis for subsequent heat kernel diffusion and convolution kernel adjustment.

[0061] In practical applications, the neighborhood radius for curvature calculation can be adjusted appropriately for different morphologies of renal tumors. For tumors with smooth boundaries, the neighborhood radius can be set to 3-5 mm; for tumors with irregular boundaries, the neighborhood radius can be set to 1-3 mm to better capture local details.

[0062] Step 303: construct a tumor boundary manifold model based on the boundary topological relationship, the weighted neighborhood connectivity relationship, and the curvature tensor field.

[0063] For the embodiments of the present disclosure, after the tumor boundary manifold model is constructed, the multimodal medical imaging data can be input into the constructed tumor boundary manifold model to extract the multimodal tumor boundary features in the multimodal medical imaging data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model.

[0064] Among them, Figure 4 As shown in Figure 1, based on the tumor boundary manifold model, a multi-scale representation describing the boundary structure is generated by simulating the heat conduction process. Accordingly, the tumor boundary manifold model can be used to generate a heat distribution probability map based on multimodal medical imaging data, which may include:

[0065] Step 401: Based on the tumor boundary manifold model, a discrete differential operator is constructed. The local curvature of each tumor boundary point is calculated based on the discrete differential operator. The thermal conductivity coefficient of each tumor boundary point is set according to the local curvature so that the thermal conductivity coefficient of the high curvature area is lower than the thermal conductivity coefficient of the low curvature area.

[0066] The discrete differential operator may be a discrete Laplace-Beltrami operator.

[0067] In this step, first, a discrete Laplace-Beltrami operator can be constructed based on the tumor boundary manifold model. Unlike traditional uniform heat conduction, this disclosure uses a non-uniform heat conduction model, where the heat conduction coefficient is inversely proportional to the local curvature. The discrete Laplace-Beltrami operator is constructed as follows:

[0068]

[0069] Among them, L ij is the i-th row and j-th column element of the Laplace matrix, which is a dimensionless scalar; W ij is the aforementioned connection weight; D i =∑ j W ij is the degree of vertex i, which represents the sum of the weights of all edges connected to vertex i; is the normalization factor; “connected” indicates that there is a connection between point i and point j; the summation symbol ∑ k≠i It means to accumulate all the points k connected to i except i.

[0070] In order to achieve non-uniform heat conduction, the curvature adjustment factor is introduced:

[0071]

[0072] in, is the adjusted Laplace matrix element; C ij is the curvature adjustment factor:

[0073]

[0074] Among them, β is the control parameter, which is set to 10 and dimensionless; κ i and κ j where exp() is the exponential function. This results in a lower thermal conductivity in high-curvature regions (such as sharp corners at the edge of a tumor), preserving boundary details; and a higher thermal conductivity in low-curvature regions (such as the smooth transition zone at the edge of a tumor), smoothing out noise.

[0075] Step 402: Set multiple heat diffusion time steps, and use discrete differential operators, local curvature, and heat conductivity coefficient to calculate the heat kernel function of the multiple heat diffusion time steps to obtain the heat distribution under different time steps. The heat diffusion time step includes a short time step and a long time step. The short time step is used to capture local detail features, and the long time step is used to capture global structural features.

[0076] In this step, multiple heat diffusion time steps can be set to form a multi-scale representation. The heat kernel function is calculated as follows:

[0077]

[0078] Among them, H t (i, j) is the value of the heat kernel function between points i and j at time t, dimensionless; λ k is the Laplace matrix The kth eigenvalue of , dimensionless; φ k is the corresponding kth eigenvector, whose dimension is equal to the total number of boundary points N; k (i) represents the i-th component of the k-th eigenvector; t is the thermal diffusion time parameter, dimensionless; is the exponential decay term; ∑ k =1 N The sum of all N eigenvalues ​​and eigenvectors is calculated. The choice of time step t significantly affects the scale characteristics of the heat kernel function: smaller t values ​​(such as 0.01-0.1) capture local detail features, while larger t values ​​(such as 1.0-10.0) capture global structure features. In the kidney tumor segmentation scenario, three time steps are preferably set: t1 = 0.05 (detail), t2 = 0.5 (medium scale), and t3 = 5.0 (global structure).

[0079] Step 403: Regularize the heat distribution into a probability distribution to generate a heat distribution probability map, wherein the high probability area in the heat distribution probability map corresponds to the key part of the tumor boundary, and the low probability area corresponds to the smooth transition area of ​​the tumor boundary.

[0080] In this step, the heat distribution can be normalized into a probability distribution to generate a heat distribution probability map. For each boundary point i, its heat probability is defined as:

[0081]

[0082] Among them, P(i) is the heat probability of point i, a scalar with a value range of [0,1]; H t (i,i) is the self-heating kernel function value of point i at time t, ∑ j H t (j, j) is the sum of the self-heating kernel function values ​​of all points, which is the normalization factor; the summation symbol ∑ j =(\frac{i}{i}) represents the cumulative probability of all boundary points. This probability reflects the importance of point i in the boundary structure. High-probability regions correspond to key boundary locations (such as sharp corners and areas of high curvature), while low-probability regions correspond to smooth transition areas. In practice, the probability threshold is set to the top 20% of the sorted points. These points are considered key points of the boundary and serve as anchor points for subsequent convolution kernel adjustments.

[0083] For the embodiment of the present disclosure, after generating the heat distribution probability map, as shown in FIG. Figure 5 As shown in the figure, the tumor boundary manifold model can be used to dynamically adjust the convolution kernel parameters based on the heat distribution probability map to achieve adaptive changes in the convolution kernel shape, allowing the convolution operation to accurately adapt to the local geometric characteristics of the tumor boundary. This topology-aware dynamic convolution kernel adjustment mechanism can significantly improve the accuracy of tumor boundary recognition, improving boundary clarity by 40% to 60% compared to traditional methods. Specifically, it may include:

[0084] Step 501: construct a deformation field based on the curvature tensor field and the heat distribution probability map, wherein the deformation field satisfies the local homeomorphism mapping condition and keeps the local topological structure unchanged;

[0085] In this step, the deformation field is calculated as follows:

[0086]

[0087] Where D(x) is the deformation field at position x, which is a 3×3 matrix; Ω is the set of boundary key points, which contains points with heat probability higher than the threshold; P(i) is the heat probability of point i, which ranges from [0,1]; K i is the curvature tensor at point i, which is a 3×3 matrix; x and x i are the spatial position and the position coordinates of point i, respectively, in millimeters; G(xx i ,σ i ) is centered at point i, σ i is the Gaussian function of the standard deviation, with a value range of (0,1); σ iIt is inversely proportional to the local curvature of point i and is calculated as σ i =σ0 / (1+γ·|κ i |), where σ0 is set to 3 mm, γ is set to 5, |κ i | is the absolute value of the average principal curvature at point i; ∑ i∈Ω Indicates the accumulation of all boundary key points. This deformation field satisfies the local homeomorphism mapping condition and keeps the local topology unchanged, which is the basis for topology-aware convolution.

[0088] Step 502: Adjust the shape of the convolution kernel based on the deformation field so that the adjusted convolution kernel has a greater ability to capture the deformed multimodal tumor boundary features than a preset ability, wherein the shape adjustment strategy includes scaling transformation, rotation transformation, and non-rigid deformation.

[0089] In this step, the shape of the standard convolution kernel is adjusted based on the deformation field. In traditional convolutional neural networks, the convolution kernel is usually of a fixed shape (such as a 3×3 or 5×5 square). This disclosure introduces three adjustment strategies: scaling transformation, rotation transformation, and non-rigid deformation.

[0090] The scaling transformation adjusts the receptive field size of the convolution kernel according to the curvature:

[0091] s(x)=s0·(1-δ·|κ(x)|)

[0092] Here, s(x) is the dimensionless kernel size coefficient at position x, typically ranging from 0.4 to 1.6; s0 is the base size (set to 1.0), representing the standard kernel size; δ is a tuning parameter (set to 0.6) that controls the effect of curvature on kernel size; and |κ(x)| is the average absolute value of the curvature at position x, a scalar normalized to the range [0, 1]. This results in smaller kernel sizes in areas with large curvature (such as sharp corners at the edge of a tumor), improving local accuracy; larger kernel sizes are used in areas with less curvature, increasing the receptive field.

[0093] The rotation transformation aligns the convolution kernel's main axis with the main direction of curvature:

[0094]

[0095] Where R(x) is the rotation matrix at position x, with a dimension of 3×3; and is the main direction of curvature at x, which is a normalized three-dimensional unit vector. This rotation ensures that the convolution kernel can extract features along the main direction of the boundary and enhance directional sensitivity.

[0096] Non-rigid deformation adjusts the internal structure of the convolution kernel according to the deformation field:

[0097] T(p)=p+η·D(p)

[0098] Where T(p) is the deformed position of point p inside the convolution kernel, in millimeters; p is the original position, in millimeters; η is the deformation strength parameter (set to 0.5), which controls the degree of deformation; and D(p) is the deformation field at position p. This nonrigid deformation enables the convolution kernel to adapt to complex boundary shapes, and is particularly effective for irregular tumor boundaries.

[0099] Step 503: Perform a position-adaptive convolution operation on the multimodal medical imaging data using the adjusted convolution kernel to dynamically adjust the convolution kernel parameters, wherein the shape of the convolution kernel changes with position, the convolution operation adopts a dynamic sampling point strategy, and the sampling point position is determined by the deformation field.

[0100] In this step, the position-adaptive convolution operation is defined:

[0101]

[0102] Among them, F out (x) is the value of the output feature map at position x; F in (x) is the value of the input feature map at position x; N(x) is the convolution kernel neighborhood at position x, the size of which is determined by the aforementioned scaling transformation; w(p) is the convolution kernel weight, which is learned during the training process; p is the relative position within the convolution kernel; T(x+p) is the sampling point position, which is determined by the aforementioned non-rigid deformation; ∑ p∈N(x) Indicates the accumulation of all points in the convolution kernel neighborhood. Since the sampling points may not be on the integer grid, bilinear interpolation is required to obtain the corresponding eigenvalues.

[0103] In practical applications, to improve computational efficiency, convolution kernels for common deformation patterns can be pre-calculated, a lookup table can be created, and interpolation can be used for complex deformations. Furthermore, dynamic convolution kernels can be used for boundary regions, while standard convolution kernels can be used for regions far from the boundary. This strategy significantly improves computational efficiency while maintaining segmentation accuracy.

[0104] Step 103 : performing nonlinear feature weight adjustment on the multimodal tumor boundary feature, and fusing the adjusted feature weight with the multimodal tumor boundary feature to generate a multimodal tumor boundary enhancement feature, so as to generate a tumor segmentation result based on the multimodal tumor boundary enhancement feature.

[0105] In the disclosed embodiments, nonlinear weight control enables stability assessment and dynamic weight adjustment of extracted features. This innovative approach incorporates nonlinear dynamical system theory, treating feature evolution as trajectories in phase space and optimizing feature selection through stability analysis.

[0106] like Figure 6As shown in the figure, after extracting the multimodal tumor boundary features, the multimodal tumor boundary features can be subjected to nonlinear feature weight adjustment. Specifically, the feature stability control mechanism of the nonlinear dynamic system theory can be adopted to significantly enhance the anti-noise capability. In addition, good segmentation effect can be maintained even in low-quality images, and the anti-noise performance is improved by about 35%. Specifically, it may include:

[0107] Step 601: Mapping multimodal tumor boundary features into a feature evolution phase space, and identifying key points of the multimodal tumor boundary features in the feature evolution phase space. The key points include at least stable points, bifurcation points, and attractors, corresponding to stable features, feature changes, and convergence states, respectively.

[0108] In this step, the feature evolution phase space is constructed. The change of features with the depth of the convolution layer is regarded as a trajectory in the phase space:

[0109] X l =Φ(F l )

[0110] Among them, X l is the coordinate of the l-th layer feature in the phase space, and the dimension is usually a vector of 3-5; F l is the original feature at layer l, which may have a high dimensionality (e.g., hundreds or more); Φ is a dimensionality reduction mapping (e.g., principal component analysis (PCA)), which projects high-dimensional features into a low-dimensional space; l is the network layer index, ranging from 1 to L, where L is the total number of network layers. In kidney tumor segmentation applications, high-dimensional features are typically reduced to 3-5 dimensions, retaining at least 85% of the feature variance.

[0111] Identify key points in phase space: stable points, bifurcation points, and attractors. Among them, stable points satisfy:

[0112] ||X l+1 -X l ||<∈

[0113] Among them, ||X l+1 -X l || is the Euclidean distance between features of adjacent layers in the phase space, indicating the magnitude of feature change; ∈ is the threshold, set to 0.01, indicating the standard for stability judgment. The bifurcation point occurs when a significant change in the feature trajectory occurs at a certain layer:

[0114]

[0115] in, is the ratio of the feature change rates of two adjacent layers, representing the acceleration of feature change; τ is the threshold, set to 3.0, representing the criterion for determining bifurcation. An attractor is the state where the feature trajectory ultimately converges, typically corresponding to a stable boundary feature representation.

[0116] Step 602: Perform Lyapunov exponent calculation on the feature trajectory of the key point to obtain an exponent calculation result, and use the exponent calculation result to evaluate the feature stability, wherein the exponent calculation result includes a positive exponent and a negative exponent, a positive exponent corresponds to an unstable feature, and a negative exponent corresponds to a stable feature;

[0117] In this step, the Lyapunov exponent of the feature trajectory is calculated to evaluate the stability of the feature:

[0118]

[0119] Among them, λ is the Lyapunov exponent, dimensionless, indicating the stability of the system; is the Jacobian matrix of the features of adjacent layers, which represents the local linear approximation of feature changes; ln is the natural logarithm; lim n→∞ represents the limit when the number of layers tends to infinity; ∑ l =1 n Indicates accumulation of all layers; is an average operation, which makes the exponent independent of the number of layers. In actual calculations, a finite time approximation is used:

[0120]

[0121] Where L is the number of network layers; The ratio of the feature's rate of change is used to approximate the eigenvalues ​​of the Jacobian matrix. A negative Lyapunov exponent (λ < 0) indicates a stable feature, while a positive Lyapunov exponent (λ > 0) indicates an unstable feature. In renal tumor segmentation, stable features typically correspond to true boundaries, while unstable features may arise from noise or spurious boundaries.

[0122] Step 603: construct a feature weight mapping based on the feature stability evaluation result, wherein the stability evaluation result includes stable features and unstable features, with stable features corresponding to high weights and unstable features corresponding to low weights;

[0123] In this step, based on the above stability evaluation results, a feature weight map is constructed:

[0124]

[0125] Where w(F) is the feature weight, ranging from (0, 1], indicating the importance of the feature; ρ is a tuning parameter, set to 2.0, which controls the rate of weight decay; and exp() is an exponential function. This weight mapping ensures that stable features receive high weights (close to 1) and unstable features receive low weights (close to 0).

[0126] Step 604: Utilize the chaos control feedback mechanism to perform nonlinear feature weight adjustment on the multimodal tumor boundary features according to feature weight mapping.

[0127] In this step, feedback control strategies can be designed by monitoring the chaotic behavior during the feature evolution process. For the detected chaotic attractor, the OGY (Ott-Grebogi-Yorke) control method is applied to guide the system to a stable periodic orbit:

[0128] u l =-K·(X l -X * )

[0129] Among them, u l is the control signal of the first layer, and its dimension is the same as that of the phase space; K is the gain matrix, which is determined by analyzing the local dynamics of the system; X * The target stable point is typically chosen as a fixed point of the system or a point on a stable periodic orbit; the minus sign indicates the negative feedback nature of the feedback control. The control signal influences the convolution kernel parameters and feature weights, achieving adaptive adjustment. In the context of renal tumor segmentation, this chaos control mechanism significantly enhances the robustness of the system, significantly reducing the sensitivity of the segmentation results to image quality and initial parameter settings.

[0130] For the embodiments of the present disclosure, Figure 7 As shown in the figure, based on information theory and statistical manifold theory, a dynamic multimodal fusion strategy is designed to achieve adaptive fusion of CT and MRI image features, effectively improving the utilization efficiency of complementary information. Specifically, the adjusted feature weights can be fused with multimodal tumor boundary features to generate multimodal tumor boundary enhancement features, so as to generate tumor segmentation results based on the multimodal tumor boundary enhancement features. The multimodal tumor boundary features may include CT modality tumor boundary features and MRI modality tumor boundary features, and specifically may include:

[0131] Step 701: Calculate the information entropy of the tumor boundary region based on the CT modality tumor boundary features and the MRI modality tumor boundary features, respectively, wherein a high information entropy indicates rich information, and a low information entropy indicates redundant or insufficient information;

[0132] The information entropy of the tumor boundary region under the CT modality tumor boundary features and the MRI modality tumor boundary features is calculated respectively, which may specifically include:

[0133] Divide the tumor boundary area into multiple local blocks, and dynamically adjust the sizes of the multiple local blocks according to the local complexity;

[0134] Calculate the information entropy of each local block and perform unbiased estimation of the information entropy;

[0135] The information entropy of each local block processed by the unbiased estimation technology is processed by the entropy smoothing technology.

[0136] In this step, the tumor area is divided into local blocks. The size of each block is dynamically adjusted according to the local complexity, ranging from 5×5 to 15×15 pixels. The information entropy is calculated for each local block:

[0137]

[0138] Among them, H(X) is information entropy, the unit is bit, which represents the uncertainty of information; p(x i ) is the gray value x i The probability of is estimated by statistical histogram; log is the logarithm with base 2; ∑ i Indicates the accumulation of all possible grayscale values. To improve the estimation accuracy, an unbiased estimation technique is used:

[0139]

[0140] Among them, H unbias (X) is the unbiased estimated information entropy in bits; H(X) is the original information entropy estimate; b is the number of discretizations of grayscale values ​​(usually 256); n is the number of samples, that is, the number of pixels in the small block; is the deviation correction term. At the same time, entropy smoothing technology is applied to reduce the influence of noise:

[0141]

[0142] Among them, H smooth (X) is the smoothed information entropy, in bits; H unbias (X) is the unbiased estimated information entropy; is the average entropy of the neighborhood blocks; α is a smoothing coefficient, set to 0.3, which controls the influence of the neighborhood average; (1-α) and α are the weights of the current block and the neighborhood, respectively. In this way, a block-by-block information entropy map is generated, reflecting the information richness of different regions.

[0143] The multimodal feature adaptive fusion strategy disclosed in this paper based on information entropy can make full use of the complementary information of CT and MRI images, improve the efficiency of multimodal information utilization by about 50%, and optimize the allocation of computing resources through regional adaptive computing strategy, reducing computing resource usage by about 45% while maintaining high-precision segmentation results.

[0144] Step 702: construct an information entropy distribution map based on the information entropy, and use the information entropy distribution map to identify information contribution areas of CT modality tumor boundary features and MRI modality tumor boundary features;

[0145] The information entropy distribution map is constructed based on the information entropy, which may specifically include:

[0146] The information entropy of each smoothed local block is mapped to the tumor boundary area to generate an information entropy distribution map.

[0147] In this step, the information contribution areas of each modality are identified. Generally speaking, CT images have high information entropy in bone and calcified areas, while MRI images have high information entropy in soft tissue areas. For renal tumors, MRI generally provides richer information within the tumor and at the boundary transition zone, while CT provides clearer boundary information at the interface between the tumor and surrounding tissue.

[0148] Step 703: Calculate the conditional mutual information between the CT modality tumor boundary features and the MRI modality tumor boundary features based on the information contribution region, so as to evaluate the complementarity between the CT modality tumor boundary features and the MRI modality tumor boundary features using the conditional mutual information;

[0149] In this step, the conditional mutual information between the CT modality and the MRI modality can be calculated to evaluate the complementarity:

[0150] I(X;Y|Z)=H(X,Z)+H(Y,Z)-H(X,Y,Z)-H(Z)

[0151] Where I(X;Y|Z) is the conditional mutual information (I / Z) between CT feature X and MRI feature Y at a given spatial location Z, expressed in bits; H(X,Z) is the joint entropy of the CT features and the spatial location; H(Y,Z) is the joint entropy of the MRI features and the spatial location; H(X,Y,Z) is the joint entropy of the CT features, MRI features, and the spatial location; and H(Z) is the entropy of the spatial location. High mutual information indicates that the two modalities provide complementary information, while low mutual information indicates redundant or independent information.

[0152] The joint distribution is estimated using the kernel density estimation method:

[0153]

[0154] Where p(x,y) is the joint probability density function; n is the number of samples; K h is the kernel function, usually the Gaussian kernel is selected; h is the bandwidth parameter, which controls the degree of smoothing; (xx i ,yy i ) is the distance from the sample point to the estimated point; Indicates the accumulation of all sample points; The bandwidth parameter h is dynamically adjusted according to the number of samples, using the empirical formula h=1.06·σ·n -1 / 5 , where σ is the sample standard deviation.

[0155] Step 704: Based on the modal fusion rule of maximizing conditional mutual information, the two modal tumor boundary features are evenly fused in the region where the complementarity is higher than a preset complementarity threshold, and / or, when the information entropy of the tumor boundary feature of a single modality is higher than a preset information entropy threshold, the single modality tumor boundary feature with the information entropy higher than the preset information entropy threshold is preferentially fused, so as to perform feature fusion on the CT modality tumor boundary features and the MRI modality tumor boundary features to generate a multimodal tumor boundary enhancement feature;

[0156] In this step, based on the above evaluation, a modal fusion rule is designed to maximize the conditional mutual information:

[0157] F fused (x) = ω CT (x)·F CT (x)+ω MRI (x)·F MRI (x)

[0158] Among them, F fused (x) is the fusion feature at position x; F CT (x) and F MRI (x) are the CT features and MRI features at position x, respectively, with the same feature dimension; ω CT (x) and ω MRI (x) is the position-related weight coefficient, a scalar in the range [0,1], satisfying ω CT (x)+ω MRI (x)=1.

[0159] The weight coefficient is calculated as follows:

[0160]

[0161] ω MRI (x)=1-ω CT (x)

[0162] Among them, H CT (x) and H MRI (x) are the information entropy of CT and MRI at position x, in bits; I(X;Y|x) is the conditional mutual information at position x, in bits; β is the complementarity adjustment parameter, set to 0.5, to control the influence of mutual information on weights; is the basic weight distribution based on information entropy; (1+β·I(X; Y|x)) is the complementary adjustment term.

[0163] This weight calculation method ensures that the two modal features are balanced in the area with strong complementarity, and is biased towards the mode with high information entropy in the area with single modality advantage.

[0164] Step 705: Perform optimal transmission on the statistical manifold for the multimodal tumor boundary enhancement features to generate a tumor segmentation result.

[0165] In this step, the multimodal features are regarded as probability distributions on a statistical manifold, and the optimal transmission problem is constructed:

[0166]

[0167] Where W2 is the 2-Wasserstein distance, and its unit depends on the unit of the feature space; p1 and p2 are the probability distributions of different modal features; Γ(p1, p2) is the set of joint distributions that satisfy the marginal distribution constraints; inf represents the infimum, that is, the minimum value among all possible joint distributions; ||xy|| 2 is the squared Euclidean distance in the feature space; ∫dγ(x,y) represents the integration of the joint distribution γ.

[0168] By solving the optimal transmission problem, geometrically consistent fusion of multimodal features is achieved:

[0169] F opt =T(F CT ,F MRI )

[0170] Among them, F opt is the optimal fusion feature; T is the optimal transmission mapping, which maps the CT feature space to the MRI feature space; F CT and F MRi are CT features and MRI features, respectively. In the actual calculation, the Sinkhorn algorithm is used to solve the discrete optimal transmission problem, with the number of iterations set to 50 and the regularization parameter set to 0.01.

[0171] Through the above-mentioned multimodal fusion strategy, the present disclosure fully utilizes the complementary information of CT and MRI to improve the accuracy of tumor boundary recognition, especially the segmentation effect at the junction of tumor and normal tissue is significantly improved.

[0172] In the embodiment of the present disclosure, before generating a tumor segmentation result based on the multimodal tumor boundary enhancement feature, the present disclosure further includes boundary refinement and verification steps to further improve segmentation accuracy and reliability, which may specifically include:

[0173] Step 801: performing boundary refinement processing on the tumor boundary enhancement feature, wherein the boundary refinement processing at least includes accurately locating the boundary position, optimizing the boundary connectivity, and smoothing the boundary;

[0174] In this step, the boundary position is accurately located, and a sub-pixel boundary extraction algorithm is used, with a positioning accuracy of 0.1 pixel. The boundary connectivity is optimized and possible boundary breaks are repaired. The boundary is smoothed to remove unnecessary noise and burrs while maintaining key morphological features. The boundary smoothing uses the adaptive level set method:

[0175]

[0176] Where φ is the level set function, which represents the implicit boundary; is the rate of change of the level set function over time; t is the evolution time parameter; g is the boundary stop function, which ranges from [0,1], close to 0 at the boundary and close to 1 at non-boundary areas; κ is the curvature, in mm^(-1); is the modulus of the gradient of the level set function, which represents the normal rate of change of the isosurface; is the gradient of the stopping function; is the gradient of the level set function; α is a balancing parameter, set to 0.5, controlling the relative importance of the two terms. Adaptiveness is reflected in the curvature weight being inversely proportional to the local boundary clarity, ensuring that the original shape is preserved in areas with clear boundaries and that appropriate smoothing is applied in blurred areas.

[0177] Step 802: Verify the tumor boundary enhancement features after boundary refinement, where the verification process at least includes checking topological consistency, morphological verification, and evaluating the confidence of the segmentation result.

[0178] Generate tumor segmentation results based on multimodal tumor boundary enhancement features, which may include:

[0179] Generate tumor segmentation results based on the multimodal tumor boundary enhancement features after verification processing.

[0180] In this step, the results of boundary refinement are verified, which may include: checking topological consistency to ensure that the segmentation results maintain the expected topological structure (for example, renal tumors are usually single-connected areas); performing morphological verification to check whether the segmentation results conform to the general morphological characteristics of renal tumors; and evaluating the confidence of the segmentation results to identify possible error areas and provide uncertainty estimates. Confidence assessment uses a probabilistic boundary map:

[0181]

[0182] Among them, P boundary (x) is the boundary probability at position x, with a value range of [0,1]; E(x) is the boundary energy function, which represents the cost of position x becoming the boundary; T is the temperature parameter, set to 0.1, which controls the smoothness of the probability distribution; exp() is the exponential function; ∑ yIt represents the accumulation of all possible boundary positions y; the denominator is a normalization factor to ensure that the sum of all probabilities is 1. High-confidence areas (such as probability greater than 0.85) are directly accepted as segmentation results, low-confidence areas (such as probability less than 0.65) are marked as potential error areas, and intermediate areas are tested twice.

[0183] To meet the needs of real-time clinical diagnosis, this disclosure also includes an asynchronous parallel reasoning process based on edge computing, which achieves efficient distributed processing and enables the system to adapt to the needs of real-time clinical diagnosis. This process optimizes computing resource allocation and improves system response speed. Specifically, it may include:

[0184] Step 901: Assign the tumor segmentation task to multiple edge computing devices, where each edge computing device is used to process the multimodal medical imaging data assigned according to the tumor segmentation task and perform parallel reasoning. Each edge computing device adopts a multi-threaded asynchronous mechanism to asynchronously obtain data blocks from the control device. Each edge computing device saves the intermediate reasoning results in the edge device storage. After completing part of the reasoning task, it returns part of the reasoning results to the central server. The central server is used to integrate the reasoning results of all edge devices to generate the final tumor segmentation result.

[0185] Specifically, computing tasks are assigned to edge computing devices. Based on task complexity and device capabilities, computing tasks are dynamically allocated to ensure load balancing. Highly complex tasks (such as dynamic convolution kernel generation) are assigned to devices with greater computing power; highly parallel tasks (such as multi-scale feature extraction) are distributed across multiple devices for parallel processing.

[0186] Multimodal image inference is implemented in parallel on edge computing devices. Each device processes a portion of the data block, typically 64×64×64 voxels, with a 25% overlap between adjacent blocks to ensure continuity of border processing.

[0187] A multi-threaded asynchronous mechanism is used to retrieve data blocks from the control device and perform accelerated inference. The number of threads is dynamically adjusted based on the device's computing power, typically set to 1.5 times the number of CPU cores to maximize computing resource utilization. A data prefetching strategy is used to prefetch the next data block while processing the current block, reducing I / O wait time.

[0188] The intermediate inference results are stored in edge device storage for subsequent processing. These include feature maps, convolution kernel parameters, and weight maps. These results are stored in a compressed format, achieving a compression rate of up to 60% while maintaining data integrity.

[0189] Partial inference results are returned to the central server for reference by other edge devices. This collaborative inference strategy is particularly suitable for handling large or multiple tumors, improving overall inference efficiency and consistency. The returned results include boundary key points, curvature tensor fields, and heat distribution probability maps. Data transmission uses an incremental update strategy, transmitting only the parts that have significantly changed.

[0190] Finally, the inference results from all edge devices are integrated to generate the final segmentation result. This integration process uses a weighted voting mechanism, with weights related to the device's computing power and past inference accuracy. For areas with inconsistent results, the aforementioned confidence assessment strategy is applied to make decisions, ensuring the consistency and accuracy of the integrated results.

[0191] like Figure 8 As shown, the tumor intelligent segmentation system based on dynamic feature fusion of multimodal images disclosed in the present invention includes the following modules: an image acquisition module 10, used to acquire multimodal medical image data of patients with renal tumors; a boundary manifold modeling module 20, used to construct a tumor boundary manifold model; a heat kernel mapping module 30, used to generate a heat kernel diffusion map; a dynamic convolution kernel adjustment module 40, used to dynamically adjust the convolution kernel parameters based on the heat distribution probability map; a feature extraction module 50, used to extract features from multimodal medical image data based on the dynamically adjusted convolution kernel parameters; a nonlinear control module 60, used to perform nonlinear weight control on the extracted features; a feature fusion module 70, used to fuse the features of multimodal medical image data based on feature weights; a boundary refinement module 80, used to perform boundary refinement processing on tumor boundary enhancement features; a verification module 90, used to verify the results after boundary refinement processing; a segmentation output module 100, used to generate renal tumor segmentation results; and an edge computing module 110, used to implement an asynchronous parallel reasoning process.

[0192] The data flow relationship between each module is as follows: the CT and MRI images collected by the image acquisition module 10 are input into the boundary manifold modeling module 20 after preprocessing; the tumor boundary manifold model constructed by the boundary manifold modeling module 20 is input into the heat kernel mapping module 30; the heat kernel diffusion map and heat distribution probability map generated by the heat kernel mapping module 30 are input into the dynamic convolution kernel adjustment module 40; the dynamically adjusted convolution kernel parameters generated by the dynamic convolution kernel adjustment module 40 are input into the feature extraction module 50; the features extracted by the feature extraction module 50 are input into the nonlinear control module 60; the feature weights adjusted by the nonlinear control module 60 are input into the feature fusion module 70; the multimodal features fused by the feature fusion module 70 are input into the boundary refinement module 80; the results processed by the boundary refinement module 80 are input into the verification module 90; the results verified by the verification module 90 are input into the segmentation output module 100; the edge computing module 110 interacts with the above modules to realize distributed asynchronous computing.

[0193] In summary, according to the tumor segmentation method based on multimodal image dynamic feature fusion provided by the present disclosure, compared with the existing technology, the present disclosure obtains multimodal medical imaging data; inputs the multimodal medical imaging data into a constructed tumor boundary manifold model to extract multimodal tumor boundary features in the multimodal medical imaging data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model, wherein the tumor boundary manifold model is used to generate a heat distribution probability map based on the multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map; nonlinear feature weight adjustment is performed on the multimodal tumor boundary features, and the adjusted feature weights are fused with the multimodal tumor boundary features to generate multimodal tumor boundary enhancement features, so as to generate tumor segmentation results based on the multimodal tumor boundary enhancement features. By applying the scheme of the present disclosure, by obtaining multimodal medical imaging data and comprehensively utilizing the advantages of different modal images, the data of multiple modalities are fused to more comprehensively capture tumor features; the multimodal medical imaging data is input into the constructed tumor boundary manifold model. The model can generate a heat distribution probability map based on multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map. This dynamic adjustment mechanism can better adapt to the tumor boundary features in different imaging data, thereby improving the accuracy of boundary recognition; the present application scheme performs nonlinear feature weight adjustment on the multimodal tumor boundary features, and adopts the feature stability control mechanism of nonlinear dynamic system theory, which can dynamically allocate weights according to the importance and stability of the features, suppress the interference of noise features, and enhance the influence of effective features. The adjusted feature weights are then fused with the multimodal tumor boundary features to generate multimodal tumor boundary enhancement features. This fusion method comprehensively considers the importance and stability of different modal features, making the generated enhanced features more accurate and reliable, further improving the accuracy and reliability of image segmentation, and maintaining good segmentation effects even in low-quality images.

[0194] Based on the above Figure 1 The specific implementation of the method shown in the embodiment provides a tumor segmentation device based on multimodal image dynamic feature fusion, such as Figure 9 As shown, the device includes: an acquisition module 31, an extraction module 32, and a generation module 33;

[0195] An acquisition module 31 is used to acquire multimodal medical imaging data;

[0196] an extraction module 32, configured to input the multimodal medical image data into the constructed tumor boundary manifold model, and extract multimodal tumor boundary features from the multimodal medical image data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model, wherein the tumor boundary manifold model is configured to generate a heat distribution probability map based on the multimodal medical image data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map;

[0197] The generation module 33 is used to perform nonlinear feature weight adjustment on the multimodal tumor boundary feature, and fuse the adjusted feature weight with the multimodal tumor boundary feature to generate a multimodal tumor boundary enhancement feature, so as to generate a tumor segmentation result based on the multimodal tumor boundary enhancement feature.

[0198] In specific application scenarios, such as Figure 9 As shown, the device further includes: a building module 34;

[0199] The construction module 34 is used to construct a tumor boundary manifold model.

[0200] In a specific application scenario, the construction module 34 may be used to extract a set of tumor boundary points from the multimodal medical imaging data, and construct a boundary topological relationship of the set of tumor boundary points through a local neighborhood connection strategy;

[0201] performing weighted processing on the neighborhood connectivity relationships between the tumor boundary points in the tumor boundary point set according to the boundary topological relationship, and calculating the curvature tensor field of the tumor boundary points after the weighted processing;

[0202] The tumor boundary manifold model is constructed based on the boundary topological relationship, the weighted neighborhood connectivity relationship, and the curvature tensor field.

[0203] In a specific application scenario, the construction module 34 can be used to locally fit the tumor boundary points to generate a quadratic surface of the tumor boundary points;

[0204] Based on the quadratic surface, the principal curvature and the direction of the tumor boundary point are calculated to form a curvature tensor field of the tumor boundary point based on the principal curvature and the direction.

[0205] In a specific application scenario, the extraction module 32 may be used to construct a discrete differential operator based on the tumor boundary manifold model;

[0206] calculating a local curvature of each of the tumor boundary points based on the discrete differential operator, and setting a thermal conductivity coefficient of each of the tumor boundary points according to the local curvature so that the thermal conductivity coefficient of a high curvature region is lower than the thermal conductivity coefficient of a low curvature region;

[0207] Setting a plurality of heat diffusion time steps, wherein the heat diffusion time steps include a short time step and a long time step, the short time step is used to capture local detail features, and the long time step is used to capture global structural features;

[0208] Calculating the heat kernel function of the plurality of heat diffusion time steps using the discrete differential operator, the local curvature, and the heat transfer coefficient to obtain heat distribution at different time steps;

[0209] The heat distribution is regularized into a probability distribution to generate the heat distribution probability map, wherein the high probability area in the heat distribution probability map corresponds to the key part of the tumor boundary, and the low probability area corresponds to the smooth transition area of ​​the tumor boundary.

[0210] In a specific application scenario, the extraction module 32 may be used to construct a deformation field according to the curvature tensor field and the heat distribution probability map, wherein the deformation field satisfies a local homeomorphism mapping condition and maintains a local topological structure unchanged;

[0211] Adjusting the shape of the convolution kernel based on the deformation field so that the adjusted convolution kernel has a greater ability to capture the deformed multimodal tumor boundary features than a preset ability, wherein the adjustment strategy of the shape adjustment includes scaling transformation, rotation transformation, and non-rigid deformation;

[0212] The adjusted convolution kernel is used to perform a position-adaptive convolution operation on the multimodal medical imaging data to dynamically adjust the convolution kernel parameters, wherein the shape of the convolution kernel changes with position, the convolution operation adopts a dynamic sampling point strategy, and the sampling point position is determined by the deformation field.

[0213] In a specific application scenario, the generation module 33 can be used to map the multimodal tumor boundary features into a feature evolution phase space and identify key points of the multimodal tumor boundary features in the feature evolution phase space. The key points include at least stable points, bifurcation points, and attractors, corresponding to stable features, feature changes, and convergence states, respectively.

[0214] Performing Lyapunov exponent calculation on the characteristic trajectory of the key point to obtain an exponent calculation result, and using the exponent calculation result to evaluate the characteristic stability, wherein the exponent calculation result includes a positive exponent and a negative exponent, the positive exponent corresponds to an unstable characteristic, and the negative exponent corresponds to a stable characteristic;

[0215] Constructing a feature weight mapping based on feature stability evaluation results, wherein the stability evaluation results include stable features and unstable features, the stable features correspond to high weights, and the unstable features correspond to low weights;

[0216] A chaos control feedback mechanism is utilized to perform nonlinear feature weight adjustment on the multimodal tumor boundary features according to the feature weight mapping.

[0217] In a specific application scenario, the multimodal tumor boundary features include CT modality tumor boundary features and MRI modality tumor boundary features; the generation module 33 can be used to calculate the information entropy of the tumor boundary area under the CT modality tumor boundary features and the MRI modality tumor boundary features, respectively, wherein a high information entropy indicates rich information, and a low information entropy indicates redundant or insufficient information;

[0218] constructing an information entropy distribution map according to the information entropy, and using the information entropy distribution map to identify information contribution areas of the CT modality tumor boundary feature and the MRI modality tumor boundary feature;

[0219] calculating conditional mutual information between the CT modality tumor boundary feature and the MRI modality tumor boundary feature based on the information contribution area, so as to evaluate the complementarity between the CT modality tumor boundary feature and the MRI modality tumor boundary feature using the conditional mutual information;

[0220] Based on the modality fusion rule of maximizing conditional mutual information, the two modal tumor boundary features are evenly fused in the area where the complementarity is higher than a preset complementarity threshold, and / or, when the information entropy of the tumor boundary feature of a single modality is higher than a preset information entropy threshold, the single modality tumor boundary feature with the information entropy higher than the preset information entropy threshold is preferably fused, so as to perform feature fusion on the CT modality tumor boundary feature and the MRI modality tumor boundary feature to generate the multimodal tumor boundary enhancement feature;

[0221] The multimodal tumor boundary enhancement features are optimally transmitted on a statistical manifold to generate the tumor segmentation result.

[0222] In a specific application scenario, the generation module 33 may be used to divide the tumor boundary area into a plurality of local small blocks, and dynamically adjust the sizes of the plurality of local small blocks according to the local complexity;

[0223] Calculating the information entropy of each of the local small blocks and performing unbiased estimation technology on the information entropy;

[0224] The information entropy of each local block processed by the unbiased estimation technology is processed by the entropy smoothing technology.

[0225] The smoothed information entropy of each local block is mapped to the position of the tumor boundary area to generate the information entropy distribution map.

[0226] In specific application scenarios, such as Figure 9 As shown, the device further includes: a processing module 35;

[0227] a processing module 35 for performing boundary refinement processing on the tumor boundary enhancement feature, wherein the boundary refinement processing at least includes accurately locating the boundary position, optimizing boundary connectivity, and smoothing the boundary;

[0228] The tumor boundary enhancement feature after boundary refinement is verified, and the verification process at least includes checking topological consistency, morphological verification, and evaluating the confidence of the segmentation result.

[0229] In a specific application scenario, the generating module 33 may be used to generate a tumor segmentation result based on the multimodal tumor boundary enhancement feature after verification processing.

[0230] In specific application scenarios, such as Figure 9 As shown, the device further includes: a distribution module 36;

[0231] The allocation module 36 is used to allocate the tumor segmentation task to multiple edge computing devices, wherein each edge computing device is used to process the multimodal medical imaging data assigned according to the tumor segmentation task and perform parallel reasoning. Each edge computing device adopts a multi-threaded asynchronous mechanism to asynchronously obtain data blocks from the control device. Each edge computing device stores the intermediate reasoning results in the edge device storage. After completing part of the reasoning task, the part of the reasoning results is returned to the central server. The central server is used to integrate the reasoning results of all edge devices to generate the final tumor segmentation result.

[0232] It should be noted that for other corresponding descriptions of the functional units involved in the tumor segmentation device based on multimodal image dynamic feature fusion provided in this embodiment, please refer to Figure 1 The corresponding description in will not be repeated here.

[0233] Based on the above Figure 1 The method shown in FIG. 1 is a method for performing the above-mentioned steps. Accordingly, this embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned steps are performed. Figure 1 The method shown.

[0234] Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present disclosure.

[0235] Based on the above Figure 1 The method shown, and Figure 9In order to achieve the above-mentioned purpose, the embodiment of the present disclosure further provides an electronic device, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The method shown.

[0236] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, and the like. The user interface may include a display, an input unit such as a keyboard, and the like. The optional user interface may also include a USB interface, a card reader interface, and the like. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), and the like.

[0237] Those skilled in the art will understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0238] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.

[0239] Through the description of the above embodiments, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software plus the necessary general hardware platform, or by hardware. Compared with the prior art, the technical solution in the present disclosure obtains multimodal medical imaging data; inputs the multimodal medical imaging data into a constructed tumor boundary manifold model to extract multimodal tumor boundary features in the multimodal medical imaging data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model, wherein the tumor boundary manifold model is used to generate a heat distribution probability map based on the multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map; nonlinear feature weight adjustment is performed on the multimodal tumor boundary features, and the adjusted feature weights are fused with the multimodal tumor boundary features to generate multimodal tumor boundary enhancement features, so as to generate tumor segmentation results based on the multimodal tumor boundary enhancement features. By applying the solution of the present disclosure, by obtaining multimodal medical imaging data, the advantages of different modal images are comprehensively utilized, and the data of multiple modalities are fused to more comprehensively capture tumor features; the multimodal medical imaging data is input into the constructed tumor boundary manifold model. The model can generate a heat distribution probability map based on multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map. This dynamic adjustment mechanism can better adapt to the tumor boundary features in different imaging data, thereby improving the accuracy of boundary recognition; the present application scheme performs nonlinear feature weight adjustment on the multimodal tumor boundary features, and adopts the feature stability control mechanism of nonlinear dynamic system theory, which can dynamically allocate weights according to the importance and stability of the features, suppress the interference of noise features, and enhance the influence of effective features. The adjusted feature weights are then fused with the multimodal tumor boundary features to generate multimodal tumor boundary enhancement features. This fusion method comprehensively considers the importance and stability of different modal features, making the generated enhanced features more accurate and reliable, further improving the accuracy and reliability of image segmentation, and maintaining good segmentation effects even in low-quality images.

[0240] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further limitations, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0241] The foregoing are merely specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not to be limited to these embodiments, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A tumor image segmentation method based on multimodal image dynamic feature fusion, characterized in that: The method comprises: Acquire multimodal medical imaging data; Inputting the multimodal medical imaging data into the constructed tumor boundary manifold model to extract multimodal tumor boundary features in the multimodal medical imaging data using the convolution kernel parameters dynamically adjusted by the tumor boundary manifold model, wherein the tumor boundary manifold model is used to generate a heat distribution probability map based on the multimodal medical imaging data, and dynamically adjust the convolution kernel parameters based on the heat distribution probability map; Nonlinear feature weight adjustment is performed on the multimodal tumor boundary feature, and the adjusted feature weight is fused with the multimodal tumor boundary feature to generate a multimodal tumor boundary enhancement feature, so as to generate a tumor segmentation result based on the multimodal tumor boundary enhancement feature.

2. The method according to claim 1, characterized in that The process of constructing the tumor boundary manifold model includes: Extracting a tumor boundary point set from the multimodal medical imaging data, and constructing a boundary topological relationship of the tumor boundary point set through a local neighborhood connection strategy; performing weighted processing on the neighborhood connectivity relationships between the tumor boundary points in the tumor boundary point set according to the boundary topological relationship, and calculating the curvature tensor field of the tumor boundary points after the weighted processing; The tumor boundary manifold model is constructed based on the boundary topological relationship, the weighted neighborhood connectivity relationship, and the curvature tensor field.

3. The method according to claim 2, characterized in that The calculating of the curvature tensor field of the tumor boundary point after weighted processing includes: locally fitting the tumor boundary points to generate a quadratic surface of the tumor boundary points; Based on the quadratic surface, the principal curvature and the direction of the tumor boundary point are calculated to form a curvature tensor field of the tumor boundary point based on the principal curvature and the direction.

4. The method according to claim 2, characterized in that Generating a heat distribution probability map according to the multimodal medical image data includes: constructing a discrete differential operator based on the tumor boundary manifold model; calculating a local curvature of each of the tumor boundary points based on the discrete differential operator, and setting a thermal conductivity coefficient of each of the tumor boundary points according to the local curvature so that the thermal conductivity coefficient of a high curvature region is lower than the thermal conductivity coefficient of a low curvature region; Setting a plurality of heat diffusion time steps, wherein the heat diffusion time steps include a short time step and a long time step, the short time step is used to capture local detail features, and the long time step is used to capture global structural features; Calculating the heat kernel function of the plurality of heat diffusion time steps using the discrete differential operator, the local curvature, and the heat transfer coefficient to obtain heat distribution at different time steps; The heat distribution is regularized into a probability distribution to generate the heat distribution probability map, wherein the high probability area in the heat distribution probability map corresponds to the key part of the tumor boundary, and the low probability area corresponds to the smooth transition area of ​​the tumor boundary.

5. The method according to claim 2 or 3, characterized in that The dynamically adjusting the convolution kernel parameters according to the heat distribution probability map includes: Constructing a deformation field according to the curvature tensor field and the heat distribution probability map, wherein the deformation field satisfies a local homeomorphism mapping condition and keeps a local topological structure unchanged; Adjusting the shape of the convolution kernel based on the deformation field so that the adjusted convolution kernel has a greater ability to capture the deformed multimodal tumor boundary features than a preset ability, wherein the adjustment strategy of the shape adjustment includes scaling transformation, rotation transformation, and non-rigid deformation; The adjusted convolution kernel is used to perform a position-adaptive convolution operation on the multimodal medical imaging data to dynamically adjust the convolution kernel parameters, wherein the shape of the convolution kernel changes with position, the convolution operation adopts a dynamic sampling point strategy, and the sampling point position is determined by the deformation field.

6. The method according to claim 1, wherein The performing nonlinear feature weight adjustment on the multimodal tumor boundary feature includes: Mapping the multimodal tumor boundary features into a feature evolution phase space, and identifying key points of the multimodal tumor boundary features in the feature evolution phase space, wherein the key points include at least stable points, bifurcation points, and attractors, corresponding to stable features, feature changes, and convergence states, respectively; Performing Lyapunov exponent calculation on the characteristic trajectory of the key point to obtain an exponent calculation result, and using the exponent calculation result to evaluate the characteristic stability, wherein the exponent calculation result includes a positive exponent and a negative exponent, the positive exponent corresponds to an unstable characteristic, and the negative exponent corresponds to a stable characteristic; Constructing a feature weight mapping based on feature stability evaluation results, wherein the stability evaluation results include stable features and unstable features, the stable features correspond to high weights, and the unstable features correspond to low weights; A chaos control feedback mechanism is utilized to perform nonlinear feature weight adjustment on the multimodal tumor boundary features according to the feature weight mapping.

7. The method according to claim 1, characterized in that The multimodal tumor boundary features include CT modality tumor boundary features and MRI modality tumor boundary features; The step of fusing the adjusted feature weights with the multimodal tumor boundary features to generate a multimodal tumor boundary enhancement feature, so as to generate a tumor segmentation result based on the multimodal tumor boundary enhancement feature, includes: Calculating the information entropy of the tumor boundary area under the CT modality tumor boundary feature and the MRI modality tumor boundary feature respectively, wherein a high information entropy indicates rich information, and a low information entropy indicates redundant or insufficient information; constructing an information entropy distribution map according to the information entropy, and using the information entropy distribution map to identify information contribution areas of the CT modality tumor boundary feature and the MRI modality tumor boundary feature; calculating conditional mutual information between the CT modality tumor boundary feature and the MRI modality tumor boundary feature based on the information contribution area, so as to evaluate the complementarity between the CT modality tumor boundary feature and the MRI modality tumor boundary feature using the conditional mutual information; Based on the modality fusion rule of maximizing conditional mutual information, the two modal tumor boundary features are evenly fused in the area where the complementarity is higher than a preset complementarity threshold, and / or, when the information entropy of the tumor boundary feature of a single modality is higher than a preset information entropy threshold, the single modality tumor boundary feature with the information entropy higher than the preset information entropy threshold is preferably fused, so as to perform feature fusion on the CT modality tumor boundary feature and the MRI modality tumor boundary feature to generate the multimodal tumor boundary enhancement feature; The multimodal tumor boundary enhancement features are optimally transmitted on a statistical manifold to generate the tumor segmentation result.

8. The method according to claim 7, characterized in that The respectively calculating the information entropy of the tumor boundary area under the CT modality tumor boundary feature and the MRI modality tumor boundary feature includes: Dividing the tumor boundary area into a plurality of local small blocks, and dynamically adjusting the sizes of the plurality of local small blocks according to local complexity; Calculating the information entropy of each of the local small blocks and performing unbiased estimation technology on the information entropy; Performing entropy smoothing technology processing on the information entropy of each local small block processed by the unbiased estimation technology; The constructing of an information entropy distribution map according to the information entropy includes: The smoothed information entropy of each local block is mapped to the position of the tumor boundary area to generate the information entropy distribution map.

9. The method according to claim 1, characterized in that Before generating a tumor segmentation result based on the multimodal tumor boundary enhancement feature, the method further includes: Performing boundary refinement processing on the tumor boundary enhancement feature, wherein the boundary refinement processing at least includes accurately locating the boundary position, optimizing boundary connectivity, and boundary smoothing processing; Performing verification processing on the tumor boundary enhancement features after boundary refinement processing, wherein the verification processing at least includes checking topological consistency, morphological verification, and evaluating the confidence of the segmentation result; Generating a tumor segmentation result based on the multimodal tumor boundary enhancement feature includes: A tumor segmentation result is generated based on the multimodal tumor boundary enhancement feature after verification processing.

10. The method according to claim 1, characterized in that The method further comprises: The tumor segmentation task is assigned to multiple edge computing devices, wherein each edge computing device is used to process the multimodal medical imaging data assigned according to the tumor segmentation task and perform parallel reasoning. Each edge computing device adopts a multi-threaded asynchronous mechanism to asynchronously obtain data blocks from the control device. Each edge computing device stores the intermediate reasoning results in the edge device storage. After completing part of the reasoning task, the part of the reasoning results is returned to the central server. The central server is used to integrate the reasoning results of all edge devices to generate the final tumor segmentation result.

Images