An endoluminal image real-time lesion recognition and boundary segmentation system

Abstract

The application relates to the technical field of medical image processing, and particularly discloses a cavity image real-time lesion recognition and boundary segmentation system. A cross-sectional image sequence in a cavity is acquired through an interventional imaging catheter, and after shared features are extracted by using a deep network, pixel-level tissue boundary segmentation and overall lesion property recognition are synchronously performed. The core lies in that a dynamic gradient coordination mechanism is introduced, gradient direction conflicts between the segmentation and recognition double tasks are detected and resolved in real time during the training process, and adaptive projection transformation based on the task convergence state is used to fuse the gradients, so that the shared network can learn balanced features which are optimal for both tasks; the method realizes end-to-end real-time processing from image input to synchronous output of the segmentation and recognition results, and significantly improves the synergy, accuracy of lesion analysis and real-time auxiliary efficiency of clinical operation navigation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a real-time lesion recognition and boundary segmentation system for intracavitary images. Background Technology

[0002] In the field of interventional vascular diagnosis and treatment, intravascular imaging technologies such as intravascular ultrasound (IVUS) and optical coherence tomography (OCT) are important tools for assessing vascular lesions. Current technologies widely utilize deep learning-based image recognition methods to automate the analysis of acquired intravascular images, enabling the identification and segmentation of lesions. In particular, a multi-task learning architecture employing a shared base network has been used to attempt to simultaneously perform pixel-level segmentation of target boundaries and image recognition and classification of lesion properties within a single model, aiming to improve analytical efficiency.

[0003] The existing technology has the following shortcomings:

[0004] In real-time intracavitary image analysis, the fundamental difference in feature requirements between pixel-level segmentation and overall lesion recognition leads to directional conflicts in the gradient vectors of multiple tasks during shared network training, resulting in severe oscillations in the optimization process and difficulty in model convergence to an effective equilibrium point. Summary of the Invention

[0005] The purpose of this invention is to provide a real-time lesion identification and boundary segmentation system for intracavitary imaging to solve the problems mentioned above.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A real-time lesion identification and boundary segmentation system for intracavitary imaging includes:

[0008] The image acquisition and input module scans the cavities of a biological body through an interventional imaging catheter, acquiring a continuous sequence of cross-sectional images of the cavity in real time.

[0009] The shared deep feature generation module is used to learn and extract deep features from intracavitary cross-sectional image sequences, and outputs a set of shared deep features that simultaneously contain local fine structure and overall contextual information.

[0010] The synchronous dual-task parsing module is used to simultaneously input shared deep features into the pixel-level parsing path and the overall semantic induction path; the pixel-level parsing path processes the shared deep features and outputs a pixel-level segmentation map that accurately identifies the tissue contour; the overall semantic induction path processes the shared deep features and outputs a semantic attribute determination that identifies the nature of the lesion.

[0011] The dynamic gradient coordination and fusion module is used to obtain the parameter update vectors of the pixel-level parsing path and the overall semantic induction path in real time during the training phase of deep feature learning; to perform conflict detection on the two parameter update vectors, and when a serious conflict is detected, to perform a projection operation on one of the parameter update vectors to eliminate the conflicting components of the parameter update vectors, and then fuse them to obtain a coordinated unified update vector.

[0012] The network optimization and real-time output module uses a unified update vector to iteratively optimize the parameters of the deep feature learning process. With the optimized parameter configuration, it processes the intracavitary cross-sectional image of each frame, and synchronously outputs and presents the corresponding pixel-level segmentation map and semantic attribute determination.

[0013] As a further aspect of the present invention: the output process of the shared deep features is as follows:

[0014] Multi-level convolution processing is performed on the original intracavitary image to obtain an initial set of feature maps containing information at different scales.

[0015] Based on the dynamic evaluation of the contribution of features at different scales in the initial feature map set to the current dual task, adaptive fusion weights are assigned to features at each scale.

[0016] The initial feature map set is weighted and fused according to the fusion weight, and a nonlinear transformation is used to generate the final shared deep feature that simultaneously contains local fine structure and overall contextual information.

[0017] As a further aspect of the present invention: the multi-level convolutional processing of the original intracavitary image to obtain an initial feature map set containing information at different scales specifically includes:

[0018] The original intracavity image is processed by a convolution kernel with an adaptive receptive field to generate a first-level feature map.

[0019] After downsampling the first-level feature map, parallel processing is performed using dilated convolution with increasing dilation rate to generate multiple second-level feature maps that capture contextual information in different visual ranges.

[0020] Multiple second-level feature maps are concatenated along the channel dimension, and then dimensionality reduction and feature integration are performed through pointwise convolution to form an initial feature map set containing information at different scales.

[0021] As a further aspect of the present invention: the output of a pixel-level segmentation map that accurately identifies tissue contours specifically includes:

[0022] An initial upsampling operation is performed on the shared deep features to generate primary upsampled features with higher spatial resolution;

[0023] The initial upsampled features are input into a spatial attention-guided feature enhancement network. The feature enhancement network dynamically enhances the feature response of the boundary region based on the correlation strength between each position in the feature map and the adjacent tissue boundary, thus generating boundary enhancement features.

[0024] A pixel-by-pixel nonlinear transformation and category mapping are performed on the boundary enhancement features to directly generate a pixel-level segmentation map that identifies the tissue contour.

[0025] As a further aspect of the present invention: the semantic attribute determination of the output identifier of the lesion nature specifically includes:

[0026] Multi-scale adaptive spatial pooling is performed on shared deep features to aggregate contextual information from different fields of view and generate a global context vector;

[0027] Based on the global context vector, the contribution of each channel dimension in the shared deep features to the determination of the nature of the lesion is weighted to generate semantically refined features;

[0028] The semantically refined features are subjected to dimensionality reduction and logistic regression to output semantic attribute determinations that identify the nature of the lesion.

[0029] As a further aspect of the present invention: the fusion yields a coordinated unified update vector, specifically including:

[0030] Calculate the degree of inconsistency in direction between the parameter update vector of the pixel-level parsing path and the parameter update vector of the overall semantic induction path;

[0031] The degree of inconsistency is compared with a preset threshold to determine whether a serious conflict has occurred;

[0032] When a serious conflict is determined, a projection transformation is performed on the parameter update vector of the overall semantic induction path. The projection transformation sets the component of the parameter update vector in the direction that conflicts with the parameter update vector of the pixel-level parsing path to zero.

[0033] The parameter update vector of the overall semantic induction path after projection transformation is combined with the parameter update vector of the pixel-level parsing path to obtain a unified update vector.

[0034] As a further aspect of the present invention: the projection transformation of the parameter update vector of the overall semantic induction path specifically includes:

[0035] Based on the direction of the parameter update vector of the pixel-level parsing path, construct an orthogonal projection plane;

[0036] Based on the convergence state of the current task loss in the overall semantic induction path, the projection intensity coefficient is adaptively calculated.

[0037] The parameter update vector of the overall semantic induction path is weighted by the projection intensity coefficient and the orthogonal projection plane to generate a transformation vector after setting the conflict components to zero.

[0038] As a further aspect of the present invention: the calculation process of the projection intensity coefficient is as follows:

[0039] Obtain the task loss value generated by the overall semantic induction path in the current training iteration cycle, and the task loss value generated in the previous iteration cycle;

[0040] Calculate the absolute difference between the current task loss value and the previous task loss value, and divide the absolute difference by the previous task loss value to obtain the relative rate of change of task loss;

[0041] The relative rate of change is compared with a preset convergence threshold, and a level value is selected from a set of discrete intensity level values ​​as the base projected intensity based on the comparison result.

[0042] The projection intensity coefficients used in the weighted projection calculation are obtained by multiplying the base projection intensity by a weighting factor that dynamically decays according to the historical conflict frequency.

[0043] As a further aspect of the present invention: the output and presentation of the corresponding pixel-level segmentation map and semantic attribute determination specifically includes:

[0044] Receive the pixel-level segmentation map and the semantic attribute determination, and convert the semantic attribute determination into the corresponding visual color coding rule;

[0045] Based on color coding rules, the contour regions of different categories of tissues in the pixel-level segmentation map are simultaneously colored and rendered to generate a semantically distinguishable color segmentation map.

[0046] The color segmentation image is semi-transparently overlaid and fused with the original intracavitary cross-sectional image to generate the final real-time diagnostic fusion view, which is then output and displayed.

[0047] The beneficial effects of this invention are:

[0048] (1) This invention quantifies the degree of conflict by calculating the cosine similarity between gradient vectors in real time, and introduces an adaptive projection intensity coefficient based on the convergence state of the task loss to accurately suppress and fuse conflict gradient components. This enables the shared encoder to learn a balanced feature representation that is more favorable for both boundary segmentation and lesion recognition, thereby improving the accuracy of lesion region segmentation and the accuracy of overall property determination while ensuring real-time performance.

[0049] (2) This invention integrates feature extraction, dual-task parsing, gradient coordination optimization, and result fusion display into a coherent process. During the training phase, the gradient coordination mechanism optimizes the network parameters; during the real-time application phase, the optimized network can process hundreds of frames of image streams per second in real time and simultaneously generate a fused view overlaid with color segmentation maps and semantic warnings. This provides intuitive and quantitative real-time navigation for clinical interventional surgery, helping surgeons to instantly determine the nature and boundaries of plaques, thereby improving surgical accuracy while reducing operational uncertainty and risks caused by reliance on postoperative offline analysis. Attached Figure Description

[0050] The invention will now be further described with reference to the accompanying drawings.

[0051] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Please see Figure 1 As shown, the present invention is a real-time lesion identification and boundary segmentation system for intracavitary imaging, comprising:

[0054] The image acquisition and input module scans the cavities of a biological body through an interventional imaging catheter, acquiring a continuous sequence of cross-sectional images of the cavity in real time.

[0055] The shared deep feature generation module is used to learn and extract deep features from intracavitary cross-sectional image sequences, and outputs a set of shared deep features that simultaneously contain local fine structure and overall contextual information.

[0056] The synchronous dual-task parsing module is used to simultaneously input shared deep features into the pixel-level parsing path and the overall semantic induction path; the pixel-level parsing path processes the shared deep features and outputs a pixel-level segmentation map that accurately identifies the tissue contour; the overall semantic induction path processes the shared deep features and outputs a semantic attribute determination that identifies the nature of the lesion.

[0057] The dynamic gradient coordination and fusion module is used to obtain the parameter update vectors of the pixel-level parsing path and the overall semantic induction path in real time during the training phase of deep feature learning; to perform conflict detection on the two parameter update vectors, and when a serious conflict is detected, to perform a projection operation on one of the parameter update vectors to eliminate the conflicting components of the parameter update vectors, and then fuse them to obtain a coordinated unified update vector.

[0058] The network optimization and real-time output module uses a unified update vector to iteratively optimize the parameters of the deep feature learning process. With the optimized parameter configuration, it processes the intracavitary cross-sectional image of each frame, and synchronously outputs and presents the corresponding pixel-level segmentation map and semantic attribute determination.

[0059] The image acquisition and input module uses a miniature imaging catheter that can be inserted into a biological cavity (such as a blood vessel). The catheter's tip integrates a high-frequency ultrasound transducer or an optical coherence tomography (OCT) probe. During the interventional procedure, the operator advances the catheter to the target area.

[0060] Once the catheter is in place, its internal imaging core is driven to perform a uniform retraction scan. During retraction, the probe emits energy pulses at a preset frame rate (e.g., 180 frames per second) towards the catheter wall and receives the reflected or scattered signals. These raw analog signals are transmitted in real time to the external host system via miniature transmission lines within the catheter.

[0061] A dedicated high-speed data acquisition card within the host system receives these analog signals and converts them into digital signals. Subsequently, the built-in signal processor performs a series of preprocessing steps on the digital signals, including gain compensation, noise reduction filtering, and digital reconstruction, ultimately generating a series of clear, spatially ordered grayscale images of the intracavity cross-section, forming an image sequence for real-time analysis.

[0062] In the shared deep feature generation module, the original intracavity image is first processed through multi-level convolution to obtain multi-scale features. Specifically, a convolutional kernel with an adaptive receptive field is used for the first layer of convolution. This adaptive receptive field is implemented as follows: for each sampling position covered by the convolutional kernel, the gray-level variance within a 4×4 neighborhood window around the corresponding pixel in the input image is first calculated; then, based on the relationship between this variance and a preset threshold, the actual coordinates of the sampling point are dynamically adjusted. If the variance is greater than the threshold, standard grid sampling is maintained; if the variance is less than or equal to the threshold, the sampling coordinates are randomly perturbed within a small offset range centered on that point. The first-level feature map generated by this operation has the same spatial resolution as the input image. Subsequently, this first-level feature map is subjected to max pooling downsampling with a stride of 2. The downsampled feature map is then processed in parallel using three different dilated convolutions, with dilation rates of 2, 4, and 6, respectively, and a standard kernel size of 3×3. These three parallel processes generate second-level feature maps that capture contextual information within narrow, medium, and wide field-of-view ranges, respectively. Finally, these three second-level feature maps are concatenated along the channel direction, and then a 1×1 convolution kernel is used to perform a convolution operation to reduce the number of channels to a preset value, completing dimensionality reduction and initial integration, ultimately forming an initial set of feature maps containing information at different scales.

[0063] Secondly, the contribution of features at different scales is dynamically evaluated and adaptive fusion weights are assigned. Specifically, during the forward propagation phase of training, the frequency of each scale feature map in the initial feature map set being directly referenced in the subsequent loss calculations for the dual tasks (pixel-level parsing and overall semantic induction) is recorded. In each training iteration, the relative rate of change of the referenced proportion of each scale feature map in the current iteration compared to the referenced proportion in the previous iteration is calculated. This relative rate of change is calculated by subtracting the current proportion from the previous proportion and dividing the absolute value of the difference by the previous proportion. Then, two thresholds are set: the first threshold is 0.1, and the second threshold is 0.3. Initial weight levels are assigned based on the range in which the relative rate of change falls: if the rate of change is below the first threshold, the initial weight level is set to the highest level (value 1.0); if the rate of change is between the first and second thresholds, the initial weight level is set to the medium level (value 0.5); if the rate of change is above the second threshold, the initial weight level is set to the lowest level (value 0.1). Finally, this weight level is multiplied by the average contribution ratio (i.e., average reference ratio) of the feature map in the most recent ten consecutive iterations to obtain the final adaptive fusion weight value for the current iteration.

[0064] Finally, feature fusion is performed based on the fusion weights to generate shared deep features. Specifically, the feature map at each scale in the initial feature map set is multiplied channel-by-channel by its corresponding final adaptive fusion weight value. All weighted feature maps are then summed element-wise at corresponding positions. The resulting fused feature map is input into a structure containing two consecutive computational layers. The first layer first calculates the mean and standard deviation for all channel values ​​at each spatial location, then subtracts the mean from each original channel value and divides it by the standard deviation to normalize it. The second layer performs a non-linear transformation calculation on each normalized channel value: if the value is greater than zero, the value itself is output; if the value is less than or equal to zero, the result multiplied by a fixed coefficient (set to 0.1) is output. After this series of calculations, the output is the final shared deep feature that simultaneously contains local fine structure and overall contextual information.

[0065] In the synchronous dual-task parsing module, the shared depth features are first upsampled in the pixel-level parsing path. Specifically, an operation called transposed convolution is used, with a kernel size of 4x4 and a stride of 2. After this operation, the spatial width and height of the feature map are doubled while the number of channels remains unchanged, thus generating a primary upsampled feature. This primary upsampled feature is then input into a spatial attention-guided feature enhancement process. This process first calculates the cosine similarity between the feature vector at each spatial location in the feature map and the feature vectors at all locations within its eight neighborhoods. All similarity values ​​are averaged to obtain an association strength score between -1 and +1. This score is amplified by a preset scaling factor (set to 2.0) and then converted into an attention weight between 0 and 1 by a sigmoid function (i.e., calculating 1 divided by 1 plus e raised to the power of -x, where x is the input value). This weight is used to weight the feature vector at the original location, i.e., multiplying the original vector by the weight value, thereby achieving dynamic enhancement of the boundary region and outputting a boundary enhancement feature. Finally, pixel-by-pixel processing is performed on this boundary enhancement feature. The feature vector at each location is passed through a fully connected layer, mapping the number of channels to equal the total number of tissue categories (e.g., 4 categories). Then, a softmax operation is performed on the mapped vector at each location (i.e., calculating the score of each category in the vector as a power of e, then dividing by the sum of the powers of e across all categories), converting the scores into probabilities. The category with the highest probability is selected as the final category label for that pixel, thus directly generating a pixel-level segmentation map.

[0066] In the overall semantic induction path, multi-scale adaptive spatial pooling is first performed on the shared deep features. Specifically, three parallel pooling branches are constructed: the first branch performs global average pooling, calculating the arithmetic mean of the spatial values ​​of all locations for each channel in the feature map; the second branch divides the feature map spatially into four regions (2 rows, 2 columns), calculates the average value of each channel within each region, and then concatenates the four results; the third branch divides the feature map into nine regions (3 rows, 3 columns) and performs the same operation and concatenation. The output vectors of the three branches are then concatenated again to form a comprehensive global context vector. Next, the shared deep features are recalibrated based on this global context vector. Specifically, this vector is input into a structure with two fully connected layers: the first layer reduces the vector dimension to 1 / 16 of the original number of channels (rounded to the nearest integer) and uses the ReLU function (i.e., the larger value between the input and output values ​​and 0) for non-linear activation; the second layer restores the dimension to the original number of channels and outputs a channel weight vector between 0 and 1 using the Sigmoid function. This weight vector is multiplied channel-by-channel with the original shared deep features to generate semantically refined features. Finally, dimensionality reduction and logistic regression are performed on the refined semantic features. First, global average pooling is applied to the feature map to obtain a one-dimensional feature vector. This vector is then input into a final fully connected classification layer, whose output neurons have the same number as the number of lesion categories (e.g., 2 categories). A softmax operation is also performed on the output of this layer to obtain the probability of each lesion category. The category with the highest probability is then determined as the final semantic attribute classification result.

[0067] In the dynamic gradient coordination and fusion module, the degree of inconsistency in the direction of the parameter update vectors of the pixel-level parsing path and the overall semantic induction path is first calculated. Specifically, the parameter update values ​​of the two tasks corresponding to a certain parameter to be updated in the shared feature extraction network are treated as scalar components in two vectors. The scalar components of all parameters to be updated are then ordered to form two complete multidimensional parameter update vectors, denoted as vectors. sum vector The degree of inconsistency is quantified by calculating the cosine similarity between the two vectors. The calculation process is as follows: First, calculate the dot product of the two vectors, that is, multiply the components at corresponding positions and then sum them to obtain a total value; then calculate the magnitude of each vector, that is, sum the squares of all the components of that vector and take the square root to obtain two values; finally, divide the sum of the dot product values ​​by the product of the two magnitude values, and the result is the cosine value. The cosine value ranges from -1 to +1. The closer its absolute value is to 1, the more consistent the directions; the closer it is to 0, the more orthogonal the directions; a negative value indicates opposite directions. In this step, subtracting the cosine value from 1... This serves as the final measure of the degree of inconsistency.

[0068] Secondly, the degree of inconsistency is compared with a preset threshold to determine whether a serious conflict has occurred. The preset threshold is fixed at 0.5. When the calculated degree of inconsistency metric is greater than the preset threshold of 0.5, it is determined that a serious conflict has occurred between the two parameter update vectors, and a coordination mechanism needs to be initiated; otherwise, the conflict is considered not serious, and vector synthesis can be performed directly.

[0069] When a serious conflict is detected, the parameter update vector of the overall semantic induction path is performed. Perform a projection transformation. This process first updates the vector using the parameters of the pixel-level resolved path. Using the direction as a reference, construct its orthogonal projection plane. Mathematically, the vector... Projected onto In the orthogonal direction, the interaction with... Conflicting components. Vector Perpendicular to orthogonal projection components in the direction It can be calculated using the following formula: ;

[0070] In the formula, Representing vectors with vector dot product, Representing vectors The square of the modulus. The core of this calculation lies in first... Subtract its in The component in the direction, the remaining part is the component with respect to the direction. Conflict-free orthogonal components. However, directly using fully orthogonal components may be too aggressive. Therefore, an adaptively calculated projection intensity coefficient is introduced. Weighted adjustment is performed.

[0071] Projection intensity coefficient The calculation process is as follows: First, obtain the task loss value generated by the overall semantic induction path in the current training iteration cycle. And the task loss value generated in the previous iteration cycle. Calculate the absolute difference. And divide this absolute difference by the previous loss value. The relative rate of change of task loss was obtained. ,Right now: The preset convergence threshold is set to 0.1. The relative rate of change is compared to this convergence threshold: if... If the value is less than 0.1, the loss is considered to have converged well, and the basic projection intensity S is selected as the strong intervention level with a value of 1.0; if If the value is ≥0.1, the loss is considered to be still changing drastically, and the base projection intensity S is selected as the weak intervention level with a value of 0.3. Simultaneously, a weighting factor D is maintained, dynamically decaying based on the historical conflict frequency. D is initialized to 1.0. Each time a severe conflict is determined at this point and a projection transformation is initiated, D is multiplied by a decay rate of 0.9. If no conflict occurs for 10 consecutive iterations, D is reset to 1.0. Finally, the projection intensity coefficient... It is obtained by multiplying the basic projected intensity S by this dynamic attenuation weighting factor D, and the calculation formula is as follows: ; Obtain the projection intensity coefficient Then, it is used to weight the above orthogonal projection operations to generate the final transformation vector used for updating. Specifically, it does not directly use completely orthogonal... Instead, it is combined with the original vector. The weighted mixture is calculated using the following formula: .when When the value is 1, orthogonal components are used completely, and conflict elimination is most thorough; when... When the value is 0, the original vector is completely preserved. This can be achieved by adjusting... It enables fine-grained control of the intensity of projection intervention based on the current learning status of the classification task and historical conflict situations.

[0072] Finally, the parameter update vector of the overall semantic induction path after projection transformation is synthesized with the parameter update vector of the pixel-level parsing path. The synthesis method is a simple vector addition, that is, the components at corresponding positions are added together to obtain a unified update vector. This unified update vector is used for subsequent iterative updates of the parameters of the shared feature extraction network, thereby promoting the network to learn a general feature representation that is beneficial to both tasks at the mathematical optimization level.

[0073] In the network optimization and real-time output module, the pixel-level segmentation map and semantic attribute determination results generated by the previous steps are received first. The semantic attribute determination results, i.e., the category labels representing different lesion properties (e.g., using a value of 0 to represent "stable plaque" and a value of 1 to represent "vulnerable plaque"), are converted into corresponding visual color coding rules. Specifically, a query mapping table is established, pre-assigning a color value composed of the three primary colors of red, green, and blue for each possible category label. For example, for tissue categories, the mapping relationship can be set as follows: label 0 (background) corresponds to the color (0, 0, 0), i.e., black; label 1 (fibrous tissue) corresponds to the color (0, 255, 0), i.e., green; label 2 (lipid core) corresponds to the color (255, 255, 0), i.e., yellow; and label 3 (calcification) corresponds to the color (255, 255, 255), i.e., white. Simultaneously, the semantic attribute determination result (such as "vulnerable plaque") triggers a global warning color box rule, for example, overlaying a red semi-transparent border around the image.

[0074] Secondly, based on the established color coding rules, the pixel-level segmentation map is colored and rendered. The processing is done pixel by pixel: the category label value of the current pixel position in the segmentation map is read, and the corresponding intensity values ​​of the red, green, and blue channels are found according to the above mapping table. Then, a new color image with the same size as the original segmentation map is created, and the three intensity values ​​found are filled into the red, green, and blue channels of the corresponding pixel position in the new image, respectively. After traversing all pixels, a color segmentation map is generated in which all tissue regions are assigned a specific color according to their category. If the semantic attribute of the current frame is determined to be a type that requires warning, a red rectangular border with a width of 10 pixels is drawn around the generated color segmentation map.

[0075] Finally, the color segmentation image is semi-transparently overlaid and fused with the original intracavitary cross-sectional grayscale image. Specifically, for each pixel in the fused image, the final red channel value is calculated as follows: the red channel value of the color segmentation image at that location is multiplied by a factor of 0.6, and then the pixel intensity value of the original grayscale image at that location is multiplied by a factor of 0.4. The final values ​​of the green and blue channels are calculated using the same weighting factor: the channel value of the color segmentation image is multiplied by 0.6, and the intensity value of the original image is multiplied by 0.4. The single intensity value of the original image is used for both green and blue channel calculations. This calculation allows the information from the color segmentation image to be overlaid on the original image with 60% transparency, clearly displaying segmentation boundaries and regions while preserving the underlying realistic anatomical texture. After overlay, if a warning border exists, its red border will also be overlaid on the final image using the same transparency rules. This final real-time diagnostic fusion view is directly output to a graphics display device for real-time reference by the surgeon. This process is performed independently for each newly input image frame to ensure real-time display.

[0076] The working principle of this invention is as follows: An intraluminal cross-sectional image sequence is acquired and input via an interventional imaging catheter; a shared depth feature generation module performs multi-level convolution and adaptive multi-scale feature fusion on the images to extract shared depth features that simultaneously contain local details and global context; a synchronous dual-task parsing module inputs these features in parallel to a pixel-level parsing path and an overall semantic induction path, respectively outputting a segmentation map accurately identifying tissue contours and a semantic attribute determination identifying the nature of lesions; during the training phase, a dynamic gradient coordination and fusion module calculates the cosine similarity of the dual-task parameter update vectors in real time to detect conflicts. When a severe conflict is detected, the projection intensity coefficient is adaptively calculated based on the task loss convergence state, and a projection transformation is applied to the classification task gradient vector to eliminate conflict components, thereby fusing to obtain a unified update vector and optimizing the network parameters; finally, a network optimization and real-time output module processes each frame of image using the optimized parameters, semi-transparently overlays the generated color segmentation map with the original image, and synchronously outputs a real-time diagnostic fusion view.

[0077] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A real-time lesion identification and boundary segmentation system for intracavitary imaging, characterized in that, include: The image acquisition and input module scans the cavities of a biological body through an interventional imaging catheter, acquiring a continuous sequence of cross-sectional images of the cavity in real time. The shared deep feature generation module is used to learn and extract deep features from intracavitary cross-sectional image sequences, and outputs a set of shared deep features that simultaneously contain local fine structure and overall contextual information. The synchronous dual-task parsing module is used to simultaneously input shared deep features into the pixel-level parsing path and the overall semantic induction path; the pixel-level parsing path processes the shared deep features and outputs a pixel-level segmentation map that accurately identifies the tissue contour; the overall semantic induction path processes the shared deep features and outputs a semantic attribute determination that identifies the nature of the lesion. The dynamic gradient coordination and fusion module is used to obtain the parameter update vectors of the pixel-level parsing path and the overall semantic induction path in real time during the training phase of deep feature learning; to perform conflict detection on the two parameter update vectors, and when a serious conflict is detected, to perform a projection operation on one of the parameter update vectors to eliminate the conflicting components of the parameter update vectors, and then fuse them to obtain a coordinated unified update vector. The network optimization and real-time output module uses a unified update vector to iteratively optimize the parameters of the deep feature learning process. With the optimized parameter configuration, it processes the intracavitary cross-sectional image of each frame, and synchronously outputs and presents the corresponding pixel-level segmentation map and semantic attribute determination.

2. The real-time intracavitary lesion identification and boundary segmentation system according to claim 1, characterized in that, The output process of the shared deep features is as follows: Multi-level convolution processing is performed on the original intracavitary image to obtain an initial set of feature maps containing information at different scales; Based on the dynamic evaluation of the contribution of features at different scales in the initial feature map set to the current dual task, adaptive fusion weights are assigned to features at each scale. The initial feature map set is weighted and fused according to the fusion weight, and a nonlinear transformation is used to generate the final shared deep feature that simultaneously contains local fine structure and overall contextual information.

3. The real-time intracavitary lesion identification and boundary segmentation system according to claim 2, characterized in that, The process of performing multi-level convolution on the original intracavity image to obtain an initial feature map set containing information at different scales specifically includes: The original intracavity image is processed by a convolution kernel with an adaptive receptive field to generate a first-level feature map. After downsampling the first-level feature map, parallel processing is performed using dilated convolution with increasing dilation rate to generate multiple second-level feature maps that capture contextual information in different visual ranges. Multiple second-level feature maps are concatenated along the channel dimension, and then dimensionality reduction and feature integration are performed through pointwise convolution to form an initial feature map set containing information at different scales.

4. The real-time intracavitary lesion identification and boundary segmentation system according to claim 1, characterized in that, The output of a pixel-level segmentation map that accurately identifies the tissue contour specifically includes: An initial upsampling operation is performed on the shared deep features to generate primary upsampled features with higher spatial resolution; The initial upsampled features are input into a spatial attention-guided feature enhancement network. The feature enhancement network dynamically enhances the feature response of the boundary region based on the correlation strength between each position in the feature map and the adjacent tissue boundary, thus generating boundary enhancement features. A pixel-by-pixel nonlinear transformation and category mapping are performed on the boundary enhancement features to directly generate a pixel-level segmentation map that identifies the tissue contour.

5. The real-time intracavitary lesion identification and boundary segmentation system according to claim 1, characterized in that, The semantic attribute determination of the output identifier lesion nature specifically includes: Multi-scale adaptive spatial pooling is performed on shared deep features to aggregate contextual information from different fields of view and generate a global context vector; Based on the global context vector, the contribution of each channel dimension in the shared deep features to the determination of the nature of the lesion is weighted to generate semantically refined features; The semantically refined features are subjected to dimensionality reduction and logistic regression to output semantic attribute determinations that identify the nature of the lesion.

6. The real-time intracavitary lesion identification and boundary segmentation system according to claim 1, characterized in that, The fusion yields a coordinated unified update vector, specifically including: Calculate the degree of inconsistency in direction between the parameter update vector of the pixel-level parsing path and the parameter update vector of the overall semantic induction path; The degree of inconsistency is compared with a preset threshold to determine whether a serious conflict has occurred; When a serious conflict is determined, a projection transformation is performed on the parameter update vector of the overall semantic induction path. The projection transformation sets the component of the parameter update vector in the direction that conflicts with the parameter update vector of the pixel-level parsing path to zero. The parameter update vector of the overall semantic induction path after projection transformation is combined with the parameter update vector of the pixel-level parsing path to obtain a unified update vector.

7. The real-time intracavitary lesion identification and boundary segmentation system according to claim 6, characterized in that, The projection transformation of the parameter update vector of the overall semantic induction path specifically includes: Based on the direction of the parameter update vector of the pixel-level parsing path, construct an orthogonal projection plane; Based on the convergence state of the current task loss in the overall semantic induction path, the projection intensity coefficient is adaptively calculated. The parameter update vector of the overall semantic induction path is weighted by the projection intensity coefficient and the orthogonal projection plane to generate a transformation vector after setting the conflict components to zero.

8. The real-time intracavitary lesion identification and boundary segmentation system according to claim 7, characterized in that, The calculation process for the projection intensity coefficient is as follows: Obtain the task loss value generated by the overall semantic induction path in the current training iteration cycle, and the task loss value generated in the previous iteration cycle; Calculate the absolute difference between the current task loss value and the previous task loss value, and divide the absolute difference by the previous task loss value to obtain the relative rate of change of task loss; The relative rate of change is compared with a preset convergence threshold, and a level value is selected from a set of discrete intensity level values ​​as the base projected intensity based on the comparison result. The projection intensity coefficients used in the weighted projection calculation are obtained by multiplying the base projection intensity by a weighting factor that dynamically decays according to the historical conflict frequency.

9. The real-time intracavitary lesion identification and boundary segmentation system according to claim 1, characterized in that, The output, which presents the corresponding pixel-level segmentation map and semantic attribute determination, specifically includes: Receive the pixel-level segmentation map and the semantic attribute determination, and convert the semantic attribute determination into the corresponding visual color coding rule; Based on color coding rules, the contour regions of different categories of tissues in the pixel-level segmentation map are simultaneously colored and rendered to generate a semantically distinguishable color segmentation map. The color segmentation image is semi-transparently overlaid and fused with the original intracavitary cross-sectional image to generate the final real-time diagnostic fusion view, which is then output and displayed.

Images