B ultrasonic auxiliary diagnosis system based on artificial intelligence

Abstract

The invention discloses a B-ultrasonic auxiliary diagnosis system based on artificial intelligence. The B-ultrasonic auxiliary diagnosis system comprises a data acquisition and preprocessing module used for acquiring B-ultrasonic images and performing standardization processing; the physical characteristic extraction module is used for extracting physical characteristics of acoustic impedance, attenuation coefficients, echo intensity and scattering and reflection modes and providing embedded data for the feature fusion module; the feature fusion and representation module is used for deeply combining physical characteristics and spatial features through a multi-modal feature fusion and adaptive graph convolution method to generate high-dimensional feature representation; the deep learning diagnosis model module is used for realizing joint optimization of feature learning and diagnosis tasks on the basis of a multi-task deep neural network framework guided by physical semantics; and the visualization and auxiliary diagnosis module is used for presenting diagnosis results and providing decision support for doctors through thermodynamic diagrams, multi-modal analysis and intelligent explanation. According to the method, the limitation of a traditional deep learning model under the challenges of low contrast, noise interference, data shortage and the like can be solved.

Description

Technical Field

[0001] The present invention relates to the field of medical diagnosis technology, and in particular to an artificial intelligence-based B-ultrasound-assisted diagnosis system. Background Art

[0002] In the field of medical imaging, especially in the application of ultrasound imaging (B-ultrasound) technology, the introduction of deep learning and artificial intelligence has provided strong support for improving diagnostic efficiency and accuracy. However, traditional deep learning methods mainly rely on the spatial features of images, such as pixel values ​​and textures, ignoring the physical properties hidden behind the images, resulting in the model's diagnostic ability being insufficient in complex situations. For example, acoustic impedance is an important physical property of ultrasound propagation in tissues. It describes the reflection and transmission capabilities of tissues to sound waves. Different tissues or lesion areas, such as tumors, fat or muscles, have different reflection intensities to ultrasound due to differences in acoustic impedance. The attenuation coefficient, as the attenuation characteristic of ultrasound propagation in tissues, can reflect the density, structure and possible lesion areas of tissues. The attenuation coefficients of different tissues vary greatly, and the attenuation coefficients of lesion areas (such as tumors) are often different from those of surrounding normal tissues. Therefore, by integrating the attenuation coefficient into the neural network, the model can identify lesion areas such as tumors by learning the attenuation characteristics. Especially in low-contrast images, attenuation information is crucial for assisting in the identification of lesions. Echo intensity is the most direct perceptual feature in ultrasound images. It is a quantitative expression of the reflection intensity of ultrasound waves by tissues. Different tissues will produce different echo intensities due to their different structures. For example, tumor tissues usually have echo intensity patterns different from those of surrounding normal tissues, which provides unique clues for tumor detection. By combining echo intensity, neural networks can learn the echo reflection characteristics of different areas in the image and extract key diagnostic features based on this. Especially in the early stages of tumors, echo intensity provides an important basis for judging the lesion area. In addition, scattering and reflection patterns are key physical properties that describe the changes in ultrasound waves during propagation and reflection in tissues. Scattering patterns reflect the microstructural characteristics of tissues, which usually show scattering characteristics different from those of normal tissues in lesion tissues. Reflection patterns indicate the reflection characteristics of ultrasound waves at the tissue interface. Lesion tissues may cause reflection patterns different from those of surrounding tissues. Incorporating this information into neural networks helps improve the model's understanding of image details. Especially when diagnosing complex lesions, scattering and reflection patterns can serve as supplementary information to improve the ability to identify lesions.

[0003] Therefore, it is extremely important to effectively integrate the four physical properties of acoustic impedance, attenuation coefficient, echo intensity, scattering and reflection pattern into the feature extraction and decision-making process of neural networks. Summary of the invention

[0004] In order to effectively integrate the four physical properties of acoustic impedance, attenuation coefficient, echo intensity, scattering and reflection pattern into the feature extraction and decision-making process of the neural network, the present invention provides an artificial intelligence-based B-ultrasound auxiliary diagnosis system.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] The B-ultrasound auxiliary diagnosis system based on artificial intelligence includes: data acquisition and preprocessing module: used to collect B-ultrasound images and perform standardized processing;

[0007] Physical property extraction module: extracts the physical properties of acoustic impedance, attenuation coefficient, echo intensity, scattering and reflection patterns, and provides embedded data for the feature fusion module;

[0008] Feature fusion and representation module: Through multimodal feature fusion and adaptive graph convolution method, physical characteristics and spatial features are deeply combined to generate high-dimensional feature representation;

[0009] Deep learning diagnosis model module: A multi-task deep neural network framework guided by physical semantics to achieve joint optimization of feature learning and diagnosis tasks;

[0010] Visualization and auxiliary diagnosis module: Through heat maps, multimodal analysis and intelligent interpretation, it presents diagnostic results and provides decision support for doctors.

[0011] Further: the data acquisition and preprocessing module includes:

[0012] Data acquisition submodule: connects with B-ultrasound equipment through communication protocol to obtain images and metadata, providing a basis for physical property calculation;

[0013] Depth and time gain compensation submodule: corrects echo intensity according to pixel depth and ultrasonic attenuation law to balance the depth gain of the image;

[0014] Noise filtering and image enhancement submodule: Gaussian filtering and CLAHE method are used to remove noise and enhance tissue details;

[0015] Format and parameter standardization submodule: unifies pixel resolution, grayscale range and spatial scale, outputs standardized images and metadata, and provides guarantee for subsequent physical property extraction.

[0016] Further: the physical property extraction module includes:

[0017] Acoustic impedance calculation submodule: calculates the acoustic impedance characteristics of different tissues based on echo intensity and marks the areas of significant changes;

[0018] Attenuation coefficient analysis submodule: calculates the attenuation coefficient using the depth-dependent exponential attenuation formula and generates a two-dimensional attenuation distribution map;

[0019] Echo intensity normalization submodule: performs depth and gain compensation on the echo signal to generate a standardized echo intensity map;

[0020] Scattering and reflection pattern analysis submodule: It uses wavelet transform and Fourier transform to extract texture and frequency domain characteristics and identify the scattering and reflection characteristics of tissues.

[0021] Further: the feature fusion and representation module includes:

[0022] Physical and spatial adaptive graph convolution submodule: combines the dynamic calculation of physical weights and spatial weights to achieve collaborative modeling of physical properties and spatial features;

[0023] Multi-scale feature fusion submodule: extracts local and global features through pyramid pooling and multi-scale graph convolution to capture information at different scales;

[0024] Self-supervised learning submodule: introduces the physical property prediction task and optimizes the feature fusion effect through self-supervised loss.

[0025] Further: the deep learning diagnostic model module includes:

[0026] Input and feature fusion submodule: It uses the multi-dimensional physical characteristics and spatial features from the feature fusion module as high-dimensional input to ensure that the physical information effectively guides the diagnosis task;

[0027] Multi-task learning submodule: Improve the diagnostic performance of the model through joint optimization of tasks such as classification, segmentation, and boundary detection;

[0028] Adaptive physical property guidance submodule: adjusts the convolution kernel weights based on physical properties, allocates regional weights through the attention mechanism, and improves the model's ability to detect lesion areas;

[0029] Adaptive attention mechanism submodule: Use spatial physical attention mechanism to give higher attention weight to the key diagnosis areas;

[0030] Output and performance optimization submodule: Output generates classification results, segmentation maps and boundary information.

[0031] Further: the visualization and auxiliary diagnosis module includes:

[0032] Thermal map and semantic fusion: Generate multi-level thermal maps, superimpose the output classification results, segmentation maps and boundary information with physical characteristics, and provide doctors with joint physical semantic visual feedback;

[0033] Interactive clinical decision support: Through dynamic feedback mechanisms, doctors can adjust model results and optimize diagnostic decisions;

[0034] Multimodal analysis and physical-spatial semantic separation: Using independent physical encoders and spatial decoders to separate feature information, providing doctors with independent presentation of physical attributes and spatial distribution;

[0035] Intelligent interpretation and highlighting of suspicious areas: Generate diagnostic reports based on natural language and highlight suspicious areas to enhance the transparency and reliability of diagnosis.

[0036] Compared with the prior art, the present invention has the following technical advances:

[0037] Ultrasound images themselves provide two-dimensional tissue structure information, but it may be difficult to extract key features when the contrast between different tissues or lesion areas is low. By introducing physical properties such as acoustic impedance, attenuation coefficient, echo intensity, scattering and reflection patterns into the neural network, the system can obtain more dimensional diagnostic information. For example, tumors usually show different echo intensity or acoustic impedance from surrounding normal tissues. These differences help to accurately identify lesion areas, thereby greatly improving diagnostic accuracy, especially in the detection of early tumors or small lesions.

[0038] By combining image features with these physical properties, the neural network can capture the underlying regularity and perform reasoning analysis from multiple dimensions, enabling the model to more accurately identify lesion areas and abnormal changes. For example, the relationship between acoustic impedance and echo intensity is very obvious in certain lesion types (such as tumors), and combining these characteristics can effectively enhance the model's recognition ability.

[0039] B-ultrasound images are often affected by noise or have low contrast. Simply relying on image pixel values ​​may cause the model's performance to deteriorate in these situations. Physical properties (such as attenuation coefficient and acoustic impedance) can provide structural information of the image, helping the model to better identify and distinguish important areas, thereby maintaining good diagnostic performance in images with noise or low contrast. For example, the attenuation coefficient responds significantly differently to different tissues. Even if the image itself has low contrast, the model can still identify diseased tissues based on the attenuation characteristics.

[0040] Different patients have different body structures and tissue structures, but physical properties have a certain universality that can help the model adapt to different patient groups. By integrating these physical properties, the model not only relies on simple image information, but also relies on objective physical properties to enhance adaptability to different patients, thereby improving the generalization ability of the system.

[0041] By combining physical properties with the output of deep learning models, doctors can not only see the prediction results, but also understand the basis for the model's decision. For example, combined with acoustic impedance information, the model can clearly indicate why a certain image area is identified as a tumor area. This interpretability can help doctors understand the decision-making process of the model and provide more clinical decision support. This physical property-guided decision-making model not only provides intuitive image analysis results during diagnosis, but also provides intelligent feedback to doctors based on physical properties. For example, if the echo intensity shows that a certain area is abnormal, the system will automatically issue an alarm to alert the doctor. During auxiliary diagnosis, the provision of physical property information can help doctors take more targeted follow-up examination measures to avoid misdiagnosis.

[0042] Combining the attenuation coefficient and scattering pattern can accurately identify the lesion area, especially when the tissue boundary is not obvious. These physical characteristics can help the neural network understand the subtle differences between lesions and normal tissues. Through this additional characteristic information, the system can not only improve the classification accuracy of tumors, but also segment tumor boundaries more accurately, further enhancing the operability and reliability of the model in different clinical applications. In some cases, physical characteristics will show different performances at different scales. Integrating this information allows the neural network to better process information at different scales in the image. For example, acoustic impedance information can help locate the position of the tumor, while the attenuation coefficient can provide more detailed information about the density and morphology of the tumor, which can enable the system to have stronger lesion localization capabilities at multiple scales.

[0043] Physical properties can be provided to the model as auxiliary information, reducing the reliance on large amounts of labeled data. Traditional deep learning models often require a large amount of labeled training data to learn effectively, but after combining physical properties, the model can accelerate the learning process through these biologically and physically meaningful features, thereby improving training efficiency, especially when labeled data is scarce.

[0044] By taking physical information such as acoustic impedance and attenuation coefficient as the input of the network, the model can be exposed to more low-dimensional information during the training process, thereby reducing the overfitting phenomenon that may occur in the training process that relies solely on image pixels. Especially when the data set is small, this physics-guided training method can improve the stability and reliability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0046] In the attached picture:

[0047] Figure 1 It is a system structure diagram of the present invention. DETAILED DESCRIPTION

[0048] The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, the present invention discloses an artificial intelligence-based B-ultrasound assisted diagnosis system, including: a data acquisition and preprocessing module, a physical property extraction module, a feature fusion and representation module, a deep learning diagnosis model module and a visualization and auxiliary diagnosis module.

[0050] Specifically, the data acquisition and preprocessing module is used to realize the acquisition, processing and standardization of B-ultrasound image data, and provide high-quality input for the subsequent physical characteristics extraction module. Its implementation is divided into the following sub-modules:

[0051] 1. Data acquisition submodule

[0052] The B-ultrasound device is connected through a communication protocol (such as DICOM, HL7, USB or Ethernet interface), the original B-ultrasound images or video streams are collected, and then metadata is extracted, the accompanying information of the B-ultrasound image (such as depth, probe frequency, gain parameters) is analyzed and stored for subsequent physical property calculations.

[0053] 2. Depth and time gain compensation submodule

[0054] According to the depth position (z) of the image pixel and the ultrasonic attenuation law, the echo intensity at different depths is corrected:

[0055]

[0056] Among them, I raw represents the raw pixel intensity, α represents the average attenuation coefficient of the tissue (obtained from the probe metadata), and z represents the pixel depth.

[0057] The time gain compensation (TGC) curve is automatically adjusted to balance the echo intensity of tissues at different depths.

[0058] 3. Noise filtering and image enhancement submodule

[0059] Use Gaussian filter to remove random noise and protect edge information. Gaussian filter formula:

[0060]

[0061] Here, σ controls the smoothness of the filter.

[0062] Contrast limited adaptive histogram equalization (CLAHE) is used to enhance tissue details and avoid over-enhancement.

[0063] 4. Format and parameter standardization submodule

[0064] The pixel resolution, intensity range (usually 0255) and physical space scale (such as mm / pixel) of the image are unified, and the image is adjusted to a fixed resolution (such as 512×512) using an interpolation algorithm (such as bilinear interpolation).

[0065] Finally, clean and standardized B-ultrasound image data and its related metadata (such as depth and probe frequency) are output to provide input guarantee for the physical property extraction module.

[0066] Specifically, the physical property extraction module calculates and analyzes the standardized B-ultrasound image data to extract four physical properties closely related to diagnosis, namely acoustic impedance, attenuation coefficient, echo intensity, and scattering and reflection patterns, and provides basic data for embedding physical properties for the feature fusion and representation module. Its implementation is divided into the following sub-modules:

[0067] 1. Acoustic impedance calculation submodule

[0068] The calculation of acoustic impedance is based on the change in echo intensity and the interface characteristics of the tissue, using the following formula:

[0069]

[0070] Among them, I reflected and I incident denote the reflected and incident wave intensities, Z ref Represents the reference acoustic impedance (obtained from the probe's metadata).

[0071] Mark areas with significant acoustic impedance changes in the image and locate high acoustic impedance boundaries based on gradient calculation:

[0072]

[0073] Among them, G(x,y) represents the gradient strength, and They are the grayscale changes of the image in the x and y directions respectively.

[0074] 2. Attenuation coefficient analysis submodule

[0075] The attenuation coefficient is calculated based on the exponential attenuation law of the intensity with depth during ultrasonic propagation, using the following formula:

[0076]

[0077] Wherein, α represents the attenuation coefficient, I(z) represents the echo intensity at depth z, and I0 represents the initial intensity.

[0078] The attenuation distribution at different depths is calculated layer by layer and a two-dimensional attenuation coefficient map is generated.

[0079] 3. Echo intensity normalization submodule

[0080] The echo intensity of the target area is normalized after depth and gain compensation, and is calculated as follows:

[0081]

[0082] Among them, I compensated Indicates the echo intensity after depth and time gain compensation.

[0083] Output normalized echo intensity distribution map to provide standardized input for subsequent model feature extraction.

[0084] 4. Scattering and reflection pattern analysis submodule

[0085] Texture details are extracted through scattering characteristic analysis algorithms (such as wavelet transform), scattering patterns (coarse scattering, fine scattering, etc.) are identified and converted into classification labels.

[0086] Analyze the directionality and symmetry of the echo and calculate the frequency domain characteristics of the reflection pattern based on Fourier transform (FFT):

[0087]

[0088] Among them, F(u,v) represents the frequency domain intensity, and f(x,y) is the spatial domain pixel value.

[0089] The final output generates a physical property map including acoustic impedance, attenuation coefficient, normalized echo intensity, and scattering and reflection pattern distribution, providing multi-dimensional feature input for the feature fusion and representation module.

[0090] Specifically, the feature fusion and representation module performs multi-scale and multi-modal fusion of physical properties (acoustic impedance, attenuation coefficient, echo intensity, scattering and reflection patterns) to achieve more efficient feature representation, thereby improving the accuracy and reliability of diagnosis.

[0091] 1. Physical and spatial adaptive graph convolution submodule

[0092] Traditional graph convolutional networks (GCNs) usually rely on graph-structured data inputs. In B-ultrasound images, physical properties (such as acoustic impedance, attenuation coefficient, etc.) are often localized, but at the same time, there are complex correlations between these properties. Therefore, simple local connection patterns cannot fully capture the dependencies between properties. To solve this problem, this embodiment designs a physical space adaptive graph convolutional network, which achieves information fusion by considering adaptive weights in both spatial (pixel neighborhood relationships) and physical (correlation between physical properties) aspects. The adaptive mechanism of the graph convolution layer introduces dynamically calculated physical weights and spatial weights for each node, so that information transmission between nodes not only relies on the traditional graph structure, but also combines the importance of physical properties. The physical weight calculation formula is:

[0093]

[0094] Among them, I phys (x,y) represents the value of a physical property at position (x,y) (such as acoustic impedance or attenuation coefficient), and Z max represents the maximum value of the physical property, and σ represents the activation function (such as the Sigmoid function), which is used to standardize the weights.

[0095] Spatial weight calculation formula:

[0096]

[0097] Among them, G(x,y) is the gradient intensity at the pixel position (x,y), which indicates the severity of spatial changes. max is the maximum gradient of the image.

[0098] The specific implementation logic is to perform convolution operations on each layer. First, the weight W based on physical characteristics is calculated. phys and the spatial weight W spatital , and then use these adaptive weights to adjust the information transmission intensity between each node (pixel), combining physical characteristics and spatial features to design the following weighted graph convolution operation:

[0099]

[0100] in, represents the feature representation of node v at the kth layer, N(v) represents the neighborhood of node v, and W phys (v,u) and W spatital (v,u) are the physical weight and spatial weight respectively, and σ is the activation function.

[0101] 2. Multi-scale feature fusion submodule

[0102] The physical characteristics in B-ultrasound images are spatially distributed in a hierarchical manner. Therefore, information at different scales needs to be considered during feature fusion. In this section, this embodiment designs a multi-scale feature fusion module to extract features of different scales through pyramid pooling and multi-scale graph convolution, and fuse them into a unified representation. The pyramid pooling formula is:

[0103] F pyranind =[F1,F2,F3,F4]

[0104] Among them, F1, F2, F3, and F4 represent the feature maps after pooling at different scales (such as 1x1, 2x2, 4x4, and 8x8). Through this multi-scale pooling method, local and global information at different scales can be captured.

[0105] Multi-scale graph convolution formula:

[0106]

[0107] Among them, W multi Represents multi-scale weights, and performs convolution operations on the graph structure of each scale, so that the features of each scale are optimized at a specific level.

[0108] 3. Self-supervised learning submodule

[0109] In order to further improve the feature fusion effect and ensure that the physical characteristics are effectively used in the learning process, this embodiment designs a self-supervised learning mechanism guided by physical characteristics. This mechanism guides the model to learn the intrinsic relationship between physical characteristics more accurately through self-supervised learning tasks (such as predicting the relationship between physical characteristics and comparing the classification differences of different physical characteristic regions). The self-supervised task formula is:

[0110]

[0111] Among them, f θ (x i ) represents the features predicted by the network, f phys Represents the expected output calculated based on physical properties (such as the predicted value of acoustic impedance), and the self-supervised loss function optimizes the model by minimizing the difference between the network output and the physical property output.

[0112] The feature map finally output by this module not only has the efficient fusion of spatial and physical property information, but also contains multi-scale feature representation, providing feature input with deep semantic information for subsequent deep learning diagnostic model modules (such as classification or regression tasks).

[0113] Specifically, the deep learning diagnosis model module is designed based on a multi-task deep neural network framework guided by physical semantics. In this module, the present embodiment uses the multi-dimensional physical characteristics and image features obtained from the feature fusion and representation module as input, and uses an innovative network architecture to perform comprehensive feature learning and automatic diagnosis:

[0114] 1. Input and feature fusion submodule

[0115] In this module, the physical characteristics output by the feature fusion and representation module and the spatial characteristics of the B-ultrasound image are used as the input of the network. Specifically, these features have been processed into high-dimensional feature maps in the feature fusion module, which include the effective fusion of physical information and spatial information. The physical characteristic image and the spatial feature image output by the feature fusion and representation module are input into the deep learning network in a weighted manner to ensure that the physical characteristics can effectively guide the model's discrimination and prediction during the feature learning process.

[0116] 2. Multi-task learning submodule

[0117] To further improve diagnostic performance, the network adopts a multi-task learning strategy and is designed as a joint learning framework, allowing the model to simultaneously process multiple related tasks in the same network architecture, including:

[0118] Classification tasks: For example, determining whether there is a tumor in an ultrasound image, or the type of tumor (benign, malignant, etc.).

[0119] Segmentation task: Based on the input B-ultrasound image and physical characteristics, output the pixel-level segmentation map of the tumor or lesion area.

[0120] Boundary detection task: Identify the boundaries of the lesion area so that doctors can perform a more accurate analysis of the tumor boundaries.

[0121] By sharing the same feature space, the synergy between multiple tasks enables the model to learn richer and more generalizable features from different tasks.

[0122] The joint loss function is designed as:

[0123] L total =α1L class +α2L seg +α3L boundary

[0124] Among them, L class is the classification loss, L seg is the segmentation loss, L boundary is the boundary detection loss, α1, α2, α3 are the weight coefficients of the task, which are adjusted according to the importance of different tasks.

[0125] 3. Adaptive physical property guidance submodule

[0126] In order to ensure that the model can make full use of the physical property information, an adaptive physical property guidance submodule is added to the deep learning network. This module optimizes the feature extraction process by adjusting the weights and activations of the convolutional layer according to the physical property information such as acoustic impedance and attenuation coefficient.

[0127] Specifically, in each layer of convolution, the physical property information is not only input into the convolution operation as an auxiliary feature map, but also affects the generation of the feature map through the weighted convolution kernel, thereby improving the sensitivity to specific lesion areas. The calculation method of each layer of convolution is:

[0128]

[0129] in, is the physically guided feature map, W phys (v,u) is the physical property weighted convolution kernel, I phys (v) is the value of the physical property at position v, and β is a hyperparameter that adjusts the influence of physical information.

[0130] 4. Adaptive Attention Mechanism Submodule

[0131] In order to enable the network to more intelligently select key areas in the image, the network also introduces an adaptive attention mechanism. This module automatically assigns different attention weights to each area based on the physical characteristics and spatial features of each area. The calculation formula of the attention mechanism is as follows:

[0132] A spatital (x,y)=σ(W·F(x,y)+b)

[0133] Among them, A spatital (x, y) is the spatial attention weight, σ is the Sigmoid activation function, W is the weight matrix, F(x, y) is the feature representation of the image at position (x, y), and b is the bias term.

[0134] This mechanism ensures that the network can assign appropriate weights to different areas of the image, thereby improving the accurate detection and classification of lesion areas.

[0135] 5. Output and performance optimization submodule

[0136] The output of the deep learning diagnostic model module includes various forms of information:

[0137] Classification results: Determine whether there is a lesion based on the features in the image, such as the presence and type of a tumor.

[0138] Segmentation map: Generate pixel-level segmentation maps for tumor or lesion areas, providing doctors with a clear view of the lesion area.

[0139] Boundary information: Provides tumor or lesion boundary detection results to help doctors accurately locate the tumor boundary.

[0140] Through the above modules, the deep learning diagnostic model module not only fully combines the physical property information from the feature fusion and representation module, but also improves the model's ability to understand B-ultrasound images through innovative adaptive guidance mechanisms, multi-task learning and attention mechanisms, enabling the model to perform more accurate automated diagnoses and provide clinicians with more reliable auxiliary decision support. It can effectively combine physical properties with deep learning, give full play to the diagnostic potential of B-ultrasound images, and coordinately optimize in multiple diagnostic tasks to ensure efficient and accurate diagnostic results.

[0141] Specifically, the implementation of the visualization and auxiliary diagnosis module includes:

[0142] In the deep learning diagnosis model module, this embodiment has obtained output results that have been optimized through physical property guidance, multi-task learning, and adaptive attention mechanisms, including classification results, segmentation maps, and boundary information. Next, the task of the visualization and auxiliary diagnosis module is to present these output results to medical personnel in an intuitive and easy-to-understand manner, and combine advanced algorithms to provide intelligent assistance for clinical decision-making.

[0143] 1. Multi-level visualization and semantic fusion sub-module based on heat map

[0144] In visualization, one of the core challenges is how to effectively present the internal features and final output results of the deep learning model to doctors to help them understand the decision-making process of the model. Therefore, this embodiment designs a multi-level visualization and semantic fusion method based on heat maps, which combines the output of the deep learning model with the physical characteristics and spatial features in the image to generate instructive visual feedback.

[0145] The specific implementation process is as follows:

[0146] Heat map generation: The output of the model in the deep learning diagnosis model module, especially the regional importance map generated by the adaptive attention mechanism, is used to generate a heat map to highlight the lesion area. The heat map is not only annotated based on the classification results, but also combined with the tumor boundary detection results to mark the key areas that the model focuses on.

[0147] Physical property superposition: Map physical properties such as acoustic impedance and attenuation coefficient to the heat map to generate a physical semantic joint heat map. For example, the acoustic impedance and attenuation coefficient of the tumor area are combined with the grayscale information in the image, allowing doctors to directly see how the physical properties interact with the image features. The generated joint heat map is in the form of:

[0148] H fusion (x,y)=α1H image (x,y)+α2H phys (x,y)

[0149] Among them, H image (x, y) represents the heat map generated based on the image content, H phys (x, y) represents the heat map after the physical property information is superimposed, and α1 and α2 are weight coefficients, which are adjusted according to the visual importance of the heat map.

[0150] Multi-level visualization: Multi-level visualization is used for different diagnostic tasks. For example, for segmentation tasks, the outline of the tumor area and the segmentation results are displayed. For classification tasks, clear labels and diagnostic conclusions are provided. For each area, the heat map can show the relationship between its physical properties and the model's attention.

[0151] 2. Interactive Clinical Decision Support System

[0152] In order to make the visualization effect and auxiliary diagnosis function more practical, this embodiment designs an interactive clinical decision support system, which not only supports traditional image viewing, but also provides intelligent auxiliary analysis, doubt prompts and decision suggestions. The specific implementation method is as follows:

[0153] Decision recommendation: Combining the output of the deep learning model (such as segmentation map, boundary information, etc.) with the visualization results, ICDSS automatically provides doctors with some potential diagnostic suggestions. For example, when the model predicts that a certain area may be a tumor, the system will automatically prompt whether further examination is needed or different diagnostic methods are used to confirm the results.

[0154] Dynamic feedback and adjustment: Through the loop feedback mechanism, doctors can adjust the model results during the diagnosis process (for example, adjust the weight coefficient of the heat map) and observe its impact on the model prediction. In this way, doctors can improve together with the model and improve the reliability of diagnosis.

[0155] 3. Multimodal analysis combining physical and spatial characteristics

[0156] In view of the complexity and multimodal information of B-ultrasound images, this embodiment proposes a multimodal analysis algorithm that combines physical characteristics and spatial features, so that the visualization result is not limited to a single image information, but also takes into account the physical properties in the image. The specific implementation includes the following steps:

[0157] Fusion of physical properties and image features: Through multimodal data fusion, information such as acoustic impedance, echo intensity, attenuation coefficient, etc. are combined with texture and spatial features in the image to further improve the accuracy of diagnosis. In this fusion method, the system can not only display the location of the lesion, but also the physical properties of the lesion area (such as its performance in different acoustic impedance areas).

[0158] Physical space semantic separation: This embodiment designs a physical space semantic separation model that can distinguish between physical properties and spatial features of images and present them to doctors separately, so as to better analyze the relationship between the physical properties and spatial distribution of lesions. Specifically, it includes a collaborative architecture of physical property encoder and spatial semantic decoder, combined with a multimodal feature alignment mechanism to distinguish and independently model the physical properties and spatial features of ultrasound images, while providing interactive physical and spatial feature visualization support.

[0159] 1. Physical property encoder

[0160] Input: Physical properties extracted from previous steps (such as acoustic impedance, attenuation coefficient, echo intensity, scattering and reflection patterns).

[0161] Task: Extract high-dimensional physical features through dedicated network branches and describe the feature expressions of different physical properties in a high-level feature space.

[0162] Output: High-dimensional tensor of physical features Among them C p , H p , W p Represent the number of feature channels, height and width respectively.

[0163] 2. Spatial semantic decoder

[0164] Input: raw pixel data of B-ultrasound images and their derived spatial texture features.

[0165] Task: Build a spatial feature representation through convolutional neural network (CNN) layers and generate a feature tensor that describes the spatial distribution and semantic relationships of the image.

[0166] Output: High-dimensional tensor of spatial features Also includes multiple layers of feature resolution.

[0167] 3. Multimodal feature alignment mechanism

[0168] Input: F p and F s .

[0169] Task: Align physical features and spatial features across domain semantics through the attention mechanism to generate a joint feature representation F c, while ensuring that the two can be displayed independently.

[0170] Output: Independently visualized physical property distribution and spatial feature distribution, as well as joint decision results.

[0171] The specific implementation process is:

[0172] The feature extraction of physical properties is performed using a transformer-based structure, which can capture the nonlinear dependencies between different physical properties:

[0173]

[0174] Among them, X p Input matrix for physical properties, N p is the number of features, d p is the dimension of each feature, and the correlation between features is calculated through the self-attention mechanism:

[0175]

[0176] Among them, Q, K, and V are query, key, and value matrices respectively.

[0177] Final output F p Includes multi-level physical feature representation to capture the complex characteristics of different physical properties.

[0178] The spatial features are extracted through the encoder part of a deep convolutional network (such as ResNet or U-Net) to extract the initial representation:

[0179] F s =CNN(X s ),X s ∈R H×W×C

[0180] Among them, X s is the input image, H, W, and C represent the height, width, and number of channels of the image respectively.

[0181] The decoder part generates a high-resolution spatial distribution map through deconvolution or upsampling layers:

[0182]

[0183] here, Represents the spatial feature output of the lth layer.

[0184] Through the cross-modal attention mechanism, physical and spatial features are semantically aligned:

[0185] F c =Cross-Attention(Fp ,F s )

[0186] Among them, the cross attention mechanism generates an aligned joint feature representation by sharing the query matrix:

[0187] Corss-Attention(Q,K p ,K s ,V p ,V s )

[0188] The outputs include independent physical property distribution maps (such as the two-dimensional distribution of acoustic impedance) and spatial feature distribution maps (such as the spatial location of the lesion area), as well as a joint decision feature map.

[0189] The physical space semantic separation model of this embodiment uses a modular network architecture to explicitly separate physical properties and spatial features and realize independent visualization of the two. This not only helps doctors analyze the physical properties of lesions, but also provides a clear spatial distribution of lesions. It adopts a cross-modal attention mechanism and realizes the alignment and interaction of physical properties and spatial semantics in multiple dimensions for the first time, so that physical and spatial features can be separated independently and work together in joint decision-making. Therefore, it can not only accurately model the distribution of physical properties and spatial features, but also provide doctors with a multi-dimensional perspective for diagnosing lesions, making the artificial intelligence-based B-ultrasound assisted diagnosis system more intelligent and applicable.

[0190] 4. Doubt Enhancement and Intelligent Interpretation

[0191] In order to help doctors better understand the decision-making process of the model, this embodiment also introduces the function of doubt enhancement and intelligent explanation. This function improves the intuitiveness of visualization in the following ways:

[0192] Highlighting of suspicious points: Based on the diagnostic conclusions output by the model (such as tumor classification), the system automatically highlights the areas in the image that are related to the diagnostic conclusions, and explains the particularity of the area through physical property annotations.

[0193] Intelligent diagnostic explanation: Combined with natural language processing technology, the system generates relevant diagnostic reports based on the model output and thermal map, automatically providing doctors with detailed diagnostic explanations. For example: "This area has high acoustic impedance and attenuation coefficient, which may indicate that the tumor tissue has high hardness."

[0194] Through the above technical design, the visualization and auxiliary diagnosis module can combine the diagnostic results, image features and physical properties of the deep learning model to provide efficient and intuitive auxiliary diagnosis functions. The combination of multi-level visualization, interactive feedback mechanism, multimodal analysis algorithm and intelligent interpretation function not only improves doctors' understanding of B-ultrasound images, but also helps clinicians make more accurate and reliable diagnostic decisions through intelligent decision support.

[0195] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of protection of the claims of the present invention.

Claims

1. The B-ultrasound-assisted diagnosis system based on artificial intelligence is characterized by: include: Data acquisition and preprocessing module: used to acquire B-ultrasound images and perform standardized processing; Physical property extraction module: extracts the physical properties of acoustic impedance, attenuation coefficient, echo intensity, scattering and reflection patterns, and provides embedded data for the feature fusion module; Feature fusion and representation module: Through multimodal feature fusion and adaptive graph convolution method, physical characteristics and spatial features are deeply combined to generate high-dimensional feature representation; Deep learning diagnosis model module: A multi-task deep neural network framework guided by physical semantics to achieve joint optimization of feature learning and diagnosis tasks; Visualization and auxiliary diagnosis module: Through heat maps, multimodal analysis and intelligent interpretation, it presents diagnostic results and provides decision support for doctors.

2. The B-ultrasound-assisted diagnosis system based on artificial intelligence according to claim 1 is characterized in that: The data acquisition and preprocessing module includes: Data acquisition submodule: connects with B-ultrasound equipment through communication protocol to obtain images and metadata, providing a basis for physical property calculation; Depth and time gain compensation submodule: corrects echo intensity according to pixel depth and ultrasonic attenuation law to balance the depth gain of the image; Noise filtering and image enhancement submodule: Gaussian filtering and CLAHE method are used to remove noise and enhance tissue details; Format and parameter standardization submodule: unifies pixel resolution, grayscale range and spatial scale, outputs standardized images and metadata, and provides guarantee for subsequent physical property extraction.

3. The B-ultrasound-assisted diagnosis system based on artificial intelligence according to claim 2 is characterized in that: The physical property extraction module comprises: Acoustic impedance calculation submodule: calculates the acoustic impedance characteristics of different tissues based on echo intensity and marks the areas of significant changes; Attenuation coefficient analysis submodule: calculates the attenuation coefficient using the depth-dependent exponential attenuation formula and generates a two-dimensional attenuation distribution map; Echo intensity normalization submodule: performs depth and gain compensation on the echo signal to generate a standardized echo intensity map; Scattering and reflection pattern analysis submodule: It uses wavelet transform and Fourier transform to extract texture and frequency domain characteristics and identify the scattering and reflection characteristics of tissues.

4. The B-ultrasound-assisted diagnosis system based on artificial intelligence according to claim 3 is characterized in that: The feature fusion and representation module includes: Physical and spatial adaptive graph convolution submodule: combines the dynamic calculation of physical weights and spatial weights to achieve collaborative modeling of physical properties and spatial features; Multi-scale feature fusion submodule: extracts local and global features through pyramid pooling and multi-scale graph convolution to capture information at different scales; Self-supervised learning submodule: introduces the physical property prediction task and optimizes the feature fusion effect through self-supervised loss.

5. The B-ultrasound-assisted diagnosis system based on artificial intelligence according to claim 4 is characterized in that: The deep learning diagnostic model module includes: Input and feature fusion submodule: It uses the multi-dimensional physical characteristics and spatial features from the feature fusion module as high-dimensional input to ensure that the physical information effectively guides the diagnosis task; Multi-task learning submodule: Improve the diagnostic performance of the model through joint optimization of tasks such as classification, segmentation, and boundary detection; Adaptive physical property guidance submodule: adjusts the convolution kernel weights based on physical properties, allocates regional weights through the attention mechanism, and improves the model's ability to detect lesion areas; Adaptive attention mechanism submodule: Use spatial physical attention mechanism to give higher attention weight to the key diagnosis areas; Output and performance optimization submodule: Output generates classification results, segmentation maps and boundary information.

6. The B-ultrasound-assisted diagnosis system based on artificial intelligence according to claim 5 is characterized in that: The visualization and auxiliary diagnosis module includes: Thermal map and semantic fusion: Generate multi-level thermal maps, superimpose the output classification results, segmentation maps and boundary information with physical characteristics, and provide doctors with joint physical semantic visual feedback; Interactive clinical decision support: Through dynamic feedback mechanisms, doctors can adjust model results and optimize diagnostic decisions; Multimodal analysis and physical-spatial semantic separation: Using independent physical encoders and spatial decoders to separate feature information, providing doctors with independent presentation of physical attributes and spatial distribution; Intelligent interpretation and highlighting of suspicious areas: Generate diagnostic reports based on natural language and highlight suspicious areas to enhance the transparency and reliability of diagnosis.

Images