Interpretable recognition system based on multi-scale probability mapping morphological feature quantization

Abstract

The invention relates to an interpretability recognition system based on multi-scale probability mapping morphological feature quantification, and aims to solve the problems that in the prior art, objective quantification information basis is lacked for nodule diagnosis, and ambiguous information and diagnosis errors occur during consultation communication due to semantic fuzziness of qualitative concept factors. The system performs lesion contour extraction, uncertain error estimation, morphological feature quantification and lesion interpretability identification through multi-scale feature distribution learning, effectively realizes automation and standardization of lesion quantitative analysis in an ultrasonic image, improves lesion identification accuracy and interpretability, and improves lesion identification accuracy and accuracy. The method has important application value for quantitatively analyzing 2D ultrasonic images and assisting clinicians in formulating accurate diagnosis and treatment plans. Furthermore, according to the system, a mask convolution guided self-supervised learning model based on hybrid self-coding is designed in model training learning, so that the workload of manual labeling is reduced, and the accurate expression of ultrasonic focus morphological characteristics is improved.

Description

Technical Field

[0001] This case involves technical fields such as computer vision and artificial intelligence, medical image processing, and in particular, an explainable recognition system based on multi-scale probability mapping and morphological feature quantification. Background Art

[0002] In medical image analysis, doctors often perform simple positioning and scale measurement of the region of interest. Since the complexity of morphological features is high and quantitative analysis is difficult, doctors often use simple concept descriptors to qualitatively describe the morphological feature factors of the lesions. Although this method has improved the interpretability of medical image diagnostic analysis to a certain extent, due to the lack of objective quantitative information basis and the semantic ambiguity of qualitative concept factors, it is easy to cause ambiguous information and diagnostic errors during consultation and communication. Specifically, due to differences in personal experience, different doctors have different judgment bases on whether the edge feature factors of ultrasound nodules are clear or irregular, which leads to some deviations in their understanding of the diagnostic basis, accompanied by serious problems of underdiagnosis or overdiagnosis. The quantitative methods of conventional image processing are mainly used in the absence of labels, feature information with prior guidance or significant rule constraints, and conventional medical image processing methods can be used to extract the corresponding feature information, such as intensity, texture, shape and other information. In particular, in the quantitative analysis of ultrasound nodules, the image quality of ultrasound imaging is different due to individual differences, making it difficult to accurately quantify the morphological feature factors of nodules. Summary of the invention

[0003] The purpose of this case is to propose an interpretable identification system based on multi-scale probability mapping and morphological feature quantification through this study, including lesion contour extraction, uncertainty error estimation, morphological feature quantification and interpretable lesion identification, effectively realizing the automation and standardization of lesion quantitative analysis in ultrasound images, further improving the accuracy and interpretability of lesion identification, and having important application value for quantitative analysis of 2D ultrasound images and assisting clinicians in formulating accurate diagnosis and treatment plans.

[0004] In order to achieve the above technical objectives, the technical solution of this case is as follows.

[0005] In the first aspect, this case proposes an interpretable identification system based on multi-scale probability mapping morphological feature quantification, the system includes an uncertainty error estimation module, a morphological feature quantification estimation module, and an interpretable identification module; the uncertainty error estimation module is configured to learn the multi-scale feature distribution of the lesion area in the ultrasound nodule image, obtain the contour segmentation probability mapping map of N lesion areas, and then obtain the lesion contour M p and the contour uncertainty error M e , N is a set value; the morphological feature quantitative estimation module is configured based on the lesion contour Mp and the contour uncertainty error M e , obtain the quantitative representation of the morphological characteristics of the nodule lesion; the interpretability recognition module is configured to firstly calculate the uncertainty error M based on the contour e The lesion area is calculated to obtain the image features of the lesion area, and then the lesion attributes are visualized based on the quantitative representation of the image features and the morphological features of the nodule lesion.

[0006] In one embodiment of the above technical solution, the uncertainty error estimation module includes a preset hybrid encoder and a multi-scale decoder; the preset hybrid encoder has multiple first feature encoding layers, encodes the original image and performs splicing and fusion of encoded features, and the multi-scale decoder has multiple second feature decoding layers, decodes and fuses the spliced ​​and fused features to obtain fusion features of a probability map for predicting lesion area segmentation; wherein the number of second feature encoding layers is 1 less than the number of first feature decoding layers, and each second feature layer is jump-connected to a first feature layer whose scale is greater than or equal to that layer; a Monte Carlo pruning layer is provided in the preset hybrid encoder and the multi-scale decoder to perform multi-scale feature distribution learning.

[0007] In one implementation of the above technical solution, the parameters of the preset hybrid encoder are obtained through mask learning pre-training, and the steps include: recording the preset hybrid encoder as E ct , constructing a hybrid encoder E ct A hybrid encoder G with the same structure EM , and construct a Transformer-based decoder G DM ; Mask the original image according to the set mask matrix and mask rate to obtain the convolution mask strategy and mask image; in pre-training, the hybrid encoder G EM Based on the convolution mask strategy, the mask image is encoded and the encoded features are concatenated and fused. The decoder G DM The spliced ​​fusion coding features are decoded to achieve the reconstruction of the original image. The loss function used is the image structure similarity metric to regularize the constrained mean square error loss function. After pre-training, the hybrid encoder G EM The parameters of the hybrid encoder E ct Parameters.

[0008] In one implementation of the above technical solution, the preset hybrid encoder has five first feature layers, which are divided into 3 convolutional layers and 2 Transformer layers, and Monte Carlo pruning layers are set in the 3rd to 5th layers.

[0009] In one implementation of the above technical solution, the multi-scale decoder is configured to have four second feature layers {U4, U3, U2, U1}, and a Monte Carlo pruning layer is set in U4 and U3. The four second feature layers {U4, U3, U2, U1} sequentially output decoding features as {F u4 , F u3 , F u2 , F u1}, where feature F u2 Based on feature F u3 The up-sampled characteristic sample and characteristic F u4 Decoding fusion, feature F u1 Based on feature F u2 The up-sampled characteristic sample and characteristic F u4 , Feature F u3 Decoding fusion.

[0010] In one embodiment of the above technical solution, the lesion contour M p and the contour uncertainty error M e The acquisition step includes: recording the probability of the i-th feedforward predicted ultrasound lesion suspicious area mapping as p mi , then the lesion contour is M p , N is the total number of times; the uncertainty cutoff value is recorded as Δp0, then the uncertainty error of the contour is M e , δ 2 (*) represents the variance calculation function of the probability map.

[0011] In one implementation of the above technical solution, the quantification of lesion morphological characteristics includes the quantification of the scale, boundary, echo quality, composition and calcification of the lesion area.

[0012] In one implementation of the above technical solution, the loss function is β is the balance factor, L is the mean square error loss function; μ is the pixel mean of the ultrasound nodule image block, is the pixel mean of the predicted ultrasound nodule image block; γ represents the covariance between image blocks, c b and c c They represent brightness difference deviation and contrast difference deviation respectively.

[0013] On the second aspect, the present case proposes a computer-readable storage medium storing a computer program that can be loaded by a processor and executed by any of the above-mentioned systems.

[0014] The beneficial technical effects of this case are as follows: (1) By providing an objective and quantitative way to explain the decision-making process of the deep learning model through quantitative analysis of nodule morphological characteristics, the credibility of the model is further increased, and doctors are encouraged to effectively use quantitative data in diagnostic analysis, thereby improving the accuracy and reliability of ultrasound lesion identification and providing data support with reference value for surgical resection or ablation treatment plans. (2) By reducing the heavy workload of manual labeling in deep learning through masked self-supervised learning, the problem of difficulty in identifying nodule lesions caused by unclear image quality and noise interference caused by individual differences in ultrasound nodule quantitative analysis can be overcome, and accurate representation of images and nodule morphological characteristics can be achieved to meet the needs of clinical auxiliary diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0016] Figure 1 , one Schematic diagram of the structure of a feature characterization module based on hybrid mask self-encoding in an implementation manner.

[0017] Figure 2 , one Schematic diagram of contour uncertainty error estimation using a hybrid encoder and a multi-scale decoder in an implementation manner.

[0018] Figure 3 , one Schematic diagram of quantitative calculation of lesion contour features guided by prior knowledge in an implementation manner.

[0019] Figure 4 , one Schematic diagram of the interpretable recognition process based on multi-scale probability mapping and morphological feature quantification in an implementation manner.

[0020] Figure 5 , one Schematic diagram of the visualization effect of thyroid ultrasound nodule lesion recognition results and nodule contour morphological characteristics in this implementation method.

[0021] Figure 6 , one Schematic diagram of the visualization effect of thyroid ultrasound nodule lesion recognition results and internal morphological characteristics of nodules in this implementation method. DETAILED DESCRIPTION

[0022] Based on the medical image analysis process and ultrasound diagnosis guideline knowledge, this case uses image processing technologies such as self-supervised learning, contour segmentation uncertainty assessment, and morphological feature characterization quantification to establish an interpretable recognition system for multi-scale probability mapping morphological feature quantification, which has important application value for quantitative analysis of 2D ultrasound images and assisting clinicians in formulating accurate diagnosis and treatment plans. The interpretability refers to the feature alignment and semantic fusion of the quantitative representation of the lesion morphological features with the image information, and the visual identification of the lesion attributes. The quantitative representation of the lesion morphological features is a quantitative representation of the scale, boundary, echo, composition, and calcification of the lesion area.

[0023] The following will be combined with the accompanying drawings to clearly and completely describe how to implement the technical solution of this case. Obviously, the described implementation method is only a part of the implementation method of this case, not all of the implementation methods. Based on the implementation method in this case, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0024] (I) Hybrid Encoder G EM

[0025] In recent years, quantification methods based on deep learning are mainly used in tasks with labels, where manual quantification and estimation are difficult or inefficient. For some tasks where manual quantification and estimation are difficult or inefficient, such as prediction of complex nonlinear relationships or trend prediction tasks, deep learning models can better handle these problems because they can automatically learn and extract complex correlation features, thereby improving the accuracy of continuous value prediction or estimation. The most common quantitative prediction methods include convolutional neural networks (CNN), recurrent neural networks (RNN) or long short-term memory networks (LSTM), etc., which are used to extract features from time series data and make predictions, or to extract key markers, contours of regions of interest, and other information. However, such methods require a large amount of annotated data to drive, and still cannot achieve accurate quantitative representation of morphological features, and thus cannot meet the needs of clinical auxiliary diagnosis. The quantitative representation of morphological features described here refers to the quantitative representation of the scale, boundary, echo, composition and calcification of the lesion area.

[0026] Based on this, in order to reduce the workload of manual annotation and improve the accurate expression of ultrasound nodule morphological features, a hybrid encoder G is specially designed. EM , combined with the decoder G DM , pre-training is performed using mask learning prediction to achieve a complete self-supervised learning framework, so that there is no need for manual labeling. Mask training is used to enable the model to represent the detailed features of ultrasound nodule images.

[0027] In the hybrid encoder G EMIn the decoder G, there can be multiple feature encoding layers. Each feature encoding layer is guided by the convolution mask strategy to realize the encoding representation of the input ultrasound nodule image features, and the last feature encoding layer concatenates and fuses the previous encoding features. DM Decode the concatenated fused coding features, predict the masked image blocks, and reconstruct the original image. When training the hybrid encoder, use the brightness loss L b , contrast loss L c , structural loss L s Construct a composite loss function to optimize the model.

[0028] See also Figure 1 In this example, three guided convolutional layers {C1, C2, C3} are used. C1, C2, and C3 all include patch merging convolutional layers and mask convolutional layer feature extraction modules. The mask convolutional layer feature extraction module is composed of a residual module combined with a Dropout layer and an Inplace-ABN activation layer. The convolution kernel in the residual module performs convolution calculations according to the designed convolutional mask block to avoid leakage of feature information in the masked area.

[0029] The mask blocks designed in the convolutional layers C1, C2, and C3 are implemented by executing the convolutional mask strategy. Figure 1 (b), the right side is the mask image, which is obtained by splitting the original image into small blocks and setting random mask regions. The mask region, its corresponding mask matrix is The mask rate is m. For example, k=14, m=0.7. Based on the mask image, its serialized code M4 can be obtained and converted into the convolution mask matrix form M3. Then, the convolution mask is upsampled by two times upsampling to obtain M2, and the operation is repeated to obtain M1. The upsampling multiple may be different in different specific implementations. The sizes of M1, M2, and M3 are the same as those of C1, C2, and C3. M3, M2, and M1 are three groups of mask convolution regions of different scales, which can be recorded as It is used to guide the convolution layer to perform calculations. Specifically, the convolution kernel of C1 is convolved according to the mask block in M1, the convolution kernel of C2 is convolved according to the mask block in M2, and the convolution kernel of C3 is convolved according to the mask block in M3. The feature representations of the output images are recorded as F c1 , F c2 , F c3 Next, use the mask matrix of M4 For the convolution feature F c3 Serialize and obtain the serialized encoding features of the input image.

[0030] After the convolutional layer C3, a Transformer-based feature representation layer is established, denoted as T4. The serialized encoded features are input into T4, and the corresponding output feature representation is Ft4 .

[0031] After T4, another Transformer layer T5 is connected. T5 uses the attention mechanism to c1 , F c2 , F c3 , F t4} performs weighted and concatenated fusion operations on {w1, w2, w3, w4} to obtain the fusion feature F t5 , as shown in formula (1):

[0032] F t5 =concat[w1F c1 , w2F c2 , w3F c3 , w4F t4 ] (1)

[0033] Among them, w1, w2, w3, w4 are attention weights.

[0034] The above five feature representation layers, namely {C1, C2, C3, T4, T5}, constitute the hybrid masked autoencoder feature representation module G EM , output feature {F c1 , F c2 , F c3 , F t4 , F t5 The mask matrix and mask rate established in the above process guide the image features to be serialized and expanded as the input data of the Transformer layer, thereby achieving seamless connection between convolution and Transformer calculations.

[0035] Decoder G DM Based on Transformer, with hybrid encoder G EM Asymmetric. Decoder and hybrid encoder G EM The model is based on a hybrid autoencoding self-supervised learning model. After training, the model can reduce the workload of manual labeling and improve the accurate expression of the morphological characteristics of ultrasound lesions.

[0036] Decoder G DM Using F t5 And convolution mask strategy to decode and reconstruct the masked area of ​​the image The process can be expressed as:

[0037]

[0038] The image structure similarity metric is used to regularize the constrained mean square error loss function to form a composite loss function L co , as shown in formula (3), the optimized enhanced hybrid encoder GEM and decoder G DM Ability to characterize features of ultrasound nodule images.

[0039]

[0040] Among them, β is the balance factor, L MSE is the mean square error loss function; μ is the pixel mean of the ultrasound nodule image block, is the pixel mean of the predicted ultrasound nodule image block; γ represents the covariance between image blocks, c b =(K1L) 2 and c c =(K2L) 2 They represent brightness difference deviation and contrast difference deviation respectively, where L is the grayscale number 255, and K1<<1 and K2<<1. In one implementation, K1≈0.01, K2≈0.03.

[0041] The above composite loss function L co , adjust the penalty intensity by balancing factor β, and optimize the enhanced hybrid encoder G EM and the first decoder G DM Ability to characterize features of ultrasound nodule images.

[0042] The hybrid encoder G EM and decoder G DM Mask learning is used for pre-training to achieve a complete self-supervised learning framework to mine the feature representation information of unlabeled ultrasound nodule images.

[0043] 2. Uncertainty Error Estimation Module

[0044] For the pre-trained hybrid encoder G EM , remove its convolution mask operation, retain the hybrid encoding structure of convolution and Transformer, and thus construct a hybrid encoder E for lesion segmentation ct , and the hybrid encoder G EM The parameters of the hybrid encoder E ct Parameters.

[0045] See also Figure 2 , hybrid encoder E ct It includes five first feature layers {C1′, C2′, C3′, T4′, T5}, including 3 convolutional layers and 2 Transformer layers, which output features from the side {F c1 , F c2 , F c3 , F t4 , F t5C1′, C2′, C3′, T4′ correspond to the convolutional layers of C1, C2, C3, T4 in sequence and have the same size. C1′, C2′, C3′, T4′ reuse the parameters of C1, C2, C3, T4 (i.e., Ci′ corresponds to Ci, i=1, 2, 3; T4′ corresponds to T4). The convolutional layers involved extract features from the complete image using conventional convolution calculation methods.

[0046] On this basis, a multi-scale decoder D is constructed. m , D m It includes four second feature layers {U4, U3, U2, U1}, and the number of second feature layers is one less than that of the first feature layer, thus forming a symmetrical structure with the hybrid encoder. The scale of the second feature layer {U4, U3, U2, U1} corresponds to the scale of {T4′, C3′, C2′, C1′}. Each second feature layer is skip-connected with the first feature layer of a scale greater than or equal to that layer to reuse all intermediate features of different layers and scales to complete the fusion task. The output features of the four second feature layers {U4, U3, U2, U1} are {F u4 , F u3 , F u2 , F u1}, and decode and fuse the multi-scale features, i.e., {U1: U1+U3+U4, U2: U2+U4}, to further enhance D m The semantic information in the prediction results is constrained, thereby improving the sensitivity to the multi-scale features of the lesion. Among them, {U1: U1+U3+U4} represents the feature F u2 Based on feature F u3 The up-sampled characteristic sample and characteristic F u4 Decoding fusion, {U2:U2+U4} represents feature F u1 Based on feature F u2 The up-sampled characteristic sample and characteristic F c4 , Feature F u3 Decoding fusion.

[0047] D m and E ct Useg, the backbone network module for ultrasound nodule image lesion segmentation, realizes the preliminary segmentation of nodule lesions in ultrasound nodule images and obtains N lesion contour probability maps, where N is a set value.

[0048] The uncertainty error estimation module is formed by introducing the Monte Carlo pruning layer in the {C3′, T4′, T5, U4, U5} layer in the backbone network module Useg. The uncertainty error estimation module uses the MC-Dropout operation to force the model to learn the feature distribution of the image to enhance the robustness of the model. The uncertainty error estimation module is based on the fusion feature F u1To predict the probability map of the lesion area segmentation, and then obtain the lesion contour M p and the contour uncertainty error M e .

[0049] The uncertainty error estimation module uses the 2D ultrasound label dataset to train the uncertainty error estimation network Mseg.

[0050] In one embodiment, the U4 structure is composed of a multi-scale feature merging layer, a multi-head self-attention layer, and an MC-Dropout layer in sequence; the U3 structure is composed of a Patch expansion convolution layer, a multi-scale feature merging layer, a multi-head self-attention layer, and an MC-Dropout layer (Monte Carlo pruning layer) in sequence; the U2 and U1 structures are composed of a Patch expansion convolution layer, a multi-scale feature merging layer, and a multi-head self-attention layer. Among them, the pruning probability Δp of the Monte Carlo pruning layer MC is 0.3, to obtain model D m The posterior probability distribution of the weights. If the probability of suspicious area mapping in the lesion suspicious area mapping probability map of a single feedforward prediction ultrasound nodule image is x is the input ultrasound image, The feature encoding of the ultrasound image after the Monte Carlo pruning layer is embedded in the feature layer [C3′, T4′, T5]. Decryption probability of features after the Monte Carlo pruning layer is embedded in the feature layer.

[0051] For the obtained N lesion contour probability maps, the average value is obtained to obtain the lesion contour M p and the contour uncertainty error M e , as shown in formulas (4) and (5).

[0052]

[0053]

[0054] Among them, δ 2 (*) represents the variance calculation function of the probability map, and the Δp0 uncertainty cutoff value is the uncertainty probability peak value. mi is the probability of mapping the suspicious area of ​​ultrasound lesions predicted by the i-th feedforward. (5) represents the uncertainty error of the contour under the condition that the uncertainty cutoff value is 0.5.

[0055] (III) Morphological feature quantitative estimation module

[0056] A quantitative estimation module for nodule morphological features based on clinical prior knowledge is established to achieve quantitative representation of the scale, boundary, echo quality, composition and calcification of the lesion area. The scale, boundary, echo quality, composition and calcification of the lesion area are quantified and recorded as Qa , Q m , Q e , Q p , Q c , the estimation method is as follows.

[0057] Nodule size quantification Q a , is calculated by calculating the contour point set P n =f n (M p ), the contour point set is used to fit an ellipse to measure the scale of the nodule lesion. n , nodule size quantization Q a The estimation is shown in formula (6).

[0058]

[0059] Among them, f area (*) is the area calculation function, W is the length of the lesion area, h is the width of the lesion area, d is the depth of the lesion area, R w / h is the aspect ratio of the lesion area, S o is the area of ​​the lesion. Figure 3 Schematic diagram of the quantitative calculation of morphological features guided by prior knowledge of the lesion area, where the contour of the lesion area is represented by M p Based on the contour information, the geometric center coordinates O, area S, width w, height h, depth d and other scale feature information are calculated, and the shape information of the nodule lesion is measured by fitting an ellipse through the contour point set. At the same time, the boundary concave-convex ratio function f un (P n ) Calculate the protruding area S of the lesion area beyond the ellipse n and the concave area S u Area. Nodule size quantification Q a , which can be used as a factor to evaluate the smoothness of nodule boundaries and to assist in identifying benign or malignant lesions.

[0060] Contour Quantization Q m , is to measure and evaluate the smoothness of the ultrasound lesion contour. Based on the obtained lesion contour M p and M e , contour quantization Q m The estimation is shown in formula (7).

[0061]

[0062] Among them, M rp , r a , r b Respectively represent the set of fitted ellipse points, the minor axis and major axis of the ellipse, and f ellispse (*) is the ellipse fitting function; S n, S u They represent the area outside the ellipse and the area inside the ellipse when the ellipse boundary intersects the lesion boundary, respectively. The area calculation function f area (*) Obtained. L(M rp ) represents the circumference of the fitted ellipse, L(P n ) represents the perimeter of the nodule contour. Δp m Represents the probability map cutoff value. R lm Indicates the similarity between the ultrasound lesion area and the ellipse, and also reflects the shape and boundary smoothness of the lesion area. cm It represents the uncertainty error range of the prediction of the ultrasound lesion contour, which is the four-quadrant area φ under the plane coordinates of the geometric center of the lesion area cmi The sum of the uncertainty errors also reflects the clarity of the lesion contour area.

[0063] Echo quality feature quantization Q e , in order to analyze and evaluate the echo quality of the lesion area. Based on the obtained lesion contour M p , echo quality feature quantization Q e The estimation is shown in formula (8).

[0064]

[0065] Among them, {t ei , i = [1, 2, 3, 4]} represent the thresholds of anechoic, hypoechoic, isoechoic, and hyperechoic regions, respectively, and the isoechoic threshold t ei =ψ(M e ), ψ is the threshold calculation function, which uses the echo intensity of the normal tissue around the lesion as the calculation function, and the calculated value is used as the equal echo threshold. That is, it is the echo mean of the normal tissue area. S e.nene , S e.low , S e.equ , S e . high Represents the echo area corresponding to the threshold calculation. ei , i = [1, 2, 3, 4]} represents the ratio of the echo area to its envelope area under the corresponding threshold, reflecting the dispersion degree of the echo area.

[0066] Component Characterization Quantization Q p , through the grayscale image threshold method t c ={t c1 , t c2 , t c3} and contour detection method to estimate the number of grayscale holes in the suspicious nodule area C b Its proportion cb , and the number of bright spots C w Its proportion cw, which represents the quantitative value of nodule component characteristics.

[0067] Calcification feature quantification Q c By first establishing the four equal lines from the contour to the geometric center, the nodule is divided into four annular areas, and then on this basis, it is divided into four quadrants based on the plane coordinate system to quantify the calcification characteristic information. p , Calcification feature quantification Q c The estimation is shown in formula (9).

[0068]

[0069] Among them, t ci is the grayscale image threshold t c The weight, η i Indicates the percentage of calcification echo threshold compared to the pixel value of 255, ranging from 70% to 98%; S ci is the area of ​​the calcified region and has an area function f area (*) and threshold-based calcification region extraction function f echo (*) calculation, x represents the input image; r ci =ψ ratio (*) indicates the ratio of the calcification area to its envelope area under the corresponding threshold, reflecting the dispersion degree of the calcification area; L ci =Φ pos (*) indicates the orientation information of the calcified interconnected area, f contours (*) indicates the contour point extraction of the calcified interconnected area, l lines Indicates that the internal area of ​​the lesion is divided into four equal parts based on the lesion contour. lines ={l1, l2, l3, l4} and l4 is the contour line, D directions It represents the four-quadrant orientation in the plane coordinates based on the geometric center of the lesion area. The orientation information of the calcification point inside the lesion can be obtained through these two information.

[0070] (IV) Interpretable Ultrasound Lesion Recognition Module

[0071] In the interpretable ultrasound lesion recognition module, the quantitative information is aligned with the image features of the lesion area and semantically fused to achieve effective interpretation of the accurately identified ultrasound nodule lesions.

[0072] Specifically, the quantitative information {Q a , Q m , Q e , Q p , Q c} as an explicit feature. At the same time, the uncertainty error M of the obtained contour is used eCalculate and determine the suspicious lesion area, and record the suspicious lesion area as X ROI . And build a classification network to extract the image features of the suspicious lesion area, and use the image features as implicit feature information F x The cross-attention mechanism is used to input the explicit and implicit features into the lesion recognition module R. bm , in order to achieve accurate identification of benign and malignant lesions, visualize the quantitative information, and improve the interpretability of the auxiliary diagnosis of the recognition model.

[0073] In summary, Figure 4 The above-mentioned interpretable recognition process using multi-scale probability mapping and morphological feature quantization is illustrated. Figure (a) shows the feature representation module of the hybrid mask self-encoder, which uses the convolutional mask strategy to seamlessly connect the convolution layer with the Transformer to establish the feature representation module G of the hybrid mask self-encoder. EM , and uses the structural similarity regularized composite loss function to optimize L co and the asymmetric decoder G DM Ultrasound nodule image mask learning is performed to enable the model to more efficiently represent the features of unlabeled ultrasound nodule image data. Figure (b) shows the uncertainty error estimation module (Mseg), which transforms G EM The mask strategy in is removed to construct the hybrid encoder E ct , and further build a symmetric decoder D by using jump-layer connection and multi-scale feature fusion operations m , and further constructs the contour uncertainty error estimation module Mseg by introducing Monte Carlo pruning in the high-dimensional semantic structure layer, so as to capture the probability mapping of the segmentation results of ultrasound nodule lesions and the contour uncertainty error. In the figure (c), the nodule morphological feature quantification estimation and interpretability recognition module is established. Based on the contour information and contour uncertainty error in the segmentation results, it uses prior knowledge combined with image processing to establish a prior knowledge-guided lesion morphological feature quantification module, which realizes the quantitative representation of the scale, boundary, echo, component and calcification of the lesion area, among which the quantification of the scale can be seen in the boundary attribute. And on this basis, a multi-information fusion interpretability recognition module is further constructed, and the explicit quantitative information and the implicit image feature information of the suspicious lesion area are aligned and fused by using the cross-attention mechanism, so as to further improve the recognition accuracy of the model for ultrasound nodule lesions and effectively enhance its recognition interpretability.

[0074] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0075] (V) Application

[0076] The backbone network module Useg for ultrasound nodule image lesion segmentation, the contour uncertainty error estimation module, and the nodule morphological feature quantitative estimation and interpretability identification module constitute a lesion interpretability identification module, among which the backbone network module Useg for ultrasound nodule image lesion segmentation is trained with ultrasound nodule images with specific contour labels.

[0077] The interpretable lesion identification module is used to identify the interpretable thyroid ultrasound nodule lesions. The contour morphological features (nodule boundary quantification) are visualized. Figure 5 , where (a) is the visualization of ultrasound nodule recognition results and contour prediction results, (b) is the visualization of nodule recognition probability mapping, (c) is the visualization of contour uncertainty error, and (d) is the visualization of nodule boundary quantization representation; and for the visualization of nodule internal morphological features, see Figure 6 , where (a) is the visualization of ultrasound nodule identification results and contour prediction results, (b) is the visualization of calcification quantification, (c) is the visualization of component quantification, and (d) is the visualization of echo quality quantification). Through the description of the above implementation methods, those skilled in the art can clearly understand that the disclosed system can be implemented by means of software plus necessary general-purpose hardware, and of course, it can also be implemented by dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present disclosure, software program implementation is a better implementation method in most cases.

[0078] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.

Claims

1. An interpretable recognition system based on multi-scale probability mapping morphological feature quantification, characterized by: The system includes an uncertainty error estimation module, a morphological feature quantitative estimation module, and an interpretability identification module; The uncertainty error estimation module is configured to learn the multi-scale feature distribution of the lesion area in the ultrasonic nodule image, obtain the contour segmentation probability mapping map of N lesion areas, and then obtain the lesion contour Mp and the contour uncertainty error M e , N is the set value; The morphological feature quantitative estimation module is configured based on the lesion contour Mp and the contour uncertainty error M e , obtain the quantitative representation of the morphological characteristics of nodular lesions; The interpretability identification module is configured to first identify the uncertainty error M of the contour based on e The lesion area is calculated to obtain the image features of the lesion area, and then the lesion attributes are visualized based on the quantitative representation of the image features and the morphological features of the nodule lesion.

2. The system according to claim 1, characterized in that The uncertainty error estimation module includes a preset hybrid encoder and a multi-scale decoder; The preset hybrid encoder has multiple first feature encoding layers, encodes the original image and performs splicing and fusion of the encoding features, and the multi-scale decoder has multiple second feature decoding layers, decodes and fuses the spliced ​​and fused features to obtain fusion features of a probability map for predicting lesion area segmentation; wherein the number of second feature encoding layers is 1 less than that of first feature decoding layers, and each second feature layer is jump-connected to a first feature layer of a scale greater than or equal to that layer; A Monte Carlo pruning layer is provided in the preset hybrid encoder and the multi-scale decoder to perform multi-scale feature distribution learning.

3. The system according to claim 2, characterized in that The parameters of the preset hybrid encoder are obtained through mask learning pre-training, and the steps include: The preset hybrid encoder is denoted as E ct , constructing a hybrid encoder E ct A hybrid encoder G with the same structure EM , and construct a Transformer-based decoder G DM ; Mask the original image according to the set mask matrix and mask rate to obtain the convolution mask strategy and mask image; In pre-training, the hybrid encoder G EM Based on the convolution mask strategy, the mask image is encoded and the encoded features are concatenated and fused. The decoder G DM Decode the spliced ​​and fused coding features to reconstruct the original image. The loss function used is the image structure similarity measure to regularize the constrained mean square error loss function. After pre-training, the hybrid encoder G EM The parameters of the hybrid encoder E ct Parameters.

4. The system according to claim 3, characterized in that The preset hybrid encoder has five first feature layers, which are divided into 3 convolutional layers and 2 Transformer layers, and Monte Carlo pruning layers are set in the 3rd to 5th layers.

5. The system according to claim 2, characterized in that The multi-scale decoder is configured to have four second feature layers {U4, U3, U2, U1}, wherein a Monte Carlo pruning layer is provided in U4 and U3, and the four second feature layers {U4, U3, U2, U1} sequentially output decoding features as {F u4 , F u3 , F u2 , F u1 }, where feature F u2 Based on feature F u3 The up-sampled characteristic sample and characteristic F u4 Decoding fusion, feature F u1 Based on feature F u2 The up-sampled characteristic sample and characteristic F u4 , Feature F u3 Decoding fusion.

6. The system according to claim 1, characterized in that Lesion contour Mp and contour uncertainty error M e , the acquisition steps include: The probability of mapping the i-th feedforward predicted ultrasound lesion suspicious area is denoted as p mi , then the lesion contour M p for: N is the total number of times; The uncertainty cutoff value is recorded as Δp0, then the uncertainty error of the contour M e : Among them, δ 2 (*) represents the variance calculation function of the probability map.

7. The system according to claim 1, characterized in that The quantification of lesion morphological characteristics includes the quantification of the scale, boundary, echo quality, composition and calcification of the lesion area.

8. The system according to claim 3, characterized in that The loss function is: Among them, β is the balance factor, L MSE is the mean square error loss function; μ is the pixel mean of the ultrasound nodule image block, is the pixel mean of the predicted ultrasound nodule image block; γ represents the covariance between image blocks, c b and c c They represent brightness difference deviation and contrast difference deviation respectively.

9. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the system as claimed in any one of claims 1 to 8.

Images