Disease Intelligent Recognition Method and System Based on Fine-Grained Domain Knowledge

Abstract

The present invention provides a disease intelligent recognition method and system based on fine-grained domain knowledge, belonging to the field of intelligent medical technology. A dual-coordinate disease recognition model is used to extract features from the patient's lesion image to obtain lesion region features and lesion boundary features. Among them, the dual-coordinate disease recognition model includes a first coordinate model for lesion region division and a second coordinate model for lesion boundary feature recognition. Through a self-attention network, lesion aggregation features are obtained according to the lesion region features and the lesion boundary features. The lesion aggregation features are classified and recognized to obtain the disease recognition result of the patient, making the fine-grained information of the disease interpretable. It is convenient to assist doctors in qualitatively and even quantitatively analyzing the lesion and other regions of interest, thus greatly improving the accuracy and reliability of medical diagnosis. It can also play an important auxiliary role in medical teaching, surgical planning, surgical simulation and various medical research.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent medicine, and particularly relates to a disease intelligent recognition method, system, electronic device and storage medium based on fine-grained domain knowledge. Background Art

[0002] In the prior art, ultrasound imaging examination has become the most commonly used tool for diagnosing cancer due to its advantages of harmlessness, non-invasiveness, low cost, rapid imaging, etc. Among them, B-mode has become the most commonly used ultrasound imaging mode in cancer examination due to its sufficient sensitivity. Based on the ultrasound images of the disease area, the computer-aided design (CAD) technology using the efficient and scalable learning of deep neural networks (DNN) performs excellently in the intelligent diagnosis of diseases. Most of them use transfer learning to overcome the problem of limited datasets in medical image processing and utilize the powerful feature representation ability of the pre-trained backbone model.

[0003] Taking thyroid diseases as an example, the Thyroid Imaging Reporting and Data System (TI-RADS) grades according to five malignant features of thyroid nodules: 1. Solid nodules; 2. Hypoechoic or very hypoechoic; 3. Lobulated or irregular margins; 4. Microcalcifications; 5. Aspect ratio ≥ 1. Currently, there are problems in diagnosing thyroid diseases using B-mode ultrasound imaging. The existing TI-RADS guidelines are relatively subjective, and the diagnosis of thyroid diseases requires high experience of radiologists. On this basis, making full use of data-driven methods to automatically diagnose thyroid diseases has become a feasible solution. In the process of intelligent diagnosis of thyroid diseases, although transfer learning is used to overcome the problem of limited datasets in medical image processing and the powerful feature representation ability of the pre-trained backbone model provides some assistance for the intelligent diagnosis of the thyroid, there are still the following drawbacks: 1) The above method regards the thyroid diagnosis task as a conventional binary classification, only reflecting two classification labels of benign / malignant. However, benign and malignant are related to multiple factors. Just modeling with benign and malignant labels is relatively lacking in sufficient interpretability.

[0004] In order to improve interpretability, the multitask learning (MTL) method is applied to the intelligent diagnosis of thyroid diseases, and the domain knowledge (DK) formed by doctors' experience is incorporated into the modeling process as supervision information; although some TI-RADS information is utilized, there are still only two classification labels of benign and malignant, and the interpretability of the fine-grained information of the classification variables is insufficient.

[0005] Therefore, there is an urgent need for a disease intelligent recognition method based on fine-grained domain knowledge. Summary of the Invention

[0006] The present invention provides a disease intelligent recognition method, system, electronic device, and storage medium based on fine-grained domain knowledge to overcome at least one technical problem existing in the prior art.

[0007] To achieve the above object, the present invention provides a disease intelligent recognition method based on fine-grained domain knowledge.

[0008] Obtain the patient's lesion image to be recognized.

[0009] Use a double-coordinate disease recognition model to extract features from the patient's lesion image to obtain lesion region features and lesion boundary features; wherein, the double-coordinate disease recognition model includes a first coordinate model for lesion region division and a second coordinate model for lesion boundary feature recognition.

[0010] Through a self-attention network, obtain lesion aggregation features according to the lesion region features and lesion boundary features.

[0011] Perform classification recognition on the lesion aggregation features to obtain the disease recognition result of the patient; wherein, the disease recognition result includes the determination of the benign / malignant category of the lesion and the lesion location.

[0012] Further, preferably, the method for obtaining lesion aggregation features according to the lesion region features and lesion boundary features through a self-attention network includes:

[0013] Fuse the lesion region features and lesion boundary features to obtain original fusion features.

[0014] After passing the original fusion features through a convolutional layer and an activation function, obtain the saliency weight features corresponding to the input features.

[0015] Multiply the saliency weight features corresponding to the input features by the original features to obtain aggregation features.

[0016] Further, preferably, the method for obtaining lesion aggregation features according to the lesion region features and lesion boundary features through a self-attention network includes:

[0017] Use a split-merge module to perform feature disentanglement on the lesion boundary features to obtain multiple lesion boundary feature blocks.

[0018] Mark multiple lesion boundary feature blocks and lesion region feature blocks, and embed class label feature blocks to obtain marked lesion feature blocks.

[0019] Use a multi-head self-attention module to obtain aggregation features using the marked lesion feature blocks.

[0020] Further, preferably, the training method of the dual - coordinate disease recognition model includes

[0021] Pre - processing the samples containing lesion region annotation information to obtain a lesion region mask and a normal region mask; obtaining a trained first - coordinate model based on the lesion region mask and the normal region mask; pre - processing the samples containing lesion dot - matrix annotation information to obtain a vectorized representation of the lesion dot - matrix; obtaining a trained second - coordinate model based on the vectorized representation of the lesion dot - matrix; wherein, the first - coordinate model is a Cartesian coordinate model; the second - coordinate model is a polar coordinate model;

[0022] Integrating the trained first - coordinate model and the second - coordinate model based on a multi - head self - attention module to obtain a dual - coordinate disease recognition model;

[0023] Training the dual - coordinate disease recognition model based on a loss function using the gradient back - propagation algorithm until convergence.

[0024] Further, preferably, the method of training the dual - coordinate disease recognition model based on a loss function using the gradient back - propagation algorithm until convergence includes

[0025] Obtaining the score distribution of the disease benign - malignant evaluation and the score distribution of the lesion location evaluation of the input samples containing lesion region annotation information through the dual - coordinate disease recognition model;

[0026] Obtaining the loss function between the score distribution of the disease benign - malignant evaluation and the original score distribution of the disease benign - malignant evaluation corresponding to the sample image dataset, and the loss function between the score distribution of the lesion location evaluation and the original score distribution of the disease lesion location evaluation corresponding to the sample image dataset;

[0027] Updating the network parameters of the dual - coordinate disease recognition model according to the loss function until the mean square errors of both the score distribution of the disease benign - malignant evaluation and the score distribution of the lesion location evaluation belong to the preset standard range.

[0028] Further, preferably, the lesion dot - matrix annotation information includes benign - malignant feature block markers, clear - edge feature block markers, spiculation feature block markers, angulated - edge feature block markers, smooth - edge feature block markers, and ultrasonic image feature block markers.

[0029] Further, preferably, the training method of the dual - coordinate disease recognition model further includes

[0030] Establishing a disease scoring system using binary logistic regression analysis;

[0031] Based on the disease scoring system, a stepwise regression method is used to screen independent variables that are correlated with the benign or malignant evaluation of the disease;

[0032] The independent variables that are correlated with the benign or malignant evaluation of the disease are used to determine the correlation between the predicted value of the double - coordinate disease recognition model and the benign or malignant nature of the disease

[0033] To solve the above problems, the present invention also provides a disease intelligent recognition system based on fine - grained domain knowledge, including:

[0034] A feature extraction unit, configured to obtain the lesion image of the patient to be recognized; use the double - coordinate disease recognition model to extract features from the patient's lesion image to obtain lesion area features and lesion boundary features; wherein, the double - coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition;

[0035] A feature aggregation unit, configured to obtain lesion aggregation features according to the lesion area features and the lesion boundary features through a self - attention network;

[0036] An identification unit, configured to perform classification and identification on the lesion aggregation features to obtain the disease recognition result of the patient; wherein, the disease recognition result includes the determination of the benign / malignant category of the lesion and the location of the lesion.

[0037] To solve the above problems, the present invention also provides an electronic device, which includes:

[0038] A memory, storing at least one instruction; and

[0039] A processor, configured to execute the instructions stored in the memory to implement the steps in the above - mentioned disease intelligent recognition method based on fine - grained domain knowledge.

[0040] To solve the above problems, the present invention also provides a computer - readable storage medium, in which at least one instruction is stored, and the at least one instruction is executed by a processor in an electronic device to implement the above - mentioned disease intelligent recognition method based on fine - grained domain knowledge.

[0041] A disease intelligent recognition method, system, electronic device and storage medium based on fine-grained domain knowledge of the present invention, by obtaining a patient lesion image to be recognized; using a double-coordinate disease recognition model to extract features from the patient lesion image to obtain lesion area features and lesion boundary features; wherein, the double-coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition; through a self-attention network, obtaining lesion aggregation features according to the lesion area features and the lesion boundary features; classifying and recognizing the lesion aggregation features to obtain a disease recognition result of the patient; wherein, the disease recognition result includes the determination of the benign / malignant category of the lesion and the lesion location; enabling the fine-grained information of disease classification variables to be interpretable; achieving the effect of facilitating the doctor to qualitatively and even quantitatively analyze the disease lesion and other regions of interest, and greatly improving the accuracy and reliability of medical diagnosis; and also playing an important auxiliary role in medical teaching, surgical planning, surgical simulation and various medical researches. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a schematic flowchart of a disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention;

[0044] Figure 2 It is a schematic principle framework diagram of a disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention;

[0045] Figure 3 It is another schematic principle diagram of a disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention;

[0046] Figure 4 It is a schematic network structure diagram of a disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention;

[0047] Figure 5 It is a schematic module diagram of a disease intelligent recognition system based on fine-grained domain knowledge provided by an embodiment of the present invention;

[0048] Figure 6 It is a schematic internal structure diagram of an electronic device for implementing a disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention;

[0049] The implementation, functional features and advantages of the present invention will be further described in conjunction with embodiments with reference to the accompanying drawings. Detailed implementation manners

[0050] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0051] Figure 1 The figure is a schematic flowchart of a disease intelligent recognition method based on fine-grained domain knowledge provided for an embodiment of the present invention. This method can be executed by a system, and the system can be implemented by software and / or hardware.

[0052] The disease intelligent recognition method based on fine-grained domain knowledge of the present invention is mainly applicable to the artificial intelligence diagnosis scenario of thyroid nodules. In the prior art, the diagnosis of thyroid nodules is carried out by observing the ultrasound images of thyroid nodules, which often requires relying on the experience of doctors to judge, consuming a lot of time, and may be affected by the subjective factors of doctors, thus affecting the accuracy of the judgment results.

[0053] Artificial Intelligence (AI): Using a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, a theory, method, technology and application system that perceives the environment, acquires knowledge and uses knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines, enabling the machines to have the functions of perception, reasoning and decision-making. Artificial intelligence technology is an interdisciplinary subject, involving a wide range of fields, including both hardware-level technologies and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0054] Computer Vision (CV): It is a science that studies how to enable machines to "see". More specifically, it refers to using cameras and computers to replace human eyes for machine vision such as target recognition, tracking, and measurement, and further performing graphic processing to make the computer-processed images more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to establish artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technologies usually include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc. technologies, and also include common biometric recognition technologies such as face recognition and fingerprint recognition.

[0055] Based on computer vision and artificial intelligence technologies, a disease intelligent recognition method, system, electronic device, and storage medium based on fine-grained domain knowledge of the present invention obtain a patient lesion image to be recognized; use a double-coordinate disease recognition model to extract features from the patient lesion image to obtain lesion area features and lesion boundary features; wherein, the double-coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition; through a self-attention network, obtain lesion aggregation features according to the lesion area features and the lesion boundary features; perform classification recognition on the lesion aggregation features to obtain a disease recognition result of the patient; wherein, the disease recognition result includes the determination of the benign / malignant category of the lesion and the lesion location, making the fine-grained information of the thyroid nodule classification variable interpretable; achieving the effect of facilitating the doctor to qualitatively and even quantitatively analyze the thyroid lesion and other regions of interest, and greatly improving the accuracy and reliability of medical diagnosis; it can also play an important auxiliary role in medical teaching, surgical planning, surgical simulation, and various medical research.

[0056] As Figure 1 shown, in this embodiment, the invention is specifically described taking thyroid diseases as an example. The disease intelligent recognition method based on fine-grained domain knowledge includes steps S110 to S130.

[0057] S110. Obtain a patient lesion image to be recognized.

[0058] It should be noted that the widely used types of medical images mainly include Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Nuclear Medicine Imaging (NMI), and Ultrasonic Imaging (UI), etc. After obtaining the medical image (i.e., the lesion image), the image is preprocessed. The preprocessing steps include, but are not limited to, techniques and processes of dividing the image into several specific regions with unique properties and extracting the target of interest. From a mathematical perspective, image segmentation is to divide a digital image into multiple non-overlapping image sub-regions (sets of pixels), which is also a labeling process, that is, assigning the same label to the pixels belonging to the same region (having the same visual characteristics).

[0059] S120. Use the dual-coordinate disease recognition model to extract features from the patient's lesion image, and obtain the lesion area features and lesion boundary features; wherein, the dual-coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition.

[0060] Specifically, the training method of the dual-coordinate disease recognition model includes steps S121 - S123.

[0061] S121. Preprocess the sample containing the lesion area annotation information to obtain the lesion area mask and the normal area mask; obtain the trained first coordinate model based on the lesion area mask and the normal area mask; preprocess the sample containing the lesion dot matrix annotation information to obtain the vectorized representation of the lesion dot matrix; obtain the trained second coordinate model based on the vectorized representation of the lesion dot matrix; wherein, the first coordinate model is a Cartesian coordinate model; the second coordinate model is a polar coordinate model.

[0062] That is to say, use the polar coordinate feature model and the Cartesian coordinate feature model to extract features from the preprocessed lesion image of the patient respectively; wherein, the regional feature vector of the lesion image is extracted through the polar coordinate feature model; the edge feature vector of the lesion image is extracted through the Cartesian coordinate feature model.

[0063] Figures 2 - 4 A holistic description of the principle of the dual-coordinate disease recognition model is given. Among them, Figure 2 is a schematic diagram of the principle framework of the disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention; Figure 3 is another schematic diagram of the principle of the disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention.

[0064] As Figure 2 and Figure 3 shown, in order to make the thyroid nodule classification method based on intelligent medicine more interpretable and accurate by making the most of domain knowledge, the present invention proposes an MTL framework based on knowledge dual-image coordinates, namely Two Image Coordinates for Knowledge Embedding in Thyroid (TICKET), for the diagnosis of benign and malignant thyroid nodules, which embeds two common types of domain knowledge, coarse-grained and fine-grained. In the Domain Knowledge Representation stage, all knowledge is divided into region-based and margin-based categories. After the Domain Knowledge Representation stage, the first coordinate model and the second coordinate model are integrated to obtain the Bi-coordinate model, i.e., the thyroid classification and recognition model; the thyroid classification and recognition model is used to learn the following aspects, the two branches of learning and predicting this knowledge, and a relationship module for further fusing the characteristics of the two branches to further improve performance. By learning this knowledge and using a progressive learning strategy, the performance of the main benign and malignant classification tasks of each MTL model is steadily improved.

[0065] For the Cartesian feature extraction model, the recognition results are weighted, summed, combined, and averaged, and binary classification is performed on the fusion result; the recognition results of each lesion image after binary classification are converted to the Cartesian coordinate system using bilinear interpolation, and the connected regions are labeled for the converted results; in the labeled connected regions, it is determined whether the region is a thyroid nodule lesion region, a malignant or benign nodule lesion region.

[0066] In the specific implementation process, for the second coordinate model, first, the original thyroid nodule lesion image is converted from the Cartesian product coordinate system to the polar coordinate system, and the thyroid nodule lesion image in the polar coordinate system is preprocessed, including removing interference information such as interfering catheters at the top and removing noise. Then, each column of the preprocessed thyroid nodule lesion image is used as a sample to reconstruct the data set. The specific implementation method can be, but is not limited to, randomly selecting 100,000 positive and negative samples from the data set to form a training set, randomly selecting 5 times in total, and using stacked autoencoders to establish learning models respectively, obtaining 5 learning models in total, and then fusing these 5 models. The fusion rule is to perform weighted summation and averaging on the obtained multiple recognition results, and perform binary classification on the fusion result.

[0067] In the existing thyroid ultrasound examination technology, when a nodule is detected, a radiologist usually locates some important features according to authoritative TI-RADS (such as ACR TI-RADS) to evaluate the risk of the nodule. In the present invention, as Figure 2 shown, region-based means that given an image patch x in a Cartesian coordinate system, a radiologist performs segmentation annotation on the corresponding region-based knowledge, represented as a binary mask. Edge-based means that in ACR TI-RADS, it is based on features such as shape, edge, and echo lesions. As Figure 2 shown, considering the annotation cost and importance of features, the following edge-based features are selected for annotation and model learning: clear, spiculated, lobulated, smooth, shadow. Among them, the thyroid TI-RADS grading standard published by ACR (American College of Radiology) in 2017, and the 5 scores mentioned in the ACR standard (nodule composition, sound, morphology, edge, calcified hyperecho).

[0068] S122. After integrating the trained first coordinate model and the second coordinate model based on the multi-head self-attention module, a dual-coordinate disease recognition model is obtained.

[0069] S123. Based on the loss function, the dual-coordinate disease recognition model is trained using the gradient backpropagation algorithm until convergence.

[0070] Specifically, the knowledge in the field of thyroid ultrasound diagnosis is divided into two categories: region-based knowledge and boundary-based knowledge. According to the different attributes of the regions involved in these two types of knowledge, they are represented in the forms of masks and vectors respectively. The regional feature vectors of the lesion images extracted by the polar coordinate feature model are represented in the form of masks; the edge feature vectors of the lesion images extracted by the Cartesian coordinate feature model are represented in the form of multiple vectors. Generally speaking, the process of classifying and identifying thyroid nodules using the disease intelligent recognition method based on fine-grained domain knowledge includes: obtaining the medical image to be recognized; passing the obtained medical image through the polar coordinate feature model and the Cartesian coordinate feature model respectively; then obtaining the recognition result of the medical image content and the feature map corresponding to the medical image to be recognized through the classifier. Based on the backpropagation algorithm, the two coordinate models, namely the polar coordinate feature model and the Cartesian coordinate feature model, are trained separately. After training the two coordinate system models, these two models are combined to form the bi-coordinate model of the present invention. And the gradient backpropagation algorithm is used to train the combined model, and the cosine annealing learning rate strategy with warmup is used to ensure that the training process is stable and can converge quickly. In terms of the loss function, the Focal loss is used for the classification variable and the vector regression task, and the Dice loss is used for the segmentation task. That is to say, it is hoped to extract features consistent with the annotation. The function of the network is feature extraction, and the ultimate goal is that the feature extraction result of the network is consistent with the annotated features. Finally, for the trained dual-coordinate disease recognition model, when the image of the thyroid lesion to be recognized is input, the dual-coordinate disease recognition model will output the types and corresponding positions of the features respectively. Specifically, the overall determination result includes benign and malignant determination, lesion location, and lesion manifestation. Among them, the lesion manifestation includes whether the edge is clear, whether there are burrs, whether the edge forms an angle, whether the edge is smooth, and whether there is a contrast-enhanced ultrasound manifestation.

[0071] The first coordinate model and the second coordinate model are any one of a residual network (ResNet), a Visual Geometry Group network (VGGNet), and a Squeeze-and-Excitation Network (SENet).

[0072] Specifically, the first coordinate model and the second coordinate model are integrated based on the multi-head self-attention structure to obtain the final classification features of the thyroid nodules; among them, the first coordinate model and the second coordinate model have the same encoder structure, independent decoders and fully connected layer structures; the thyroid nodule classifier is trained using the final classification features of the thyroid nodules to obtain a dual-coordinate disease recognition model; the dual-coordinate disease recognition model is trained and constrained through a loss function.

[0073] As Figure 3 shown, both the first coordinate model and the second coordinate model use the method of multi-task learning to learn and predict domain knowledge through a general encoder-decoder architecture. Specifically, it is desired to map the latent feature space encoded by the encoder to various fine-grained domain knowledge, such as segmentation results and localization results. On the other hand, the information from the encoder is also used to predict coarse-grained qualitative domain knowledge. The network structure diagrams of the two coordinate system models, the first coordinate model and the second coordinate model, are as Figure 3 shown. In the Cartesian coordinate system, a hard parameter sharing structure is mainly used because the corresponding auxiliary multi-tasks have high correlations. While in the polar coordinate system, a channel separation structure is used because the edge information is located differently and has weak correlations. Among them, hard parameter sharing means that the feature extraction layer parameters of the two networks are the same, that is, the encoder; while task channel splitting means that for each task, there is an independent channel for separate prediction, rather than all channels together for single-body prediction. In addition, these two structures have the same encoder structure, but the decoder structures for the Cartesian coordinate system and the polar coordinate feature model are different. A split-merge module for feature disentanglement is provided between the decoder and the encoder of the second coordinate model. The main difference is that TCS adds an additional split and merge (SM) block for feature disentanglement and uses independent decoders and fully connected layers to predict different-grained edge-based knowledge. The feature disentanglement includes performing a convolution operation on the local gradient corresponding to the feature map.

[0074] S130. Through the self-attention network, obtain the lesion aggregation features according to the lesion area features and the lesion boundary features.

[0075] Figure 4 is a schematic diagram of the network structure of the disease intelligent recognition method based on fine-grained domain knowledge provided by an embodiment of the present invention. As Figure 4As shown in the figure, after training the two coordinate models, namely the first coordinate model and the second coordinate model, the two models are combined to form the bi - coordinate model of the present invention. In the specific implementation process, the characteristics of the first coordinate model and the second coordinate model can be combined through two different attention mechanisms. And there are two integration methods. One is to use the channel weighting method in SENet (CW - SE), and the other is to use the Self - Attention module (SA - Trans) in the Transformer decoder. That is to say, an attention - based feature integration method is selected, that is, two different attention mechanisms are used to balance the features of the two coordinate models that are already good at the main task, rather than simple direct concatenation (DC).

[0076] A method for obtaining lesion aggregation features according to lesion region features and lesion boundary features through a self - attention network, including: S1311, fusing the lesion region features and the lesion boundary features to obtain original fused features; S1312, after passing the original fused features through a convolutional layer and an activation function, obtaining the saliency weight features corresponding to the input features; S1313, multiplying the saliency weight features corresponding to the input features by the original features to obtain aggregation features.

[0077] As Figure 4 As shown in the upper part of the figure, in the SENet (Squeeze - and - Excitation Networks) algorithm model, the channel weighting method is used; that is, different weights are assigned to different channels. Here, the preset weighting setting method can include, but is not limited to: the average value of the weighting values of multiple sampled feature information. In this case, when the bi - coordinate disease recognition model executes this step, specifically, it can first weight multiple sampled feature information respectively to obtain multiple weighting values, and then use the average value of the multiple weighting values as the detection score of the image within the lesion region candidate box. Among them, a certain weighting value is the product of a certain sampled feature information and the corresponding weight value, and the weight values corresponding to each sampled feature information can be preset into the bi - coordinate disease recognition model by the user in advance, or obtained through a certain training method.

[0078] In a specific embodiment, a method for obtaining lesion aggregation features according to lesion region features and lesion boundary features through a self-attention network includes: S1321. Using a split-merge module to perform feature disentanglement on the lesion boundary features to obtain multiple lesion boundary feature blocks; S1322. Labeling multiple lesion boundary feature blocks and a lesion region feature block, and embedding a class label feature block to obtain labeled lesion feature blocks; S1323. Using a multi-head self-attention module to obtain aggregation features using the labeled lesion feature blocks.

[0079] Specifically, as Figure 4 shown in the lower half of, two stacked self-attention blocks are adopted during the integration process, and a multi-head mechanism is used therein. Multi-headed Self-attention - a multi-head self-attention structure, a standard algorithm structure. Figure 4 corresponds to Figure 2 the relation module in the upper right. The outputs of the two coordinate models are combined to output the final B / M (benign / malignant).

[0080] In the self-attention structure, five feature blocks used for qualitative sub-tasks in the polar coordinate feature model are tokenized, including a clear edge feature block token, a burr feature block token, an angled edge feature block token, a smooth edge feature block token, and an ultrasound image feature block token. In addition, all features in the encoder in the Cartesian coordinate system model are compressed and tokenized, and an additional token is introduced to represent class information. The lesion dot matrix annotation information of the second coordinate model includes a benign / malignant classification token, a clear edge feature block token, a burr feature block token, an angled edge feature block token, a smooth edge feature block token, and an ultrasound image feature block token. Among them, the clear edge feature block token, the burr feature block token, the angled edge feature block token, the smooth edge feature block token, and the ultrasound image feature block token are tokens of the original five feature blocks. The newly added one is the benign / malignant classification token.

[0081] S140. Classify and identify the lesion aggregation features to obtain the disease identification result of the patient; wherein, the disease identification result includes the determination of the benign / malignant category of the lesion and the lesion location. The lesion location may further include whether the lesion shows clear edges, whether there are burrs, whether the edges are angled, whether the edges are smooth, and whether there is an ultrasound contrast performance.

[0082] Train a thyroid nodule classifier using the final classification features of the thyroid nodule to obtain a dual - coordinate disease recognition model; the dual - coordinate disease recognition model is trained and constrained by a loss function. Specifically, obtain the score distribution of the benign - malignant evaluation of the disease and the score distribution of the lesion location evaluation for the input sample containing the labeled information of the lesion area through the dual - coordinate disease recognition model; obtain the loss function between the score distribution of the benign - malignant evaluation of the disease and the original score distribution of the benign - malignant evaluation of the disease corresponding to the sample image dataset, and the loss function between the score distribution of the lesion location evaluation and the original score distribution of the disease lesion location evaluation corresponding to the sample image dataset; update the network parameters of the dual - coordinate disease recognition model according to the loss function until the mean square error of the score distribution of the benign - malignant evaluation of the disease and the mean square error of the score distribution of the lesion location evaluation both fall within the preset standard range.

[0083] In this embodiment, the Dice loss function and the mean square error loss function are used to evaluate the coincidence degree between the prediction result of the dual - coordinate disease recognition model and the label; the Adam algorithm is used to optimize the cross - entropy loss function, and the Focal loss function is used to update the network parameters of the dual - coordinate disease recognition model.

[0084] The training method of the dual - coordinate disease recognition model further includes establishing a disease scoring system using binary logistic regression analysis; based on the disease scoring system, using the stepwise regression method to screen the independent variables that are correlated with the benign - malignant evaluation of the disease; using the independent variables that are correlated with the benign - malignant evaluation of the disease to determine the correlation between the predicted value of the dual - coordinate disease recognition model and the benign - malignancy of the disease. That is to say, the independent variables with statistical significance are further analyzed by binary logistic regression to determine whether the predicted value is correlated with the benign or malignant nature of the disease. The results show that the predicted value of the dual - coordinate disease recognition model is indeed closely related to the benign - malignant classification task, which reflects that this method must be interpretable.

[0085] Specifically, use the gradient backpropagation algorithm to train the joint model (i.e., the dual - coordinate disease recognition model). Use the cosine annealing learning rate strategy with warmup to ensure the stability and fast convergence of the training process. Specifically, set warmup to 5 epochs, cosine annealing to 200 epochs, the maximum learning rate to 0.001, and the minimum learning rate to 0.0000001. During the training process, first train the models of the two branches of the polar coordinate feature model and the Cartesian coordinate feature model, and then fix the weights after 100 epochs and start training the integration module based on the attention mechanism, that is, the dual - coordinate disease recognition model. In terms of the loss function, use the focal loss for the classification variable and vector regression tasks, and use the dice loss for the segmentation task.

[0086] In addition, for performance evaluation metrics, two methods of quantitative and qualitative analysis are selected. For the main task, common metrics such as the area under the receiver operating characteristic curve (AUC), accuracy (ACC), sensitivity (SEN), and specificity (SPC) are used. For the vector regression task, the 1D Dice similarity coefficient (DSC) and mean squared error (MSE) are used as evaluation metrics, and for the segmentation task, the 2D DSC is used as the evaluation metric.

[0087] In a specific embodiment, in the thyroid nodule dataset used to train the dual - coordinate disease recognition model, the annotation situations of the training set and validation set of the dataset are shown in Table 1, and the sample numbers of the training set, validation set, and test set are shown in Table 2.

[0088] Table 1: Annotation situations of the training set and validation set

[0089]

[0090] Table 2: Sample numbers of the training set, validation set, and test set

[0091]

[0092] Under different coordinate systems, the dual - coordinate disease recognition model trained with the above - mentioned dataset and a general CNN network model are tested for performance using the test set, and the test results are shown in Table 3.

[0093] Table 3: Performance situations of the present invention and a general CNN network model on the test set

[0094]

[0095] By observing Table 3, it can be seen that under different coordinate systems, the performance of the general CNN network model is low, while the dual - coordinate disease recognition model of the present invention, after combining domain knowledge and the fusion module, achieves optimal performance and has statistically significant characteristics.

[0096] The dual - coordinate disease recognition model trained with the above - mentioned dataset is applied to the scenario of benign and malignant assessment of thyroid nodules, and the comparison results are made with the assessment results of doctors with different years of experience, and the comparison results are shown in Table 4.

[0097] Table 4: Comparison situations of the assessment results of the present invention and doctors with different years of experience

[0098]

[0099] By observing Table 4, it can be seen that the assessment performance score of the present invention is higher than that of general doctors.

[0100] Due to the cost of annotation, there is little verified data with fine-grained annotations, and it is necessary to further confirm whether the auxiliary task has really learned more meaningful feature representations. Therefore, in this embodiment, by imitating the authoritative TI-RADS, a benign scoring system is established using the inference results of the auxiliary task to indirectly prove the accuracy and importance of the prediction of the auxiliary task.

[0101] If the prediction results of the auxiliary task are significantly correlated with benignancy and malignancy, then it is equivalent to that this method not only predicts the main task, but also provides more understandable evidence, thereby enhancing interpretability. Logistic regression is performed by using qualitative and quantitative inferences on the data in the un-finely labeled training set from the auxiliary task. Then, the optimized logistic regression is used as the scoring method to test the benignancy and malignancy classification of the test set. The test results are shown in Table 5.

[0102]

[0103] By observing Table 5, it can be seen that the prediction results of the auxiliary task of the disease intelligent recognition method based on fine-grained domain knowledge of the present invention are indeed related to the main label. Therefore, it is proved that the disease intelligent recognition method based on fine-grained domain knowledge of the present invention has interpretability.

[0104] In summary, the disease intelligent recognition method based on fine-grained domain knowledge of the present invention uses the domain knowledge of thyroid ultrasound diagnosis based on regions and the domain knowledge of thyroid ultrasound diagnosis based on boundaries to establish a polar coordinate feature model and a Cartesian coordinate feature model, extracts features in two different coordinate systems, and then obtains the determination result of the thyroid nodule; the determination result includes not only the determination of benignancy and malignancy, but also various fine-grained information such as the location of the lesion and the manifestation of the lesion, making the fine-grained information of the thyroid nodule classification variable interpretable. It is convenient to assist doctors in making qualitative and even quantitative analyses of thyroid lesions and other regions of interest, thereby greatly improving the accuracy and reliability of medical diagnosis; it can also play an important auxiliary role in medical teaching, surgical planning, surgical simulation, and various medical research.

[0105] As Figure 5 shown, the present invention provides a disease intelligent recognition system 500 based on fine-grained domain knowledge, and the present invention can be installed in an electronic device. According to the functions achieved, the disease intelligent recognition system 500 based on fine-grained domain knowledge can include a feature extraction unit 510, a feature aggregation unit 520, and an identification unit 530. The units of the present invention can also be referred to as modules, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0106] In this embodiment, the functions of each module / unit are as follows:

[0107] A feature extraction unit 510 is configured to obtain a patient lesion image to be recognized; use a dual - coordinate disease recognition model to extract features from the patient lesion image to obtain lesion area features and lesion boundary features; wherein, the dual - coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition;

[0108] A feature aggregation unit 520 is configured to obtain lesion aggregation features according to the lesion area features and the lesion boundary features through a self - attention network;

[0109] An identification unit 530 is configured to classify and identify the lesion aggregation features to obtain a disease recognition result of the patient; wherein, the disease recognition result includes a determination of the benign / malignant category of the lesion and the location of the lesion.

[0110] The disease intelligent recognition system 500 based on fine - grained domain knowledge of the present invention obtains a patient lesion image to be recognized; uses a dual - coordinate disease recognition model to extract features from the patient lesion image to obtain lesion area features and lesion boundary features; wherein, the dual - coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition; obtains lesion aggregation features according to the lesion area features and the lesion boundary features through a self - attention network; classifies and identifies the lesion aggregation features to obtain a disease recognition result of the patient; wherein, the disease recognition result includes a determination of the benign / malignant category of the lesion and the location of the lesion, making the fine - grained information of the thyroid nodule classification variable interpretable; realizes the effect of facilitating the doctor to qualitatively and even quantitatively analyze the thyroid lesion and other regions of interest, and greatly improves the accuracy and reliability of medical diagnosis; also plays an important auxiliary role in medical teaching, surgical planning, surgical simulation and various medical researches.

[0111] As Figure 6 shown, the present invention provides an electronic device 6 for a disease intelligent recognition method based on fine - grained domain knowledge.

[0112] The electronic device 6 may include a processor 60, a memory 61 and a bus, and may also include a computer program stored in the memory 61 and executable on the processor 60, such as a disease intelligent recognition program 62 based on fine - grained domain knowledge. The memory 61 may also include both an internal storage unit of the disease intelligent recognition system based on fine - grained domain knowledge and an external storage device. The memory 61 can be used not only to store installed application software and various data, such as the code of the disease intelligent recognition program based on fine - grained domain knowledge, etc., but also to temporarily store data that has been output or will be output.

[0113] Among them, the memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as the mobile hard disk of the electronic device 6. In some other embodiments, the memory 61 may also be an external storage device of the electronic device 6, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device 6. Further, the memory 61 may also include both the internal storage unit of the electronic device 6 and the external storage device. The memory 61 can be used not only to store application software installed in the electronic device 6 and various types of data, such as the code of the disease intelligent recognition program based on fine-grained domain knowledge, etc., but also to temporarily store data that has been output or will be output.

[0114] In some embodiments, the processor 60 may be composed of integrated circuits. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same or different functions, including a combination of one or more Central Processing Units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips, etc. The processor 60 is the control core (Control Unit) of the electronic device, connecting various components of the entire electronic device through various interfaces and lines, and by running or executing programs or modules stored in the memory 61 (such as the disease intelligent recognition program based on fine-grained domain knowledge, etc.), and calling the data stored in the memory 61, to execute various functions of the electronic device 6 and process data.

[0115] The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is set to realize the connection and communication between the memory 61 and at least one processor 60, etc.

[0116] Figure 6 Only the electronic device with components is shown. Those skilled in the art can understand that Figure 6The structures shown do not constitute a limitation on the electronic device 6, and may include fewer or more components than those shown, or combine certain components, or have different component arrangements.

[0117] For example, although not shown, the electronic device 6 may further include a power source (such as a battery) for powering each component. Preferably, the power source may be logically connected to the at least one processor 60 through a power management system, so as to implement functions such as charge management, discharge management, and power consumption management through the power management system. The power source may also include any components such as one or more DC or AC power sources, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator. The electronic device 6 may also include a variety of sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be elaborated here.

[0118] Furthermore, the electronic device 6 may further include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 6 and other electronic devices.

[0119] Optionally, the electronic device 6 may further include a user interface. The user interface may be a display, an input unit (such as a keyboard), and optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display the information processed in the electronic device 6 and to display a visual user interface.

[0120] It should be understood that the above embodiments are only for illustration purposes and are not limited by this structure in the scope of the patent application.

[0121] The disease intelligent recognition program 62 based on fine-grained domain knowledge stored in the memory 61 of the electronic device 6 is a combination of multiple instructions. When running in the processor 60, it can achieve: obtaining a patient lesion image to be recognized; using a dual-coordinate disease recognition model to extract features from the patient lesion image to obtain lesion area features and lesion boundary features; wherein, the dual-coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition; through a self-attention network, obtaining lesion aggregation features according to the lesion area features and the lesion boundary features; classifying and recognizing the lesion aggregation features to obtain a disease recognition result of the patient; wherein, the disease recognition result includes a determination of the benign / malignant category of the lesion and the location of the lesion.

[0122] Specifically, the specific implementation method of the processor 60 for the above instructions can refer to Figure 1 the description of the relevant steps in the corresponding embodiment, which will not be elaborated here. It should be emphasized that to further ensure the privacy and security of the above disease intelligent recognition program based on fine-grained domain knowledge, the above database highly available processed data is stored in the nodes of the blockchain where the server cluster is located.

[0123] Furthermore, if the modules / units integrated in the electronic device 6 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM, Read-Only Memory).

[0124] The embodiment of the present invention also provides a computer-readable storage medium. The storage medium may be non-volatile or volatile. The storage medium stores a computer program, and when the computer program is executed by a processor, it can achieve: obtaining a patient lesion image to be recognized; using a dual-coordinate disease recognition model to extract features from the patient lesion image to obtain lesion area features and lesion boundary features; wherein, the dual-coordinate disease recognition model includes a first coordinate model for lesion area division and a second coordinate model for lesion boundary feature recognition; through a self-attention network, obtaining lesion aggregation features according to the lesion area features and the lesion boundary features; classifying and recognizing the lesion aggregation features to obtain a disease recognition result of the patient; wherein, the disease recognition result includes a determination of the benign / malignant category of the lesion and the location of the lesion.

[0125] Specifically, the specific implementation method when the computer program is executed by the processor can refer to the description of the relevant steps in the disease intelligent recognition method based on fine-grained domain knowledge in the embodiment, which will not be elaborated here.

[0126] In several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there can be other division methods in actual implementation.

[0127] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, in each embodiment of the present invention, the functional modules can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a combination of hardware and software functional modules.

[0129] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms.

[0130] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any associated drawing marks in the claims should not be regarded as limiting the claimed rights.

[0131] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithms. Blockchain, essentially a decentralized database, is a string of data blocks generated by using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity (anti-counterfeiting) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer, etc.

[0132] In addition, obviously, the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or systems stated in the system claims can also be implemented by one unit or system through software or hardware. Words such as second are used to denote names and do not denote any particular order.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A disease intelligent recognition method based on fine-grained domain knowledge, characterized in that, Including: Obtain the patient's lesion image to be recognized; Use a dual - coordinate disease recognition model to extract features from the patient's lesion image, and obtain lesion region features and lesion boundary features; wherein, the dual - coordinate disease recognition model includes a first coordinate model for lesion region division and a second coordinate model for lesion boundary feature recognition; the training method of the dual - coordinate disease recognition model includes pre - processing samples containing lesion region annotation information to obtain a lesion region mask and a normal region mask; obtaining a trained first coordinate model based on the lesion region mask and the normal region mask; pre - processing samples containing lesion dot - matrix annotation information to obtain a vectorized representation of the lesion dot - matrix; obtaining a trained second coordinate model based on the vectorized representation of the lesion dot - matrix; wherein, the first coordinate model is a Cartesian coordinate model; the second coordinate model is a polar coordinate model; integrating the trained first coordinate model and the second coordinate model based on a multi - head self - attention module to obtain a dual - coordinate disease recognition model; training the dual - coordinate disease recognition model based on a loss function using the gradient backpropagation algorithm until convergence; Through a self - attention network, obtain lesion aggregation features according to the lesion region features and the lesion boundary features; wherein, the method of obtaining lesion aggregation features according to the lesion region features and the lesion boundary features through a self - attention network includes de - entangling the lesion boundary features using a split - merge module to obtain multiple lesion boundary feature blocks; marking the multiple lesion boundary feature blocks and the lesion region feature block, and embedding a class - label feature block to obtain a marked lesion feature block; using a multi - head self - attention module to obtain aggregation features using the marked lesion feature block; Classify and recognize the lesion aggregation features to obtain the disease recognition result of the patient; wherein, the disease recognition result includes the determination of the benign / malignant category of the lesion and the lesion location.

2. The disease intelligent recognition method based on fine-grained domain knowledge according to claim 1, characterized in that, The method of obtaining lesion aggregation features according to the lesion region features and the lesion boundary features through a self - attention network includes: Fuse the lesion region features and the lesion boundary features to obtain an original fusion feature; After passing the original fusion feature through a convolutional layer and an activation function, obtain the significance weight feature corresponding to the input feature; Multiply the significance weight feature corresponding to the input feature by the original feature to obtain an aggregation feature.

3. The disease intelligent recognition method based on fine-grained domain knowledge according to claim 1, characterized in that, The method of training the dual - coordinate disease recognition model based on a loss function using the gradient backpropagation algorithm until convergence includes: Obtain the score distribution of the disease benign - malignant evaluation and the score distribution of the lesion location evaluation of the input samples containing lesion region annotation information through the dual - coordinate disease recognition model; Obtain the loss function between the score distribution of the disease benign - malignant evaluation and the original disease benign - malignant evaluation score distribution corresponding to the sample image dataset, and the loss function between the score distribution of the lesion location evaluation and the original disease lesion location evaluation score distribution corresponding to the sample image dataset; Update the network parameters of the dual - coordinate disease recognition model according to the loss function until the mean square error of the score distribution of the benign - malignant evaluation of the disease and the mean square error of the score distribution of the lesion location evaluation both fall within the preset standard range.

4. The disease intelligent recognition method based on fine-grained domain knowledge according to claim 1, characterized in that, The lesion dot - matrix annotation information includes benign - malignant feature block markers, clear - edge feature block markers, spiculation feature block markers, angulated - edge feature block markers, smooth - edge feature block markers, and ultrasonic image feature block markers.

5. The disease intelligent recognition method based on fine-grained domain knowledge according to claim 1, characterized in that, The training method of the dual - coordinate disease recognition model further includes Establish a disease scoring system using binary logistic regression analysis; Based on the disease scoring system, use step - wise regression to screen the independent variables that are correlated with the benign - malignant evaluation of the disease; Use the independent variables that are correlated with the benign - malignant evaluation of the disease to determine the correlation between the predicted value of the dual - coordinate disease recognition model and the benign - malignancy of the disease.

6. A disease intelligent recognition system based on fine-grained domain knowledge, characterized in that, It includes: A feature extraction unit, configured to obtain the patient's lesion image to be recognized; use the dual - coordinate disease recognition model to extract features from the patient's lesion image to obtain lesion area features and lesion boundary features; wherein, the dual - coordinate disease recognition model includes a first - coordinate model for lesion area division and a second - coordinate model for lesion boundary feature recognition; the training method of the dual - coordinate disease recognition model includes pre - processing a sample containing lesion area annotation information to obtain a lesion area mask and a normal area mask; obtaining the trained first - coordinate model based on the lesion area mask and the normal area mask; pre - processing a sample containing lesion dot - matrix annotation information to obtain a vectorized representation of the lesion dot - matrix; obtaining the trained second - coordinate model based on the vectorized representation of the lesion dot - matrix; wherein, the first - coordinate model is a Cartesian coordinate model; the second - coordinate model is a polar coordinate model; integrating the trained first - coordinate model and the second - coordinate model based on a multi - head self - attention module to obtain the dual - coordinate disease recognition model; training the dual - coordinate disease recognition model using the gradient back - propagation algorithm based on the loss function until convergence; A feature aggregation unit, configured to obtain lesion aggregation features according to the lesion area features and the lesion boundary features through a self - attention network; wherein, the method of obtaining lesion aggregation features according to the lesion area features and the lesion boundary features through a self - attention network includes de - entangling the lesion boundary features using a split - merge module to obtain multiple lesion boundary feature blocks; marking the multiple lesion boundary feature blocks and the lesion area feature blocks and embedding class - label feature blocks to obtain marked lesion feature blocks; using a multi - head self - attention module to obtain aggregation features using the marked lesion feature blocks; An identification unit, configured to classify and identify the lesion aggregation features to obtain the disease recognition result of the patient; wherein, the disease recognition result includes the determination of the benign / malignant category of the lesion and the lesion location.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps in the method for intelligent disease recognition based on fine-grained domain knowledge according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for intelligent disease recognition based on fine-grained domain knowledge according to any one of claims 1 to 5.

Images