Intelligent diagnosis auxiliary system and method for pediatric respiratory diseases

Abstract

The invention relates to the field of intelligent auxiliary diagnosis, and particularly discloses an intelligent auxiliary diagnosis system and method for pediatric respiratory diseases, and the method comprises the steps: firstly collecting the doctor inquiry record and auxiliary examination information of a child, inputting the information into a typical symptom diagnosis model for processing, then enabling the model to generate an auxiliary diagnosis result, the results are obtained based on comprehensive analysis, and the auxiliary diagnosis results are displayed in an information display interface. Therefore, the auxiliary diagnosis result can be automatically generated, decision support is provided for doctors, and accuracy and efficiency of diagnosis are ensured.

Description

Technical Field

[0001] The present application relates to the field of intelligent assisted diagnosis, and more specifically, to an intelligent assisted diagnosis system and method for pediatric respiratory diseases. Background Art

[0002] Pediatric respiratory diseases (such as asthma, pneumonia, bronchitis, etc.) are diverse, changeable and complex, and early diagnosis and precise treatment are crucial to improving the prognosis of children. Patent CN109935336B proposes an intelligent auxiliary diagnosis system for pediatric respiratory diseases, which collects doctor's consultation records and obtains auxiliary examination information when necessary, and screens key information, and then uses the children's disease knowledge base and typical symptom diagnosis model analysis to identify possible diseases and their characteristics and present diagnostic results.

[0003] The method in the patent mainly relies on the integration of doctor's consultation record information and auxiliary examination information, and then screens the key information and inputs it into the typical symptom diagnosis model for disease prediction. If the model only relies on pre-defined key information for diagnosis, when these keywords appear less frequently in the data set, it will make it difficult for the model to obtain enough features to learn effective patterns, thus affecting the accuracy of the diagnosis. In addition, the existing patents process the consultation records and auxiliary examination information independently and directly input them into the diagnosis model without deeply exploring the complex correlation between the two, thus affecting the comprehensiveness of the diagnosis. In addition, the typical symptom diagnosis model is trained based on a pre-established children's disease knowledge base system, which shows that the performance of the model depends largely on the information and update speed of the knowledge base. If the knowledge base fails to promptly incorporate the latest medical research results or fails to cover the regional disease characteristics of a specific region, the applicability and accuracy of the model may be affected.

[0004] Therefore, an optimized intelligent diagnostic assistance solution for pediatric respiratory diseases is desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed.

[0006] According to one aspect of the present application, an intelligent diagnosis assistance system for pediatric respiratory diseases is provided, which includes: a consultation record acquisition module, used to obtain doctor consultation record information of a child to be diagnosed; an auxiliary information acquisition module, used to obtain auxiliary examination information of the child to be diagnosed; a result generation module, used to input the doctor consultation record information and the auxiliary examination information into a typical symptom diagnosis model for processing to generate an auxiliary diagnosis result; a result display module, used to display the auxiliary diagnosis result in an information display interface, wherein the result generation module includes: A medical consultation record semantic coding unit, used for semantically coding the doctor's medical consultation record information to obtain a semantic coding feature of the doctor's medical consultation record; An auxiliary inspection information processing unit, used for respectively performing semantic feature extraction and visual feature extraction on the auxiliary inspection information to obtain auxiliary inspection image grid semantic features and auxiliary inspection image visual semantic features; an auxiliary inspection information multi-dimensional encoding unit, used for performing auxiliary inspection information communication based on a cross attention network on the auxiliary inspection image grid semantic features and the auxiliary inspection image visual semantic features to obtain a multi-dimensional semantic representation of the auxiliary inspection image; The consultation auxiliary feature interaction unit is used to perform fine-grained interaction of external knowledge features on the semantic coding features of the doctor's consultation record and the multi-dimensional semantic representation of the auxiliary examination image to obtain fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features, and obtain the auxiliary diagnosis result based on the fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features.

[0007] Furthermore, the medical consultation record semantic encoding unit is used to: pass the doctor's consultation record information through a semantic encoder including a Bert model to obtain a doctor's consultation record semantic encoding feature vector as the doctor's consultation record semantic encoding feature.

[0008] Furthermore, the auxiliary inspection information processing unit includes: An auxiliary inspection image grid semantic extraction subunit is used to grid the auxiliary inspection information and then pass it through an auxiliary inspection image grid semantic feature extractor based on a CNN-LSTM model to obtain an auxiliary inspection image grid semantic feature vector as the auxiliary inspection image grid semantic feature; The auxiliary inspection image visual semantic feature extraction subunit is used to pass the auxiliary inspection information through a visual feature extractor based on the FCN model to obtain an auxiliary inspection image visual semantic feature vector as the auxiliary inspection image visual semantic feature.

[0009] Furthermore, the auxiliary inspection information multi-dimensional encoding unit is used to: Performing a linear transformation on the auxiliary inspection image grid semantic feature vector to obtain an auxiliary inspection image grid semantic query vector, an auxiliary inspection image grid semantic key vector, and an auxiliary inspection image grid semantic value vector; Performing a linear transformation on the auxiliary inspection image visual semantic feature vector to obtain an auxiliary inspection image visual semantic query vector, an auxiliary inspection image visual semantic key vector, and an auxiliary inspection image visual semantic value vector; Inputting the auxiliary inspection image grid semantic query vector, the auxiliary inspection image visual semantic key vector and the auxiliary inspection image visual semantic value vector into the attention feature interaction module of the converter model to obtain an image vision guided auxiliary inspection image grid semantic attention interaction feature vector; Inputting the auxiliary inspection image visual semantic query vector, the auxiliary inspection image grid semantic key vector and the auxiliary inspection image grid semantic value vector into the attention feature interaction module of the converter model to obtain a grid semantic guided auxiliary inspection image visual semantic attention interaction feature vector; The image vision-guided auxiliary inspection image grid semantic attention interaction feature vector and the grid semantics-guided auxiliary inspection image visual semantic attention interaction feature vector are fused to obtain an auxiliary inspection image multidimensional semantic representation vector as the auxiliary inspection image multidimensional semantic representation.

[0010] Furthermore, the medical consultation auxiliary feature interaction unit includes: An external knowledge feature fine-grained interaction subunit is used to perform external knowledge feature fine-grained interaction on the semantic encoding feature vector of the doctor's consultation record and the multi-dimensional semantic representation vector of the auxiliary examination image to obtain a fine-grained interaction feature vector between the doctor's consultation and auxiliary examination semantic features as the fine-grained interaction feature between the doctor's consultation and auxiliary examination semantic features; The auxiliary diagnosis result generating subunit is used to obtain the auxiliary diagnosis result according to the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features.

[0011] Furthermore, the external knowledge feature fine-grained interaction sub-unit includes: The medical consultation record-auxiliary examination fine-grained interaction secondary subunit is used to perform fine-grained feature interaction on the semantic encoding feature vector of the doctor's medical consultation record and the multi-dimensional semantic representation vector of the auxiliary examination image to obtain a medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix; The external knowledge optimization secondary sub-unit is used to perform external knowledge attention optimization on the medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix to obtain the medical consultation record-auxiliary examination semantic external knowledge optimization fine-grained feature interaction matrix; The modulation and fusion secondary sub-unit is used to optimize the fine-grained feature interaction matrix according to the medical consultation record-auxiliary examination semantic external knowledge, and to modulate and fuse the doctor's medical consultation record semantic encoding feature vector and the auxiliary examination image multidimensional semantic representation vector to obtain the fine-grained interaction feature vector between the doctor's medical consultation-auxiliary examination semantic features.

[0012] Furthermore, the modulation fusion secondary subunit is used to: The semantic coding feature vector of the doctor's consultation record is linearly transformed to obtain a consultation record query feature vector and a consultation record value feature vector, and the consultation record-auxiliary examination semantic external knowledge optimized fine-grained feature interaction matrix is ​​used as a key matrix, and the consultation record query feature vector, the consultation record value feature vector and the key matrix are modulated based on the Transformer structure to obtain an optimized semantic coding feature vector of the doctor's consultation record; Performing a linear transformation on the auxiliary inspection image multidimensional semantic representation vector to obtain an auxiliary inspection query feature vector and an auxiliary inspection value feature vector, and using the medical inquiry record-auxiliary inspection semantic external knowledge optimized fine-grained feature interaction matrix as a key matrix, performing the Transformer structure-based modulation on the auxiliary inspection query feature vector, the auxiliary inspection value feature vector and the key matrix to obtain an optimized auxiliary inspection image multidimensional semantic representation vector; The point-by-point division between the optimized semantic encoding feature vector of the doctor's consultation record and the optimized multi-dimensional semantic representation vector of the auxiliary examination image is calculated to obtain a fine-grained interaction feature vector between the semantic features of the doctor's consultation and the auxiliary examination.

[0013] Furthermore, the auxiliary diagnosis result generating subunit is used to: input the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features into the trained typical symptom diagnosis model to obtain the auxiliary diagnosis result.

[0014] According to another aspect of the present application, an intelligent diagnosis assistance method for pediatric respiratory diseases is provided, comprising: obtaining doctor's consultation record information of a child to be diagnosed; obtaining auxiliary examination information of the child to be diagnosed; inputting the doctor's consultation record information and the auxiliary examination information into a typical symptom diagnosis model for processing to generate an auxiliary diagnosis result; and displaying the auxiliary diagnosis result in an information display interface, wherein the doctor's consultation record information and the auxiliary examination information are input into the typical symptom diagnosis model for processing to generate an auxiliary diagnosis result, including: Performing semantic coding on the doctor's consultation record information to obtain a semantic coding feature of the doctor's consultation record; Respectively performing semantic feature extraction and visual feature extraction on the auxiliary inspection information to obtain auxiliary inspection image grid semantic features and auxiliary inspection image visual semantic features; Performing auxiliary inspection information exchange based on a cross attention network on the auxiliary inspection image grid semantic features and the auxiliary inspection image visual semantic features to obtain a multi-dimensional semantic representation of the auxiliary inspection image; Fine-grained interaction of external knowledge features is performed on the semantic coding features of the doctor's consultation record and the multidimensional semantic representation of the auxiliary examination image to obtain fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features, and the auxiliary diagnosis result is obtained based on the fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features.

[0015] Compared with the prior art, the present application provides an intelligent diagnostic assistance system and method for pediatric respiratory diseases, which uses natural language analysis technology and image processing algorithms based on deep learning to semantically encode the doctor's consultation record information, and extracts the auxiliary examination information and visual features respectively, and performs cross-attention multi-dimensional fusion of the auxiliary examination image grid semantic features and the auxiliary examination image visual semantic features, so as to automatically generate auxiliary diagnosis results based on the external knowledge fine-grained interaction features between the semantic encoding features of the doctor's consultation record and the multi-dimensional semantic representation of the auxiliary examination image. Compared with simple keyword matching, the present application can more comprehensively understand the patient's symptom description and auxiliary information, and consider the inherent connection between multiple information sources, so as to draw a more comprehensive and accurate diagnosis conclusion. At the same time, the features in the external knowledge base are used to guide the diagnosis process, so that the model can benefit from the latest medical research results, thereby maintaining a high diagnostic accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 4 is a block diagram of an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application.

[0018] Figure 2 It is a block diagram of a result generation module in an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application.

[0019] Figure 3 The present invention is a block diagram of an auxiliary examination information processing unit in an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application.

[0020] Figure 4 It is a block diagram of a consultation assistance feature interaction unit in an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application.

[0021] Figure 5The figure is a flow chart of an intelligent diagnosis assistance method for pediatric respiratory diseases according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0023] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0024] It is worth noting that in this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0025] Therefore, in response to the problems in the above-mentioned background technology, the present application proposes an intelligent diagnosis assistance system for pediatric respiratory diseases. Figure 1 FIG. 1 is a block diagram of an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application. Specifically, Figure 1 As shown, according to an embodiment of the present application, the intelligent diagnosis assistance system 100 for pediatric respiratory diseases includes: a consultation record acquisition module 110, used to obtain the doctor's consultation record information of the child to be diagnosed; an auxiliary information acquisition module 120, used to obtain the auxiliary examination information of the child to be diagnosed; a result generation module 130, used to input the doctor's consultation record information and the auxiliary examination information into a typical symptom diagnosis model for processing to generate an auxiliary diagnosis result; a result display module 140, used to display the auxiliary diagnosis result in an information display interface.

[0026] Specifically, the consultation record acquisition module 110 is used to obtain the doctor's consultation record information of the child to be diagnosed. It should be understood that the doctor's consultation record information includes: auxiliary examination information of non-expert doctors, recorded family history information, and allergy history information. This information is the basis of diagnosis and can help the system to more comprehensively understand the health status of the child and possible risk factors. Therefore, in order to provide more accurate child diagnosis information and improve the accuracy of subsequent result generation, in the technical solution of the present application, the doctor's consultation record information of the child to be diagnosed is obtained. In this way, by analyzing detailed consultation records, the intelligent system can provide more personalized diagnostic suggestions. For example, if there are certain hereditary diseases in the family history, the possibility of being related to these genetic factors may be given priority when considering pediatric respiratory diseases.

[0027] In particular, the electronic health record (EHR) system of a hospital or clinic plays a central role. The EHR system typically stores the patient's entire medical history, including but not limited to medical history, medication, allergic reactions, previous surgical experience, etc. When a child comes for a visit, the doctor will create a new consultation entry in the EHR system, recording the specific information of the visit, such as the chief symptoms, physical examination findings, initial impressions, and diagnostic and treatment measures taken. In order to ensure the quality of the information, doctors need to fill in these contents according to a certain structured template, which helps to improve the consistency and readability of the data. Next, the intelligent auxiliary diagnosis system will be connected to the EHR system to automatically extract relevant consultation records through the API interface.

[0028] Specifically, the auxiliary information acquisition module 120 is used to obtain the auxiliary examination information of the child to be diagnosed. Accordingly, the auxiliary examination information includes: pathological examination information, ultrasound auxiliary examination information, X-ray auxiliary examination information, CT, nuclear magnetic resonance auxiliary examination information, etc. In particular, auxiliary examinations (such as pathological examinations, ultrasound, X-rays, CT, nuclear magnetic resonance, etc.) can provide intuitive and objective medical images or laboratory data, which can serve as an important basis for diagnosis. For example, chest X-rays can help identify problems such as pneumonia and pneumothorax, while CT or nuclear magnetic resonance imaging can show the internal structure in more detail and help to find more complex lesions. Therefore, in order to assist the doctor's consultation record information to make a more accurate diagnosis of the child, in the technical solution of the present application, the auxiliary examination information of the child to be diagnosed is obtained. That is, the auxiliary examination information can verify the symptoms mentioned in the consultation and supplement those situations that are difficult to confirm by consultation alone. For example, a history of allergies may indicate certain types of respiratory problems, but the specific lung condition still needs to be determined by imaging examinations.

[0029] Specifically, when the doctor determines that further auxiliary examinations are needed based on the initial consultation, the corresponding examination form will be issued. The hospital's internal information management system will automatically record these requests and notify the relevant departments to prepare for the examination. For example, during a pathological examination, laboratory technicians will collect tissue or cell samples and observe their morphological changes under a microscope to look for possible pathological characteristics. The examination results will be stored in the hospital's electronic health record (EHR) system in the form of a report, and the generated image data will also be digitized and saved for subsequent analysis.

[0030] For imaging tests, such as ultrasound, X-rays, CT and MRI, patients will be arranged to receive the examination in a special department. The images produced by these devices not only reflect the state of the internal structure of the human body, but also reveal specific signs of certain diseases. Take ultrasound as an example. It uses high-frequency sound waves to create real-time images of organs in the body, which is particularly suitable for observing changes in soft tissue. X-rays can clearly show the condition of bone structures and parts such as the lungs, and are often used to detect problems such as fractures or pneumonia. CT scans combine X-ray images from multiple angles to build a more detailed three-dimensional anatomical model, while MRI, with its powerful magnetic field and radio wave technology, is good at showing areas with high soft tissue contrast, such as brain or spinal cord lesions.

[0031] In order to ensure that all auxiliary examination information can be effectively used by the intelligent diagnosis system, a close data interface is established between the hospital information system and the diagnosis platform. This enables the original images and explanatory reports to be quickly transmitted to the database of the diagnosis system after the examination is completed.

[0032] Specifically, the result generation module 130 is used to input the doctor's consultation record information and the auxiliary examination information into the typical symptom diagnosis model for processing to generate an auxiliary diagnosis result. Therefore, in order to integrate multi-source medical data to provide a more accurate, fast and personalized diagnosis support tool, thereby improving the accuracy and efficiency of early diagnosis of pediatric respiratory diseases, in the result generation module, the technical concept of the present application is to use natural language analysis technology and image processing algorithms based on deep learning to semantically encode the doctor's consultation record information, and extract the auxiliary examination information and visual features respectively, and perform cross-attention multi-dimensional fusion of the auxiliary examination image grid semantic features and the auxiliary examination image visual semantic features, so as to automatically generate auxiliary diagnosis results based on the external knowledge fine-grained interaction features between the semantic encoding features of the doctor's consultation record and the multi-dimensional semantic representation of the auxiliary examination image. Compared with simple keyword matching, the present application can more comprehensively understand the patient's symptom description and auxiliary information, and consider the intrinsic connection between multiple information sources, so as to draw a more comprehensive and accurate diagnosis conclusion. At the same time, features in the external knowledge base are used to guide the diagnosis process, so that the model can benefit from the latest medical research results and maintain a high diagnostic accuracy.

[0033] Figure 2 FIG. 1 is a block diagram of a result generation module in an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application. Specifically, Figure 2 As shown, the result generation module 130 includes: a medical consultation record semantic encoding unit 131, which is used to perform semantic encoding on the doctor's medical consultation record information to obtain the semantic encoding features of the doctor's medical consultation record; an auxiliary examination information processing unit 132, which is used to perform semantic feature extraction and visual feature extraction on the auxiliary examination information to obtain auxiliary examination image grid semantic features and auxiliary examination image visual semantic features; an auxiliary examination information multidimensional encoding unit 133, which is used to perform auxiliary examination information exchange based on a cross-attention network on the auxiliary examination image grid semantic features and the auxiliary examination image visual semantic features to obtain the auxiliary examination image multidimensional semantic representation; a medical consultation auxiliary feature interaction unit 134, which is used to perform external knowledge feature fine-grained interaction on the doctor's medical consultation record semantic encoding features and the auxiliary examination image multidimensional semantic representation to obtain fine-grained interaction features between doctor's medical consultation and auxiliary examination semantic features, and obtain the auxiliary diagnosis result based on the fine-grained interaction features between doctor's medical consultation and auxiliary examination semantic features.

[0034] In an embodiment of the present application, the medical consultation record semantic encoding unit 131 is used to semantically encode the doctor's consultation record information to obtain the doctor's consultation record semantic encoding features. Specifically, in an embodiment of the present application, the medical consultation record semantic encoding unit 131 is used to: pass the doctor's consultation record information through a semantic encoder including a Bert model to obtain a doctor's consultation record semantic encoding feature vector as the doctor's consultation record semantic encoding feature. It should be understood that, considering that the doctor's consultation record information contains key and implicit deep semantic information about children's respiratory diseases, in order to capture the complex semantic information in the text and more accurately understand the patient's specific situation, in the technical solution of the present application, the doctor's consultation record information is semantically captured and contextually understood by a semantic encoder including a Bert model to obtain a doctor's consultation record semantic encoding feature vector. In particular, the Bert model can extract features of different abstract levels of text at different levels through a multi-layer Transformer architecture. This means that it can not only recognize simple lexical meanings, but also capture the overall semantic structure of sentences and even paragraphs. This deep understanding is particularly important for complex medical terms and descriptions.

[0035] In the embodiment of the present application, the auxiliary inspection information processing unit 132 is used to perform semantic feature extraction and visual feature extraction on the auxiliary inspection information to obtain auxiliary inspection image grid semantic features and auxiliary inspection image visual semantic features. Figure 3 is a block diagram of an auxiliary examination information processing unit in an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application. Specifically, in the embodiment of the present application, Figure 3 As shown, the auxiliary inspection information processing unit 132 includes: an auxiliary inspection image grid semantic extraction subunit 1321, which is used to grid the auxiliary inspection information and then pass it through an auxiliary inspection image grid semantic feature extractor based on a CNN-LSTM model to obtain an auxiliary inspection image grid semantic feature vector as the auxiliary inspection image grid semantic feature; an auxiliary inspection image visual semantic feature extraction subunit 1322, which is used to pass the auxiliary inspection information through a visual feature extractor based on an FCN model to obtain an auxiliary inspection image visual semantic feature vector as the auxiliary inspection image visual semantic feature.

[0036] Specifically, in an embodiment of the present application, the auxiliary inspection image grid semantic extraction subunit 1321 is used to grid the auxiliary inspection information and then obtain the auxiliary inspection image grid semantic feature vector as the auxiliary inspection image grid semantic feature through an auxiliary inspection image grid semantic feature extractor based on a CNN-LSTM model. Accordingly, considering that different local areas in the auxiliary inspection information contain different pathological structures and features, in order to achieve more refined and accurate extraction and analysis of local features, in the technical solution of the present application, the auxiliary inspection information is gridded to divide the entire image into multiple small areas (grids), which helps to identify small lesions or subtle changes that may be overlooked from a global perspective, such as lung nodules, inflammatory plaques, etc. Although the image is divided into multiple grids, the spatial relationship between adjacent grids still exists. Therefore, the auxiliary inspection information after grid division is further passed through the auxiliary inspection image grid semantic feature extractor based on the CNN-LSTM model to utilize CNN to capture the local feature details of each grid through multi-layer convolution and pooling operations, and use LSTM to process the contextual information between each grid and the grids around it, so as to better understand the spatial distribution and interaction of local features, and obtain the auxiliary inspection image grid semantic feature vector.

[0037] Specifically, in the embodiment of the present application, the auxiliary inspection image visual semantic feature extraction subunit 1322 is used to pass the auxiliary inspection information through a visual feature extractor based on the FCN model to obtain an auxiliary inspection image visual semantic feature vector as the auxiliary inspection image visual semantic feature. It should be understood that, considering that there are some key and significant regional features in the auxiliary inspection information, but there are also some irrelevant background interferences, therefore, in order to accurately identify significant disease information such as lung nodules or inflammatory areas, in the technical solution of the present application, the auxiliary inspection information is passed through a visual feature extractor based on the FCN model to capture and extract specific attributes of specific diseases in each key area, such as shape, size, color, etc., to obtain an auxiliary inspection image visual semantic feature vector. It can be understood that FCN is a deep learning model specially designed for image segmentation tasks. Compared with the traditional convolutional neural network (CNN), FCN removes the fully connected layer and is composed entirely of convolutional layers, which allows it to process input images of any size and output a pixel-level classification map of the same size as the input image. In particular, FCNs typically combine features from different convolutional layers, enabling understanding of image content at multiple scales. This multi-scale contextual understanding is very helpful for capturing lesions of different sizes and shapes, especially for abnormalities of various types and sizes that may exist in the respiratory system, helping to fully understand the health status of children.

[0038] In an embodiment of the present application, the auxiliary inspection information multi-dimensional encoding unit 133 is used to exchange auxiliary inspection information based on the cross-attention network for the auxiliary inspection image grid semantic features and the auxiliary inspection image visual semantic features to obtain a multi-dimensional semantic representation of the auxiliary inspection image. Accordingly, considering that the auxiliary inspection image grid semantic features are features captured by dividing the entire image into multiple small areas (grids), they mainly focus on more fine-grained local details such as the specific location, shape, texture, etc. and spatial relationships of the lesions. The auxiliary inspection image visual semantic features pay more attention to the key and significant global information in the image, reflecting the specific attribute information of each pixel level. Therefore, in order to capture the precise association between details such as the location and morphology of local lesions and the overall anatomical structure to better characterize the auxiliary inspection, in the technical solution of the present application, the auxiliary inspection image grid semantic features and the auxiliary inspection image visual semantic features are exchanged based on the cross-attention network to obtain a multi-dimensional semantic representation of the auxiliary inspection image. In this way, through the cross-attention network, the system can effectively and deeply fuse the information of the two features and ensure that the data of the two modalities effectively interact and complement each other at multiple levels, forming a more comprehensive and rich multidimensional semantic representation vector, thereby improving the understanding and description capabilities of complex lesions.

[0039] Specifically, in an embodiment of the present application, the auxiliary inspection information multi-dimensional encoding unit 133 is used to: perform a linear transformation on the auxiliary inspection image grid semantic feature vector to obtain an auxiliary inspection image grid semantic query vector, an auxiliary inspection image grid semantic key vector and an auxiliary inspection image grid semantic value vector; perform a linear transformation on the auxiliary inspection image visual semantic feature vector to obtain an auxiliary inspection image visual semantic query vector, an auxiliary inspection image visual semantic key vector and an auxiliary inspection image visual semantic value vector; input the auxiliary inspection image grid semantic query vector, the auxiliary inspection image visual semantic key vector and the auxiliary inspection image visual semantic value vector into the attention feature interaction module of the converter model to obtain an image vision-guided auxiliary inspection image grid semantic attention interaction feature vector; input the auxiliary inspection image visual semantic query vector, the auxiliary inspection image grid semantic key vector and the auxiliary inspection image grid semantic value vector into the attention feature interaction module of the converter model to obtain a grid semantics-guided auxiliary inspection image visual semantic attention interaction feature vector; fuse the image vision-guided auxiliary inspection image grid semantic attention interaction feature vector and the grid semantics-guided auxiliary inspection image visual semantic attention interaction feature vector to obtain an auxiliary inspection image multi-dimensional semantic representation vector as the auxiliary inspection image multi-dimensional semantic representation.

[0040] In an embodiment of the present application, the medical consultation auxiliary feature interaction unit 134 is used to perform fine-grained interaction of external knowledge features on the semantic coding features of the doctor's medical consultation record and the multi-dimensional semantic representation of the auxiliary examination image to obtain fine-grained interaction features between the doctor's medical consultation and auxiliary examination semantic features, and obtain the auxiliary diagnosis result based on the fine-grained interaction features between the doctor's medical consultation and auxiliary examination semantic features.

[0041] Figure 4 FIG. 1 is a block diagram of a diagnosis assistance feature interaction unit in an intelligent diagnosis assistance system for pediatric respiratory diseases according to an embodiment of the present application. Specifically, Figure 4 As shown, the medical consultation auxiliary feature interaction unit 134 includes: an external knowledge feature fine-grained interaction sub-unit 1341, which is used to perform external knowledge feature fine-grained interaction on the semantic encoding feature vector of the doctor's medical consultation record and the multi-dimensional semantic representation vector of the auxiliary examination image to obtain a fine-grained interaction feature vector between doctor's medical consultation and auxiliary examination semantic features as the fine-grained interaction feature between doctor's medical consultation and auxiliary examination semantic features; an auxiliary diagnosis result generation sub-unit 1342, which is used to obtain the auxiliary diagnosis result according to the fine-grained interaction feature vector between doctor's medical consultation and auxiliary examination semantic features.

[0042] Specifically, the external knowledge feature fine-grained interaction subunit 1341 is used to perform external knowledge feature fine-grained interaction on the semantic encoding feature vector of the doctor's consultation record and the multidimensional semantic representation vector of the auxiliary examination image to obtain a fine-grained interaction feature vector between the doctor's consultation-auxiliary examination semantic features as the fine-grained interaction feature between the doctor's consultation-auxiliary examination semantic features. It should be understood that the symptom description (such as cough, fever, dyspnea, etc.) in the doctor's consultation record can directly correspond to the abnormal area observed in the auxiliary examination image. For example, the description of "inflammation in the lungs" can be associated with the lung infiltration area shown in the CT scan. Therefore, in order to deeply understand the complex implicit connection between the two, in this technical solution, the external knowledge feature fine-grained interaction is used to process the semantic encoding feature vector of the doctor's consultation record and the multidimensional semantic representation vector of the auxiliary examination image to obtain a fine-grained interaction feature vector between the doctor's consultation-auxiliary examination semantic features. In particular, by introducing external medical knowledge (such as disease databases, clinical guidelines, etc.), the relationship between the symptom description in the consultation record and the imaging manifestations can be more accurately understood. For example, a specific combination of symptoms may indicate a certain type of lesion, and these relationships can be found in external knowledge, thereby making full use of these associations to significantly improve the performance and reliability of the intelligent diagnosis assistance system for pediatric respiratory diseases.

[0043] Specifically, in an embodiment of the present application, the external knowledge feature fine-grained interaction sub-unit 1341 includes: a medical consultation record-auxiliary examination fine-grained interaction secondary sub-unit, which is used to perform fine-grained feature interaction on the doctor's medical consultation record semantic encoding feature vector and the auxiliary examination image multidimensional semantic representation vector to obtain a medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix; an external knowledge optimization secondary sub-unit, which is used to perform external knowledge attention optimization on the medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix to obtain a medical consultation record-auxiliary examination semantic external knowledge optimization fine-grained feature interaction matrix; a modulation fusion secondary sub-unit, which is used to modulate and fuse the doctor's medical consultation record semantic encoding feature vector and the auxiliary examination image multidimensional semantic representation vector according to the medical consultation record-auxiliary examination semantic external knowledge optimization fine-grained feature interaction matrix to obtain the fine-grained interaction feature vector between the doctor's medical consultation-auxiliary examination semantic features.

[0044] Specifically, the medical consultation record-auxiliary examination fine-grained interaction secondary subunit is used to perform fine-grained feature interaction on the semantic encoding feature vector of the doctor's medical consultation record and the multidimensional semantic representation vector of the auxiliary examination image to obtain a medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix. It should be understood that in order to capture the precise association between the symptom description in the medical consultation record and the imaging manifestation, the fine-grained feature interaction is performed on the semantic encoding feature vector of the doctor's medical consultation record and the multidimensional semantic representation vector of the auxiliary examination image to capture and extract the deep connection between the two at different levels to obtain a medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix.

[0045] The processing process of the above-mentioned consultation record-auxiliary examination fine-grained interaction secondary sub-unit can be expressed by the following formula: ; in, A semantic encoding feature vector for the doctor's consultation record, is the multi-dimensional semantic representation vector of the auxiliary inspection image, for The transposed vector of is matrix multiplication, It is the medical record-auxiliary examination fine-grained semantic feature interaction matrix.

[0046] Specifically, the external knowledge optimization secondary subunit is used to perform external knowledge attention optimization on the medical record-auxiliary examination fine-grained semantic feature interaction matrix to obtain the medical record-auxiliary examination semantic external knowledge optimization fine-grained feature interaction matrix. Accordingly, the medical record-auxiliary examination fine-grained semantic feature interaction matrix is ​​optimized with external knowledge attention to introduce external medical knowledge (such as ICD coding, SNOMED CT, clinical guidelines, etc.) to integrate the expertise in these fields into the feature interaction, helping to capture implicit relationships that may be overlooked in a single modality or direct interaction, thereby more accurately understanding the complex potential relationship between medical records and imaging data, so as to obtain the medical record-auxiliary examination semantic external knowledge optimization fine-grained feature interaction matrix; The above external knowledge optimization secondary sub-unit processing process can be expressed by the following formula: ; in, It is the medical record-auxiliary examination fine-grained semantic feature interaction matrix. and They represent the learnable memory parameter matrix, for The transposed matrix of represents the normalization function, It is the medical consultation record-auxiliary examination semantic external knowledge optimization fine-grained feature interaction matrix.

[0047] More specifically, in the embodiment of the present application, the modulation fusion secondary subunit is used to: The semantic coding feature vector of the doctor's medical consultation record is linearly transformed to obtain a medical consultation record query feature vector and a medical consultation record value feature vector, and the medical consultation record-auxiliary examination semantic external knowledge optimized fine-grained feature interaction matrix is ​​used as a key matrix. The medical consultation record query feature vector, the medical consultation record value feature vector and the key matrix are modulated based on the Transformer structure to obtain an optimized semantic coding feature vector of the doctor's medical consultation record. This step can be expressed by the following formula: ; ; ; in, A semantic encoding feature vector for the doctor's consultation record, is matrix multiplication, It is the medical record-auxiliary inspection semantic external knowledge optimization fine-grained feature interaction matrix, and are the query embedding matrix of the medical record and the first bias vector of the medical record, respectively. is the medical record query feature vector, and are the embedding matrix of the medical consultation record value and the second bias vector of the medical consultation record, respectively. is the characteristic vector of the consultation record value, for The transposed matrix of yes Length, yes function, is the optimized semantic encoding feature vector of the doctor's consultation record; The auxiliary inspection image multidimensional semantic representation vector is linearly transformed to obtain an auxiliary inspection query feature vector and an auxiliary inspection value feature vector, and the medical record-auxiliary inspection semantic external knowledge optimized fine-grained feature interaction matrix is ​​used as a key matrix. The auxiliary inspection query feature vector, the auxiliary inspection value feature vector and the key matrix are modulated based on the Transformer structure to obtain an optimized auxiliary inspection image multidimensional semantic representation vector. This step can be expressed by the following formula: ; ; ; in, is the multi-dimensional semantic representation vector of the auxiliary inspection image, It is the medical record-auxiliary inspection semantic external knowledge optimization fine-grained feature interaction matrix, yes function, and are the auxiliary check query embedding matrix and the auxiliary check first bias vector, respectively. is the auxiliary inspection query feature vector, and are the auxiliary check value embedding matrix and the auxiliary check second bias vector respectively, is the auxiliary check value eigenvector, yes Length, is the optimized multi-dimensional semantic representation vector of the auxiliary inspection image; The fine-grained interaction feature vector between the doctor's consultation record semantic encoding feature vector after optimization and the auxiliary examination image multidimensional semantic representation vector after optimization is calculated and divided by the position point. This step can be expressed by the following formula: ; in, is the optimized semantic encoding feature vector of the doctor's consultation record, is the optimized multi-dimensional semantic representation vector of the auxiliary inspection image, It is the fine-grained interaction feature vector between the semantic features of the doctor's consultation and auxiliary examination.

[0048] It should be understood that the doctor's consultation record semantic coding feature vector is linearly transformed to obtain the consultation record query feature vector and the consultation record value feature vector, and the consultation record-auxiliary examination semantic external knowledge optimized fine-grained feature interaction matrix after external knowledge integration is used as the key matrix, and it is modulated based on the Transformer structure. In particular, the self-attention mechanism in the Transformer structure can achieve accurate fine-grained matching between the doctor's consultation record features and the key matrix, thereby promoting the information exchange and fusion of the doctor's consultation record features and external knowledge, so that it can fully utilize the advantages of external knowledge, and finally obtain the modulated and optimized doctor's consultation record semantic coding feature vector.

[0049] Similarly, the auxiliary inspection image multidimensional semantic representation vector also performs the same operation as above, and enhances the expression ability by integrating the auxiliary inspection image features extracted from the external knowledge, thereby obtaining an optimized auxiliary inspection image multidimensional semantic representation vector.

[0050] Finally, the optimized semantic encoding feature vector of the doctor's consultation record and the optimized multidimensional semantic representation vector of the auxiliary examination image are divided by the position points to obtain a fine-grained interactive feature vector between the semantic features of the doctor's consultation and the auxiliary examination, which integrates the information and interactive information of the two original features.

[0051] Specifically, the auxiliary diagnosis result generating subunit 1342 is used to obtain the auxiliary diagnosis result according to the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features. Specifically, in the embodiment of the present application, the auxiliary diagnosis result generating subunit 1342 is used to: input the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features into the trained typical symptom diagnosis model to obtain the auxiliary diagnosis result. In particular, the diagnosis result includes at least one possible disease, and the characteristics of the child to be diagnosed corresponding to each possible disease. That is, the typical symptom diagnosis model is fully trained and can identify the typical pattern of a specific disease or condition. By inputting the fine-grained interactive feature vector, the model can provide more accurate auxiliary diagnosis results and help doctors make more reliable clinical decisions. In particular, please refer to patent CN109935336B for the specific network structure and detailed description of the typical symptom diagnosis model that has been trained. Since this model is not the focus of this solution, it will not be specifically explained.

[0052] It should be understandable that, considering that the semantic encoding feature vector of the doctor's consultation record and the multidimensional semantic representation vector of the auxiliary examination image represent the encoded semantic features of the doctor's consultation record information and the text-visual cross-semantic features of the auxiliary examination information respectively, when performing fine-grained feature interaction based on external knowledge modulation, the insufficient correspondence of external knowledge modulation caused by the difference in semantic representation of source data will lead to the lack of judgment of interaction feature instances of the fine-grained interaction feature vector between the doctor's consultation-auxiliary examination semantic features, thereby affecting the accuracy of the auxiliary diagnosis results obtained through the typical symptom diagnosis model completed through training.

[0053] Preferably, inputting the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features into the trained typical symptom diagnosis model to obtain the auxiliary diagnosis result includes: The distance between each pair of feature values ​​of the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features is calculated, for example L 2 distance, and take the square root of the distance to obtain the fine-grained interaction distance representation matrix between the doctor's consultation-auxiliary examination semantic features, that is: ; in, is the fine-grained interaction feature vector between the doctor's consultation-auxiliary examination semantic features, is the fine-grained interaction feature vector between the doctor's consultation and auxiliary examination semantic features. The eigenvalues ​​at the positions, is the fine-grained interaction feature vector between the doctor's consultation and auxiliary examination semantic features. The eigenvalues ​​at the positions, yes and The distance between yes and The square root of the distance between The fine-grained interaction autocorrelation matrix between the doctor's consultation and auxiliary examination semantic features of the fine-grained interaction feature vector between the doctor's consultation and auxiliary examination semantic features is obtained as a row vector, that is, ; in, is the fine-grained interaction feature vector between the doctor's consultation-auxiliary examination semantic features, and is a row vector, express The transposed vector of is the fine-grained interactive autocorrelation matrix between the doctor's consultation-auxiliary examination semantic features, Represents matrix multiplication; The fine-grained interaction feature vector between the doctor's consultation and auxiliary examination semantic features is matrix-multiplied with the fine-grained interaction distance representation matrix between the doctor's consultation and auxiliary examination semantic features to obtain the first-level mapping vector of the fine-grained interaction between the doctor's consultation and auxiliary examination semantic features, that is, ; in, is the fine-grained interaction distance representation matrix between the doctor's consultation-auxiliary examination semantic features, is the first-level mapping vector of the fine-grained interaction between the doctor's consultation and auxiliary examination semantic features; The fine-grained interaction first-level mapping vector between the doctor's consultation-auxiliary examination semantic features is matrix-multiplied with the matrix product of the fine-grained interaction distance representation matrix between the doctor's consultation-auxiliary examination semantic features and the fine-grained interaction autocorrelation matrix between the doctor's consultation-auxiliary examination semantic features to obtain the fine-grained interaction multi-level mapping vector between the doctor's consultation-auxiliary examination semantic features. ; in, is a multi-level mapping vector of fine-grained interaction between the semantic features of doctor's consultation and auxiliary examination; The fine-grained interaction multi-level mapping vector between the doctor's consultation and auxiliary examination semantic features is interpolated with the fine-grained interaction correlation eigenvector between the doctor's consultation and auxiliary examination semantic features composed of the eigenvalues ​​of the fine-grained interaction autocorrelation matrix between the doctor's consultation and auxiliary examination semantic features to obtain an optimized fine-grained interaction feature vector between the doctor's consultation and auxiliary examination semantic features, wherein interpolation or zero padding is performed when the eigenvalue is insufficient; The optimized fine-grained interactive feature vector between the doctor's consultation and auxiliary examination semantic features is input into the trained typical symptom diagnosis model to obtain an auxiliary diagnosis result.

[0054] Therefore, through the linear target mapping representation of the similarity distance representation matrix of the fine-grained interaction feature vector between the semantic features of the doctor's consultation and auxiliary examination, the complete similarity instantiation of the self-correlation of the fine-grained interaction feature vector between the semantic features of the doctor's consultation and auxiliary examination is performed based on a secondary target mapping representation based on a multi-level distribution hierarchy, and the association mismatch negative influence factor is compensated by the association fusion kernel bias to improve the degree of instance determination of the eigenvalues ​​of the fine-grained interaction feature vector between the semantic features of the doctor's consultation and auxiliary examination under the similarity constraint, that is, the degree of significance of the eigenvalue as an instance for the regression judgment, so as to improve the accuracy of the auxiliary diagnosis results obtained by the typical symptom diagnosis model trained by inputting the fine-grained interaction feature vector between the semantic features of the doctor's consultation and auxiliary examination.

[0055] In summary, the result generation module 130 is explained, which semantically encodes the doctor's consultation record information by using natural language analysis technology based on deep learning and image processing algorithms, and extracts the auxiliary examination information and visual features respectively, and performs cross-attention multi-dimensional fusion of the auxiliary examination image grid semantic features and the auxiliary examination image visual semantic features, so as to automatically generate auxiliary diagnosis results based on the fine-grained interaction features of external knowledge between the semantic encoding features of the doctor's consultation record and the multi-dimensional semantic representation of the auxiliary examination image. Compared with simple keyword matching, the present application can more comprehensively understand the patient's symptom description and auxiliary information, and consider the intrinsic connection between multiple information sources, so as to reach a more comprehensive and accurate diagnosis conclusion. At the same time, the features in the external knowledge base are used to guide the diagnosis process, so that the model can benefit from the latest medical research results, thereby maintaining a high diagnostic accuracy.

[0056] Specifically, the result display module 140 is used to display the auxiliary diagnosis result in the information display interface. It is understandable that by displaying in the information display interface, doctors can quickly understand complex diagnostic information and make more intelligent clinical decisions accordingly.

[0057] Specifically, when presenting these auxiliary diagnosis results in the information display interface, the system adopts a structured and user-friendly approach to ensure that doctors can quickly understand and use this information to make clinical decisions. The following is a specific example to illustrate how auxiliary diagnosis results are displayed: When presenting auxiliary diagnosis results in the information display interface, the system adopts a structured and user-friendly approach to ensure that doctors can quickly understand and use this information to make clinical decisions. For example, suppose a 5-year-old boy comes to see a doctor because of persistent coughing and dyspnea. The doctor collects his consultation records and chest X-rays as auxiliary examination information through the intelligent diagnosis assistance system. After system processing and analysis, the auxiliary diagnosis results shown in the following example are generated.

[0058] In the "Main Diagnosis Suggestion Area", the system shows that the most likely disease is bronchitis, with a confidence score of 87%, mainly based on the child's symptom description (cough, dyspnea), the chest X-ray showing thickening and blurring of the lung texture, which is consistent with the imaging characteristics of bronchitis, and the age factor (5 years old) is consistent with the high-incidence age group of the disease. Secondly, in the "Other Possible Diseases and Their Characteristics Area", the system lists the secondary possibility of asthma (confidence score 63%). The feature comparison points out that there are also cough symptoms but usually accompanied by wheezing, no obvious abnormalities on the X-ray, and whether there is a history of allergic diseases in the family history is an important reference point; there is also pneumonia as another consideration (confidence score 45%). The difference in characteristics is that patients with pneumonia often have more obvious systemic symptoms such as fever and sputum, and imaging can show lung consolidation or infiltration shadows, but the current child's X-ray does not show such typical signs.

[0059] For each listed disease, the system provides a brief but detailed text description in the detailed explanation and support information area, introducing common symptoms, pathophysiological mechanisms, possible development trends and recommended follow-up steps. For example, for bronchitis, the system will point out that this is a common respiratory infection in infants and young children, usually caused by viruses, and the treatment focuses on relieving symptoms and supportive therapy. In addition, in the special situation reminder area, the system reminds doctors to pay attention to any uncertain factors or areas that need further confirmation, such as "Although bronchitis is the most likely diagnosis, due to the young age of the child, it is still necessary to closely observe changes in the condition and consider whether it is necessary to perform blood tests or other further tests to rule out other potential problems."

[0060] Based on the current diagnosis results, the system gives preliminary treatment recommendations or next action plans based on the latest clinical guidelines and best practices in the "Preliminary Treatment Recommendations Area", emphasizing that the professional judgment of doctors is crucial. For example, "It is recommended to give appropriate symptomatic treatment, such as using nebulized inhalation drugs to relieve coughing and dyspnea; at the same time, keep the indoor air circulating and avoid contact with smoke and other irritants." In order to enhance the user experience, there is also an "interactive function area" on the interface. Doctors can get more information by clicking on specific areas, such as viewing original imaging data, enlarging local details or reading relevant medical literature abstracts. The system allows doctors to customize the layout and display content to better adapt to personal work habits.

[0061] Through this structured display method, the intelligent diagnosis assistance system not only provides doctors with clear and specific diagnostic suggestions, but also helps them fully understand the patient's condition characteristics and related background information, so as to make more scientific and reasonable clinical decisions.

[0062] In summary, the intelligent diagnosis assistance system 100 for pediatric respiratory diseases based on the embodiment of the present application is explained, which first collects the doctor's consultation records and auxiliary examination information of the child, and inputs this information into the typical symptom diagnosis model for processing, and then the model generates auxiliary diagnosis results, which are obtained based on comprehensive analysis and displayed in the information display interface. In this way, auxiliary diagnosis results can be automatically generated to provide decision support for doctors and ensure the accuracy and efficiency of diagnosis.

[0063] As described above, the intelligent diagnosis assistance system 100 for pediatric respiratory diseases according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an intelligent diagnosis assistance algorithm for pediatric respiratory diseases. In one possible implementation, the intelligent diagnosis assistance system 100 for pediatric respiratory diseases according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the intelligent diagnosis assistance system 100 for pediatric respiratory diseases can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the intelligent diagnosis assistance system 100 for pediatric respiratory diseases can also be one of the many hardware modules of the wireless terminal.

[0064] Alternatively, in another example, the intelligent diagnosis assistance system 100 for pediatric respiratory diseases and the wireless terminal may also be separate devices, and the intelligent diagnosis assistance system 100 for pediatric respiratory diseases may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0065] Figure 5 FIG. 1 is a flow chart of an intelligent diagnosis assistance method for pediatric respiratory diseases according to an embodiment of the present application. Figure 5As shown, according to the intelligent diagnosis assistance method for pediatric respiratory diseases according to the embodiment of the present application, it is used to execute the above-mentioned intelligent diagnosis assistance system for pediatric respiratory diseases, and the intelligent diagnosis assistance method for pediatric respiratory diseases includes: S110, obtaining the doctor's consultation record information of the child to be diagnosed; S120, obtaining the auxiliary examination information of the child to be diagnosed; S130, inputting the doctor's consultation record information and the auxiliary examination information into the typical symptom diagnosis model for processing to generate an auxiliary diagnosis result; S140, displaying the auxiliary diagnosis result in the information display interface. Among them, the doctor's consultation record information and the auxiliary examination information are input into the typical symptom diagnosis model for processing to generate an auxiliary diagnosis result, including: semantically encoding the doctor's consultation record information to obtain the semantic coding features of the doctor's consultation record; respectively extracting semantic features and visual features of the auxiliary examination information to obtain the auxiliary examination image grid semantic features and the auxiliary examination image visual semantic features; performing auxiliary examination information exchange based on the cross-attention network on the auxiliary examination image grid semantic features and the auxiliary examination image visual semantic features to obtain the auxiliary examination image multidimensional semantic representation; performing external knowledge feature fine-grained interaction on the doctor's consultation record semantic coding features and the auxiliary examination image multidimensional semantic representation to obtain the fine-grained interaction features between the doctor's consultation-auxiliary examination semantic features, and obtaining the auxiliary diagnosis result based on the fine-grained interaction features between the doctor's consultation-auxiliary examination semantic features.

[0066] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned intelligent diagnosis auxiliary method for pediatric respiratory diseases have been described in the above reference. Figures 1 to 4 The invention has been described in detail in the description of the intelligent diagnosis assistance system for pediatric respiratory diseases, and therefore, its repeated description will be omitted.

[0067] The above descriptions of various implementations of the present disclosure are exemplary and not exhaustive. They are not limited to the disclosed implementations, and many modifications and changes are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The terms used herein are selected to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other ordinary skill in the art to understand the various practical methods disclosed herein.

Claims

1. An intelligent diagnostic assistance system for pediatric respiratory diseases, characterized in that: include: A consultation record acquisition module is used to obtain the doctor's consultation record information of the child to be diagnosed; An auxiliary information acquisition module, used to acquire auxiliary examination information of the child to be diagnosed; A result generating module, used for inputting the doctor's consultation record information and the auxiliary examination information into a typical symptom diagnosis model for processing to generate an auxiliary diagnosis result; A result display module is used to display the auxiliary diagnosis result in the information display interface, wherein the result generation module includes: A medical consultation record semantic coding unit, used for semantically coding the doctor's medical consultation record information to obtain a semantic coding feature of the doctor's medical consultation record; An auxiliary inspection information processing unit, used for respectively performing semantic feature extraction and visual feature extraction on the auxiliary inspection information to obtain auxiliary inspection image grid semantic features and auxiliary inspection image visual semantic features; an auxiliary inspection information multi-dimensional encoding unit, used for performing auxiliary inspection information communication based on a cross attention network on the auxiliary inspection image grid semantic features and the auxiliary inspection image visual semantic features to obtain a multi-dimensional semantic representation of the auxiliary inspection image; The consultation auxiliary feature interaction unit is used to perform fine-grained interaction of external knowledge features on the semantic coding features of the doctor's consultation record and the multi-dimensional semantic representation of the auxiliary examination image to obtain fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features, and obtain the auxiliary diagnosis result based on the fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features.

2. The intelligent diagnosis assistance system for pediatric respiratory diseases according to claim 1 is characterized in that: The medical consultation record semantic encoding unit is used to: pass the doctor's consultation record information through a semantic encoder including a Bert model to obtain a doctor's consultation record semantic encoding feature vector as the doctor's consultation record semantic encoding feature.

3. The intelligent diagnosis assistance system for pediatric respiratory diseases according to claim 2 is characterized in that: The auxiliary inspection information processing unit includes: An auxiliary inspection image grid semantic extraction subunit is used to grid the auxiliary inspection information and then pass it through an auxiliary inspection image grid semantic feature extractor based on a CNN-LSTM model to obtain an auxiliary inspection image grid semantic feature vector as the auxiliary inspection image grid semantic feature; The auxiliary inspection image visual semantic feature extraction subunit is used to pass the auxiliary inspection information through a visual feature extractor based on the FCN model to obtain an auxiliary inspection image visual semantic feature vector as the auxiliary inspection image visual semantic feature.

4. The intelligent diagnosis assistance system for pediatric respiratory diseases according to claim 3 is characterized in that: The auxiliary inspection information multi-dimensional encoding unit is used to: Performing a linear transformation on the auxiliary inspection image grid semantic feature vector to obtain an auxiliary inspection image grid semantic query vector, an auxiliary inspection image grid semantic key vector, and an auxiliary inspection image grid semantic value vector; Performing a linear transformation on the auxiliary inspection image visual semantic feature vector to obtain an auxiliary inspection image visual semantic query vector, an auxiliary inspection image visual semantic key vector, and an auxiliary inspection image visual semantic value vector; Inputting the auxiliary inspection image grid semantic query vector, the auxiliary inspection image visual semantic key vector and the auxiliary inspection image visual semantic value vector into the attention feature interaction module of the converter model to obtain an image vision guided auxiliary inspection image grid semantic attention interaction feature vector; Inputting the auxiliary inspection image visual semantic query vector, the auxiliary inspection image grid semantic key vector and the auxiliary inspection image grid semantic value vector into the attention feature interaction module of the converter model to obtain a grid semantic guided auxiliary inspection image visual semantic attention interaction feature vector; The image vision-guided auxiliary inspection image grid semantic attention interaction feature vector and the grid semantics-guided auxiliary inspection image visual semantic attention interaction feature vector are fused to obtain an auxiliary inspection image multidimensional semantic representation vector as the auxiliary inspection image multidimensional semantic representation.

5. The intelligent diagnosis assistance system for pediatric respiratory diseases according to claim 4 is characterized in that: The medical consultation auxiliary feature interaction unit comprises: An external knowledge feature fine-grained interaction subunit is used to perform external knowledge feature fine-grained interaction on the semantic encoding feature vector of the doctor's consultation record and the multi-dimensional semantic representation vector of the auxiliary examination image to obtain a fine-grained interaction feature vector between the doctor's consultation and auxiliary examination semantic features as the fine-grained interaction feature between the doctor's consultation and auxiliary examination semantic features; The auxiliary diagnosis result generating subunit is used to obtain the auxiliary diagnosis result according to the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features.

6. The intelligent diagnosis assistance system for pediatric respiratory diseases according to claim 5, characterized in that: The external knowledge feature fine-grained interaction sub-unit includes: The medical consultation record-auxiliary examination fine-grained interaction secondary subunit is used to perform fine-grained feature interaction on the semantic encoding feature vector of the doctor's medical consultation record and the multi-dimensional semantic representation vector of the auxiliary examination image to obtain a medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix; The external knowledge optimization secondary sub-unit is used to perform external knowledge attention optimization on the medical consultation record-auxiliary examination fine-grained semantic feature interaction matrix to obtain the medical consultation record-auxiliary examination semantic external knowledge optimization fine-grained feature interaction matrix; The modulation and fusion secondary sub-unit is used to optimize the fine-grained feature interaction matrix according to the medical consultation record-auxiliary examination semantic external knowledge, and to modulate and fuse the doctor's medical consultation record semantic encoding feature vector and the auxiliary examination image multidimensional semantic representation vector to obtain the fine-grained interaction feature vector between the doctor's medical consultation-auxiliary examination semantic features.

7. The intelligent diagnosis assistance system for pediatric respiratory diseases according to claim 6, characterized in that: The modulation fusion secondary subunit is used for: The semantic coding feature vector of the doctor's consultation record is linearly transformed to obtain a consultation record query feature vector and a consultation record value feature vector, and the consultation record-auxiliary examination semantic external knowledge optimized fine-grained feature interaction matrix is ​​used as a key matrix, and the consultation record query feature vector, the consultation record value feature vector and the key matrix are modulated based on the Transformer structure to obtain an optimized semantic coding feature vector of the doctor's consultation record; Performing a linear transformation on the auxiliary inspection image multidimensional semantic representation vector to obtain an auxiliary inspection query feature vector and an auxiliary inspection value feature vector, and using the medical inquiry record-auxiliary inspection semantic external knowledge optimized fine-grained feature interaction matrix as a key matrix, performing the Transformer structure-based modulation on the auxiliary inspection query feature vector, the auxiliary inspection value feature vector and the key matrix to obtain an optimized auxiliary inspection image multidimensional semantic representation vector; The point-by-point division between the optimized semantic encoding feature vector of the doctor's consultation record and the optimized multi-dimensional semantic representation vector of the auxiliary examination image is calculated to obtain a fine-grained interaction feature vector between the semantic features of the doctor's consultation and the auxiliary examination.

8. The intelligent diagnosis assistance system for pediatric respiratory diseases according to claim 7, characterized in that: The auxiliary diagnosis result generating subunit is used to input the fine-grained interactive feature vector between the doctor's consultation-auxiliary examination semantic features into the trained typical symptom diagnosis model to obtain the auxiliary diagnosis result.

9. An intelligent diagnostic aid method for pediatric respiratory diseases, characterized in that: include: Obtain information about doctors' consultation records of children to be diagnosed; Obtaining auxiliary examination information of the child to be diagnosed; Inputting the doctor's consultation record information and the auxiliary examination information into a typical symptom diagnosis model for processing to generate an auxiliary diagnosis result; and displaying the auxiliary diagnosis result in an information display interface, wherein inputting the doctor's consultation record information and the auxiliary examination information into a typical symptom diagnosis model for processing to generate an auxiliary diagnosis result includes: Performing semantic coding on the doctor's consultation record information to obtain a semantic coding feature of the doctor's consultation record; Respectively performing semantic feature extraction and visual feature extraction on the auxiliary inspection information to obtain auxiliary inspection image grid semantic features and auxiliary inspection image visual semantic features; Performing auxiliary inspection information exchange based on a cross attention network on the auxiliary inspection image grid semantic features and the auxiliary inspection image visual semantic features to obtain a multi-dimensional semantic representation of the auxiliary inspection image; Fine-grained interaction of external knowledge features is performed on the semantic coding features of the doctor's consultation record and the multidimensional semantic representation of the auxiliary examination image to obtain fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features, and the auxiliary diagnosis result is obtained based on the fine-grained interaction features between the doctor's consultation and auxiliary examination semantic features.

Images