Intelligent auxiliary system for diagnosis of children's respiratory diseases

Abstract

The present application relates to the field of intelligent assistance, and specifically discloses an intelligent assistance system for diagnosing children's respiratory diseases, which obtains the doctor's consultation record information of the child to be diagnosed, and in response to the existence of auxiliary examination items, obtains the auxiliary examination information of the auxiliary examination items of the child to be diagnosed, and uses image recognition and data analysis technology based on artificial intelligence to segment and aggregate the doctor's consultation record information with global semantic coding, and extracts implicit semantic features from the auxiliary examination information, so as to intelligently obtain the auxiliary diagnosis results of the respiratory disease type of the child to be diagnosed based on the cross-modal semantic joint features between the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination, and display the results. In this way, the accuracy and comprehensiveness of the auxiliary diagnosis of children's respiratory diseases can be significantly improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent assistance, and more specifically, to an intelligent assistance system for diagnosing pediatric respiratory diseases. Background Art

[0002] Pediatric respiratory diseases are among the most common diseases in pediatrics, including pneumonia, asthma, bronchitis, acute upper respiratory tract infection, etc. However, in primary medical institutions, doctors lack clinical experience and traditional training methods have limited effects. In addition, artificial intelligence is rarely used in the diagnosis and treatment of pediatric respiratory diseases, especially difficult and complicated diseases. Children have limited expressive ability and limited radiological imaging examinations, which leads to challenges in intelligent diagnosis.

[0003] Based on this, Chinese patent CN109935336B proposes an intelligent auxiliary diagnosis system for children's respiratory diseases. It collects doctors' consultation records and auxiliary examination information, screens key information and uses a pre-trained BP neural network model to obtain diagnosis results, and displays the diagnosis results for doctors to operate and view. This improves the diagnostic accuracy of primary care doctors for difficult and complicated children's respiratory diseases and optimizes the allocation of medical resources.

[0004] The diagnosis results in the above patents are obtained by screening out key information from the medical consultation records and auxiliary information, and then inputting it into the BP neural network model for processing. However, although this method is simple, it has the following defects. Specifically, only by screening out key information and inputting it into the BP neural network model, it may not be possible to capture the deep implicit features in the doctor's medical consultation records and auxiliary examination information. For example, some subtle changes in physical signs, hidden factors in family medical history, etc. may be ignored, and these implicit features are crucial for accurate diagnosis. In addition, the doctor's medical consultation records and auxiliary examination information are processed separately, lacking a comprehensive analysis of these two different modal information. This split may lead to incomplete and incoherent information, and cannot fully reflect the patient's overall condition. For example, there may be complex associations between certain symptoms and examination results, and it is difficult to discover these associations by processing them separately, thus affecting the comprehensiveness and accuracy of the diagnosis.

[0005] Therefore, an intelligent auxiliary system for diagnosing children's respiratory diseases is desired. Summary of the invention

[0006] In order to solve the above technical problems, this application is proposed.

[0007] The present application provides an intelligent auxiliary system for diagnosing pediatric respiratory diseases, which includes:

[0008] The diagnosis and consultation record acquisition module for children is used to obtain the doctor's consultation record information of the child to be diagnosed;

[0009] An auxiliary examination information acquisition module, for acquiring auxiliary examination information of the auxiliary examination items of the child to be diagnosed in response to the existence of auxiliary examination items in the doctor's consultation record information;

[0010] A consultation record information encoding module, used for performing segmentation processing and global semantic encoding aggregation on the doctor's consultation record information to obtain global semantic aggregation features of the doctor's consultation record information;

[0011] An auxiliary inspection implicit feature extraction module is used to extract implicit features from the auxiliary inspection information to obtain auxiliary inspection implicit semantic features;

[0012] A cross-modal semantic union module is used to perform cross-modal semantic union on the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination to obtain cross-modal semantic joint coding features of the doctor's consultation record-auxiliary examination, wherein the cross-modal semantic union module includes: an auxiliary feature decomposition unit, which is used to perform local feature decomposition on the implicit semantic features of the auxiliary examination to obtain a set of auxiliary examination implicit semantic local features; a prompt-guided feature union unit, which is used to perform prompt-guided feature union on the global semantic aggregation features of the doctor's consultation record information and the set of auxiliary examination implicit semantic local features to obtain the doctor's consultation record-auxiliary examination cross-modal semantic joint coding features;

[0013] The auxiliary result generation and display module is used to obtain auxiliary diagnosis results based on the cross-modal semantic joint encoding features of the doctor's consultation record-auxiliary examination, and display the auxiliary diagnosis results.

[0014] In the above-mentioned intelligent auxiliary system for diagnosing pediatric respiratory diseases, the consultation record information encoding module includes: a consultation record information segmentation encoding unit, which is used to segment the doctor's consultation record information according to the record category dimension, and then semantically encode each segment of the doctor's consultation record information in the doctor's consultation record information to obtain a sequence of doctor's consultation record information segment semantic feature vectors; a consultation record information global semantic aggregation unit, which is used to pass the sequence of doctor's consultation record information segment semantic feature vectors through a doctor's consultation record global semantic aggregator based on a bidirectional LTSM model to obtain a doctor's consultation record information global semantic aggregation feature vector as the doctor's consultation record information global semantic aggregation feature.

[0015] In the above-mentioned intelligent auxiliary system for diagnosing pediatric respiratory diseases, the auxiliary examination implicit feature extraction module is used to: pass the auxiliary examination information through an auxiliary examination implicit feature extractor based on a deep separable convolutional neural network model to obtain an auxiliary examination implicit semantic feature map as the auxiliary examination implicit semantic feature.

[0016] In the above-mentioned intelligent auxiliary system for diagnosing pediatric respiratory diseases, the auxiliary feature decomposition unit is used to: perform local feature decomposition along the channel dimension on the auxiliary examination latent semantic feature map to obtain a set of auxiliary examination latent semantic local feature matrices as the set of auxiliary examination latent semantic local features.

[0017] In the above-mentioned intelligent auxiliary system for diagnosing pediatric respiratory diseases, the prompt-guided feature combination unit includes: a doctor's consultation record-auxiliary examination query prompt subunit, which is used to use the global semantic aggregation feature vector of the doctor's consultation record information as the query vector and each auxiliary examination implicit semantic local feature matrix in the set of the auxiliary examination implicit semantic local feature matrix as the key matrix, and input the query vector and the key matrix into a prompt learning network based on a converter structure to obtain a set of doctor's consultation record-auxiliary examination local query prompt semantic coding vectors; a doctor's consultation record-auxiliary examination feature mask subunit, which is used to input the global semantic aggregation feature vector of the doctor's consultation record information and each doctor's consultation record-auxiliary examination local query prompt semantic coding vector in the set of the doctor's consultation record-auxiliary examination local query prompt semantic coding vector into a cross-modal mask weaving network based on prompt information to obtain a prompt information-based ... A set of doctor's consultation records-auxiliary examination cross-modal local feature mask weight matrices based on prompt information; a doctor's consultation records-auxiliary examination feature fine-grained interaction sub-unit, used to calculate the point-by-point multiplication between each corresponding group of doctor's consultation records-auxiliary examination cross-modal local feature mask weight matrices based on prompt information and the set of auxiliary examination implicit semantic local feature matrices to obtain a set of doctor's consultation records-auxiliary examination cross-modal local granularity significant interaction matrices; a doctor's consultation records-auxiliary examination feature joint sub-unit, used to perform feature aggregation on the set of doctor's consultation records-auxiliary examination cross-modal local granularity significant interaction matrices to obtain a doctor's consultation records-auxiliary examination cross-modal semantic joint encoding feature map as the doctor's consultation records-auxiliary examination cross-modal semantic joint encoding feature.

[0018] In the above-mentioned intelligent auxiliary system for diagnosing pediatric respiratory diseases, the doctor's consultation record-auxiliary examination query prompt subunit includes: a doctor's consultation record-auxiliary examination similarity score calculation secondary subunit, which is used to calculate the similarity score between the global semantic aggregation feature vector of the doctor's consultation record information and each row vector in the auxiliary examination implicit semantic local feature matrix to obtain a set of doctor's consultation record-auxiliary examination similarity scores; a doctor's consultation record-auxiliary examination weight calculation secondary subunit, which is used to standardize each doctor's consultation record-auxiliary examination similarity score in the set of doctor's consultation record-auxiliary examination similarity scores to obtain a set of doctor's consultation record-auxiliary examination weight coefficients; a doctor's consultation record-auxiliary examination feature weighting secondary subunit, which is used to calculate the weighted sum of each row vector in the auxiliary examination implicit semantic local feature matrix based on the set of doctor's consultation record-auxiliary examination weight coefficients to obtain the doctor's consultation record-auxiliary examination local query prompt semantic encoding vector.

[0019] In the above-mentioned intelligent auxiliary system for diagnosing pediatric respiratory diseases, the secondary sub-unit for calculating the similarity score of the doctor's consultation record-auxiliary examination is used to: calculate the binary norm of each row vector in the auxiliary examination implicit semantic local feature matrix to obtain a set of auxiliary examination implicit semantic local norm values; calculate the binary norm of the global semantic aggregation feature vector of the doctor's consultation record information to obtain the global semantic norm value of the doctor's consultation record information; multiply the global semantic norm value of the doctor's consultation record information and the set of the auxiliary examination implicit semantic local norm value by position to obtain a set of doctor's consultation record-auxiliary examination joint values; calculate the product between the global semantic aggregation feature vector of the doctor's consultation record information and the transposed vector of each row vector in the auxiliary examination implicit semantic local feature matrix to obtain a set of doctor's consultation record-auxiliary examination related values; divide each group of corresponding doctor's consultation record-auxiliary examination related values ​​in the set of the doctor's consultation record-auxiliary examination related values ​​and the set of the doctor's consultation record-auxiliary examination joint values ​​by the doctor's consultation record-auxiliary examination joint value to obtain the set of doctor's consultation record-auxiliary examination similarity scores.

[0020] In the above-mentioned intelligent auxiliary system for diagnosing pediatric respiratory diseases, the secondary sub-unit for calculating the weight of the doctor's consultation record-auxiliary examination is used to: calculate the natural exponential function value with the natural constant e as the base and each doctor's consultation record-auxiliary examination similarity score in the set of the doctor's consultation record-auxiliary examination similarity scores as the exponential power to obtain a set of doctor's consultation record-auxiliary examination similarity modulation scores; calculate the set of the doctor's consultation record-auxiliary examination similarity modulation scores point by point to obtain the total sum of the doctor's consultation record-auxiliary examination similarity scores; divide the set of the doctor's consultation record-auxiliary examination similarity modulation scores by the total sum of the doctor's consultation record-auxiliary examination similarity scores by position to obtain the set of the doctor's consultation record-auxiliary examination weight coefficients.

[0021] In the above-mentioned intelligent auxiliary system for diagnosis of pediatric respiratory diseases, the doctor's consultation record-auxiliary examination feature mask sub-unit is used to: multiply the transposed vector of the doctor's consultation record-auxiliary examination local query prompt semantic encoding vector with the global semantic aggregation feature vector of the doctor's consultation record information to obtain a doctor's consultation record-auxiliary examination prompt association matrix; divide the doctor's consultation record-auxiliary examination prompt association matrix by the square root of the scale of the doctor's consultation record-auxiliary examination prompt association matrix by position to obtain a doctor's consultation record-auxiliary examination prompt association modulation matrix; input the doctor's consultation record-auxiliary examination prompt association modulation matrix into the softmax function to obtain the doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrix based on prompt information.

[0022] In the above-mentioned intelligent auxiliary system for diagnosing children's respiratory diseases, the auxiliary result generation and display module is used to: pass the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map through a classifier-based children's respiratory disease auxiliary diagnostic device to obtain the auxiliary diagnosis result, and the auxiliary diagnosis result is used to represent the respiratory disease type label of the child to be diagnosed; transmit the auxiliary diagnosis result to a display terminal for display.

[0023] Compared with the prior art, the present application provides an intelligent auxiliary system for diagnosing respiratory diseases in children, which obtains the doctor's consultation record information of the child to be diagnosed, and in response to the existence of auxiliary examination items in the doctor's consultation record information, obtains the auxiliary examination information of the auxiliary examination items of the child to be diagnosed, and uses image recognition and data analysis technology based on artificial intelligence to segment and aggregate the doctor's consultation record information with global semantic coding, and extracts implicit semantic features from the auxiliary examination information, so as to intelligently obtain the auxiliary diagnosis results of the respiratory disease type of the child to be diagnosed based on the cross-modal semantic joint features between the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination, and display the results. In this way, the patient's implicit disease characteristics can be captured more comprehensively and accurately, and the complex relationship between symptoms and examination results can be discovered, ensuring the integrity and coherence of the diagnostic information, thereby significantly improving the accuracy and comprehensiveness of auxiliary diagnosis of children's respiratory diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0025] Figure 1 This is a block diagram of an intelligent assistance system for diagnosing children's respiratory diseases according to an embodiment of the present application. Figure 2 The data of the intelligent auxiliary system for diagnosing children's respiratory diseases according to the embodiment of the present application

[0026] Flow diagram.

[0027] Figure 3 This is a block diagram of a consultation record information encoding module in an intelligent assistance system for diagnosing children's respiratory diseases according to an embodiment of the present application.

[0028] Figure 4 This is a block diagram of a cross-modal semantic union module in an intelligent assistance system for diagnosing children's respiratory diseases according to an embodiment of the present application.

[0029] Figure 5 This is a block diagram of a prompt guidance feature combination unit in an intelligent assistance system for diagnosing children's respiratory diseases according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0031] It should be noted that in this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where the application is located and with the authorization of the owner of the corresponding device.

[0032] Pediatric respiratory diseases are among the most common diseases in pediatrics, including pneumonia, asthma, bronchitis, acute upper respiratory tract infection, etc. However, in primary medical institutions, doctors lack clinical experience and traditional training methods have limited effects. In addition, artificial intelligence is rarely used in the diagnosis and treatment of pediatric respiratory diseases, especially difficult and complicated diseases. Children have limited expressive ability and limited radiological imaging examinations, which leads to challenges in intelligent diagnosis.

[0033] Based on this, Chinese patent CN109935336B proposes an intelligent auxiliary diagnosis system for children's respiratory diseases. It collects doctors' consultation records and auxiliary examination information, screens key information and uses a pre-trained BP neural network model to obtain diagnosis results, and displays the diagnosis results for doctors to operate and view. This improves the diagnostic accuracy of primary care doctors for difficult and complicated children's respiratory diseases and optimizes the allocation of medical resources.

[0034] The diagnosis results in the above patents are obtained by screening out key information from the medical consultation records and auxiliary information, and then inputting it into the BP neural network model for processing. However, although this method is simple, it has the following defects. Specifically, only by screening out key information and inputting it into the BP neural network model, it may not be possible to capture the deep implicit features in the doctor's medical consultation records and auxiliary examination information. For example, some subtle changes in physical signs, hidden factors in family medical history, etc. may be ignored, and these implicit features are crucial for accurate diagnosis. In addition, the doctor's medical consultation records and auxiliary examination information are processed separately, lacking a comprehensive analysis of these two different modal information. This split may lead to incomplete and incoherent information, and cannot fully reflect the patient's overall condition. For example, there may be complex associations between certain symptoms and examination results, and it is difficult to discover these associations by processing them separately, thus affecting the comprehensiveness and accuracy of the diagnosis.

[0035] Therefore, in response to the above technical problems, the technical concept of the present application is to obtain the doctor's consultation record information of the child to be diagnosed, and in response to the existence of auxiliary examination items in the doctor's consultation record information, obtain the auxiliary examination information of the auxiliary examination items of the child to be diagnosed, and use artificial intelligence-based image recognition and data analysis technology to segment the doctor's consultation record information and perform global semantic coding aggregation, and extract implicit semantic features from the auxiliary examination information, so as to intelligently obtain the auxiliary diagnosis results of the respiratory disease type of the child to be diagnosed based on the cross-modal semantic joint features between the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination, and display the results. In this way, the patient's implicit disease characteristics can be captured more comprehensively and accurately, and the complex relationship between symptoms and examination results can be discovered, ensuring the integrity and coherence of the diagnostic information, thereby significantly improving the accuracy and comprehensiveness of the auxiliary diagnosis of children's respiratory diseases.

[0036] Figure 1 This is a block diagram of an intelligent assistance system for diagnosing children's respiratory diseases according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of an intelligent auxiliary system for diagnosing pediatric respiratory diseases according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the intelligent auxiliary system 100 for diagnosing pediatric respiratory diseases includes: a diagnosis child consultation record acquisition module 110, which is used to obtain the doctor's consultation record information of the child to be diagnosed; an auxiliary examination information acquisition module 120, which is used to obtain the auxiliary examination information of the auxiliary examination items of the child to be diagnosed in response to the existence of auxiliary examination items in the doctor's consultation record information; a consultation record information encoding module 130, which is used to perform segmentation processing and global semantic encoding aggregation on the doctor's consultation record information to obtain the global semantic aggregation feature of the doctor's consultation record information; an auxiliary examination implicit feature extraction module 140, which is used to extract implicit features of the auxiliary examination information to obtain the auxiliary examination implicit semantic features; a cross-modal semantic union module 150, which is used to perform cross-modal semantic union on the global semantic aggregation feature of the doctor's consultation record information and the auxiliary examination implicit semantic features to obtain the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature; an auxiliary result generation and display module 160, which is used to obtain the auxiliary diagnosis result based on the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature, and display the auxiliary diagnosis result.

[0037] In an embodiment of the present application, the diagnosis child consultation record acquisition module 110 is used to obtain the doctor's consultation record information of the child to be diagnosed. It should be understood that the doctor's consultation record information includes the disease classification information of the child to be diagnosed, the physical condition information of the child to be diagnosed, the current doctor's preliminary examination information of the child to be diagnosed, the medical record information of the child to be diagnosed, and the auxiliary examination items that need to be examined. This information helps to fully understand the health status of the child and the factors that may affect the development of the disease. Based on this, in the technical solution of the present application, the doctor's consultation record information of the child to be diagnosed is obtained, so that rich information can be provided to support accurate diagnosis, so as to improve the accuracy and reliability of disease diagnosis.

[0038] In an embodiment of the present application, the auxiliary examination information acquisition module 120 is used to obtain the auxiliary examination information of the auxiliary examination items of the child to be diagnosed in response to the existence of auxiliary examination items in the doctor's consultation record information. Accordingly, considering that when the doctor believes that auxiliary examinations are needed to help diagnose during the consultation process, the auxiliary examination items that need to be performed will be recorded in the doctor's consultation record. Specifically, the "auxiliary examination items" here refer to the doctor's consideration of the need to further obtain more detailed information through certain specific auxiliary examination information (such as ultrasound examination, X-ray examination, CT examination or magnetic resonance imaging examination, etc.) based on the preliminary consultation results to assist the final diagnosis. Therefore, in the technical solution of the present application, in response to the existence of auxiliary examination items in the doctor's consultation record information, the auxiliary examination information of the auxiliary examination items of the child to be diagnosed is obtained. That is, the auxiliary examination information provides objective data on the patient's physical condition, which helps to exclude or confirm the possibility of certain diseases. For example, the CT image of the CT examination can intuitively display lung abnormalities.

[0039] In the embodiment of the present application, the medical consultation record information encoding module 130 is used to perform segmentation processing and global semantic encoding aggregation on the doctor's medical consultation record information to obtain the global semantic aggregation features of the doctor's medical consultation record information. Figure 3 FIG. 1 is a block diagram of a consultation record information encoding module in an intelligent auxiliary system for diagnosing pediatric respiratory diseases according to an embodiment of the present application. Specifically, Figure 3As shown, the consultation record information encoding module 130 includes: a consultation record information segment encoding unit 131, which is used to segment the doctor's consultation record information according to the record category dimension, and then semantically encode each segment of the doctor's consultation record information in the doctor's consultation record information to obtain a sequence of semantic feature vectors of the doctor's consultation record information segment; a consultation record information global semantic aggregation unit 132, which is used to pass the sequence of semantic feature vectors of the doctor's consultation record information segment through a doctor's consultation record global semantic aggregator based on a bidirectional LTSM model to obtain a doctor's consultation record information global semantic aggregation feature vector as the doctor's consultation record information global semantic aggregation feature. It should be understood that considering that the doctor's consultation record information contains various types of clinical information such as family history, allergy history, etc. In order to perform semantic analysis on each section of information of a specific category in more detail, so as to have a deeper understanding of the specific meaning and importance of each part of the information, in the technical solution of the present application, after the doctor's consultation record information is segmented according to the record category dimension, each section of the doctor's consultation record information in the doctor's consultation record information is semantically encoded to obtain a sequence of semantic feature vectors of the doctor's consultation record information segment. In particular, in a specific embodiment of the present application, a semantic encoder based on the BERT model can be used to perform semantic analysis and understanding on each section of the doctor's consultation record information to capture and mine the context information of the consultation information in each section to obtain a sequence of semantic feature vectors of the doctor's consultation record information segment. Next, considering that there is a contextual semantic association relationship between the consultation information in different sections of the sequence of semantic feature vectors of the doctor's consultation record information segment, for example, in the doctor's consultation record, the description of a symptom may be related to the previous medical history. And considering that the bidirectional LSTM model can capture information from both the front and back directions at the same time, it is better to understand and effectively capture the complex dependencies between long distances in the text. Therefore, in the technical solution of the present application, the sequence of semantic feature vectors of the doctor's consultation record information segments is passed through a doctor's consultation record global semantic aggregator based on a bidirectional LSTM model to obtain a global semantic aggregated feature vector of the doctor's consultation record information. In this way, the obtained aggregated vector not only contains the local information of each consultation information part, but also integrates the mutual semantic association relationship between different parts, thereby forming a comprehensive understanding of the entire consultation record.

[0040] In an embodiment of the present application, the auxiliary inspection implicit feature extraction module 140 is used to extract implicit features from the auxiliary inspection information to obtain auxiliary inspection implicit semantic features. Specifically, in an embodiment of the present application, the auxiliary inspection implicit feature extraction module is used to: pass the auxiliary inspection information through an auxiliary inspection implicit feature extractor based on a deep separable convolutional neural network model to obtain an auxiliary inspection implicit semantic feature map as the auxiliary inspection implicit semantic feature. Accordingly, considering that the auxiliary inspection information contains implicit semantic information such as edges, textures, organ structures, and lesion areas, and the deep separable convolutional neural network can provide rich implicit semantic features through multiple layers of deep separable convolutions, which helps to more accurately identify and locate lesion areas. Therefore, in the technical solution of the present application, the auxiliary inspection information is passed through an auxiliary inspection implicit feature extractor based on a deep separable convolutional neural network model to obtain an auxiliary inspection implicit semantic feature map to capture more subtle structural changes and abnormalities.

[0041] In an embodiment of the present application, the cross-modal semantic union module 150 is used to perform cross-modal semantic union on the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination to obtain the doctor's consultation record-auxiliary examination cross-modal semantic union encoding features. Figure 4 : is a block diagram of a cross-modal semantic association module in an intelligent auxiliary system for diagnosing pediatric respiratory diseases according to an embodiment of the present application. Specifically, Figure 4 As shown, the cross-modal semantic union module 150 includes: an auxiliary feature decomposition unit 151, which is used to perform local feature decomposition on the auxiliary examination implicit semantic features to obtain a set of auxiliary examination implicit semantic local features; a prompt-guided feature union unit 152, which is used to perform prompt-guided feature union on the global semantic aggregation features of the doctor's consultation record information and the set of auxiliary examination implicit semantic local features to obtain the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding features. Furthermore, considering that the global semantic aggregation features of the doctor's consultation record information and the auxiliary examination implicit semantic features provide data in two different modalities, text and image, respectively. And text information and image information are often complementary. For example, the symptom description in the doctor's consultation record can supplement the information not displayed in the image, and the image information can verify or refine the description in the text. Therefore, in order to effectively integrate the information of these two modalities and form a comprehensive feature representation to improve the accuracy and comprehensiveness of respiratory disease diagnosis, in the technical solution of the present application, the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination are cross-modal semantically combined to obtain a doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature map as the doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature.

[0042] Specifically, first, the auxiliary inspection implicit semantic features are subjected to local feature decomposition to obtain a set of auxiliary inspection implicit semantic local features. It should be understood that the auxiliary inspection implicit semantic features contain a large amount of local detail information, which is very important for the diagnosis of the disease. By performing local feature decomposition, these fine-grained local features can be extracted to obtain a set of auxiliary inspection implicit semantic local feature matrices as a set of auxiliary inspection implicit semantic local features, which is helpful for subsequent more refined cross-modal fine-grained interaction.

[0043] Next, the global semantic aggregation features of the doctor's consultation record information and the set of implicit semantic local features of the auxiliary examination are subjected to prompt-guided feature union to obtain the cross-modal semantic joint coding features of the doctor's consultation record-auxiliary examination. Specifically, first, the global semantic aggregation feature vector of the doctor's consultation record information is used as the query vector and the implicit semantic local feature matrix of each auxiliary examination is used as the key matrix, and the query vector and the key matrix are input into the prompt learning network based on the converter structure to obtain a set of doctor's consultation record-auxiliary examination local query prompt semantic coding vectors. That is, the core of the converter structure is the self-attention mechanism, which can automatically learn and highlight the information most relevant to the local features and query vectors, and the prompt learning network is used to enhance the key semantic interactions between the doctor's consultation record information and the auxiliary examinations. Then, a cross-modal mask weaving network based on prompt information is used to process the global semantic aggregation feature vector of the doctor's consultation record information and the local query prompt semantic encoding vector of each doctor's consultation record-auxiliary examination to dynamically adjust and focus on the most important features and the most critical information in the semantic interaction of each local feature and consultation record, so as to obtain a set of doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrices based on prompt information. Next, the point-by-point multiplication between each group of corresponding doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrices based on prompt information and the auxiliary examination implicit semantic local feature matrix is ​​calculated to obtain a set of doctor's consultation record-auxiliary examination cross-modal local granular significant interaction matrices. In this way, the weight values ​​in the mask weight matrix can be applied to the auxiliary examination implicit semantic local feature matrix, thereby emphasizing those local features that have a greater impact on the diagnosis results. Finally, the set of doctor's consultation records-auxiliary examinations is feature aggregated to obtain a doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map containing multimodal information from text and images, which can more comprehensively describe the condition of pediatric patients. In particular, in the technical solution of the present application, the individual doctor's consultation record-auxiliary examination cross-modal local granularity significant interaction matrices in the set of doctor's consultation record-auxiliary examination cross-modal local granularity significant interaction matrices are spliced ​​along the channel dimension to obtain a doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map.

[0044] Specifically, the auxiliary feature decomposition unit 151 is used to perform local feature decomposition along the channel dimension on the auxiliary inspection latent semantic feature map to obtain a set of auxiliary inspection latent semantic local feature matrices as the set of auxiliary inspection latent semantic local features.

[0045] Figure 5 1 is a block diagram of a prompting and guiding feature combining unit in an intelligent auxiliary system for diagnosing pediatric respiratory diseases according to an embodiment of the present application. Figure 5 As shown, the prompt-guided feature combination unit 152 includes: a doctor's consultation record-auxiliary examination query prompt subunit 1521, which is used to use the global semantic aggregation feature vector of the doctor's consultation record information as the query vector and each auxiliary examination implicit semantic local feature matrix in the set of the auxiliary examination implicit semantic local feature matrix as the key matrix, and input the query vector and the key matrix into a prompt learning network based on a converter structure to obtain a set of doctor's consultation record-auxiliary examination local query prompt semantic coding vectors; a doctor's consultation record-auxiliary examination feature mask subunit 1522, which is used to input the global semantic aggregation feature vector of the doctor's consultation record information and each doctor's consultation record-auxiliary examination local query prompt semantic coding vector in the set of the doctor's consultation record-auxiliary examination local query prompt semantic coding vector into a cross-modal mask weaving network based on prompt information to obtain a doctor's consultation record based on prompt information. A set of doctor's consultation records-auxiliary examination cross-modal local feature mask weight matrices; a doctor's consultation record-auxiliary examination feature fine-grained interaction sub-unit 1523, which is used to calculate the point-by-point multiplication between the doctor's consultation records-auxiliary examination cross-modal local feature mask weight matrices based on prompt information and the set of the auxiliary examination implicit semantic local feature matrices to obtain a set of doctor's consultation records-auxiliary examination cross-modal local granularity significant interaction matrices; a doctor's consultation record-auxiliary examination feature joint sub-unit 1524, which is used to perform feature aggregation on the set of the doctor's consultation records-auxiliary examination cross-modal local granularity significant interaction matrices to obtain a doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map as the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature.

[0046] Specifically, in an embodiment of the present application, the doctor's consultation record-auxiliary examination query prompt subunit 1521 includes: a doctor's consultation record-auxiliary examination similarity score calculation secondary subunit, which is used to calculate the similarity score between the global semantic aggregation feature vector of the doctor's consultation record information and each row vector in the auxiliary examination implicit semantic local feature matrix to obtain a set of doctor's consultation record-auxiliary examination similarity scores; a doctor's consultation record-auxiliary examination weight calculation secondary subunit, which is used to standardize each doctor's consultation record-auxiliary examination similarity score in the set of doctor's consultation record-auxiliary examination similarity scores to obtain a set of doctor's consultation record-auxiliary examination weight coefficients; a doctor's consultation record-auxiliary examination feature weighting secondary subunit, which is used to calculate the weighted sum of each row vector in the auxiliary examination implicit semantic local feature matrix based on the set of doctor's consultation record-auxiliary examination weight coefficients to obtain the doctor's consultation record-auxiliary examination local query prompt semantic encoding vector.

[0047] More specifically, in an embodiment of the present application, the secondary sub-unit for calculating the similarity score of the doctor's consultation record-auxiliary examination is used to: calculate the binary norm of each row vector in the auxiliary examination implicit semantic local feature matrix to obtain a set of auxiliary examination implicit semantic local norm values; calculate the binary norm of the global semantic aggregate feature vector of the doctor's consultation record information to obtain the global semantic norm value of the doctor's consultation record information; multiply the global semantic norm value of the doctor's consultation record information with the set of the auxiliary examination implicit semantic local norm values ​​by position to obtain a set of doctor's consultation record-auxiliary examination joint values; calculate the product between the global semantic aggregate feature vector of the doctor's consultation record information and the transposed vector of each row vector in the auxiliary examination implicit semantic local feature matrix to obtain a set of doctor's consultation record-auxiliary examination related values; divide the set of the doctor's consultation record-auxiliary examination related values ​​and the set of the doctor's consultation record-auxiliary examination joint values ​​by each group of corresponding doctor's consultation record-auxiliary examination related values ​​and the doctor's consultation record-auxiliary examination joint values ​​to obtain the set of doctor's consultation record-auxiliary examination similarity scores.

[0048] More specifically, in an embodiment of the present application, the doctor's consultation record-auxiliary examination weight calculation secondary sub-unit is used to: calculate the natural exponential function value with the natural constant e as the base and each doctor's consultation record-auxiliary examination similarity score in the set of doctor's consultation record-auxiliary examination similarity scores as the exponential power to obtain a set of doctor's consultation record-auxiliary examination similarity modulation scores; calculate the set of doctor's consultation record-auxiliary examination similarity modulation scores point by point to obtain the total sum of doctor's consultation record-auxiliary examination similarity scores; divide the set of doctor's consultation record-auxiliary examination similarity modulation scores by the total sum of doctor's consultation record-auxiliary examination similarity scores by position to obtain the set of doctor's consultation record-auxiliary examination weight coefficients.

[0049] Specifically, in an embodiment of the present application, the doctor's consultation record-auxiliary examination feature mask sub-unit is used to: multiply the transposed vector of the doctor's consultation record-auxiliary examination local query prompt semantic encoding vector with the global semantic aggregation feature vector of the doctor's consultation record information to obtain a doctor's consultation record-auxiliary examination prompt association matrix; divide the doctor's consultation record-auxiliary examination prompt association matrix by the square root of the scale of the doctor's consultation record-auxiliary examination prompt association matrix by position to obtain a doctor's consultation record-auxiliary examination prompt association modulation matrix; input the doctor's consultation record-auxiliary examination prompt association modulation matrix into the softmax function to obtain the doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrix based on prompt information.

[0050] In the embodiment of the present application, specifically, the cross-modal semantic union of the global semantic aggregation feature vector of the doctor's consultation record information and the implicit semantic feature graph of the auxiliary examination can be expressed by the formula: ;

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] in, is the decomposition function, is the auxiliary inspection implicit semantic feature map, , , and are the first, second, and third in the set of the auxiliary inspection implicit semantic local feature matrix. and Auxiliary check latent semantic local feature matrix, , , and They are The first, second, and and row vectors, for coding, is the global semantic aggregation feature vector of the doctor's consultation record information, yes The transposed vector of is matrix multiplication, is the two-norm of the vector, for and The doctor consultation record-auxiliary examination similarity score between represents the exponential function value with the natural constant e as the base, for and Doctor consultation record-auxiliary examination similarity score between for Corresponding doctor consultation record-auxiliary examination weight coefficient, yes The number of row vectors in It is the semantic encoding vector of the doctor's consultation record-auxiliary examination local query prompt, for The transposed vector of The scale of the doctor's consultation record-auxiliary examination prompt association matrix is ​​the length of the matrix multiplied by the width of the matrix. yes function, yes The corresponding doctor consultation record-auxiliary examination cross-modal local feature mask weight matrix based on prompt information, , and They are , ,and The corresponding doctor consultation record-auxiliary examination cross-modal local feature mask weight matrix based on prompt information, For point-by-position multiplication, is the feature concatenation along the channel dimension, It is the cross-modal semantic joint encoding feature map of the doctor's consultation record-auxiliary examination.

[0056] In an embodiment of the present application, the auxiliary result generation and display module 160 is used to obtain an auxiliary diagnosis result based on the cross-modal semantic joint coding feature of the doctor's consultation record-auxiliary examination, and to display the auxiliary diagnosis result. Specifically, in an embodiment of the present application, the auxiliary result generation and display module is used to: pass the cross-modal semantic joint coding feature map of the doctor's consultation record-auxiliary examination through a classifier-based children's respiratory disease auxiliary diagnostic device to obtain the auxiliary diagnosis result, and the auxiliary diagnosis result is used to represent the respiratory disease type label of the child to be diagnosed; transmit the auxiliary diagnosis result to the display terminal for display. Pass the cross-modal semantic joint coding feature map of the doctor's consultation record-auxiliary examination through a classifier-based children's respiratory disease auxiliary diagnostic device to obtain an auxiliary diagnosis result, and the auxiliary diagnosis result is used to represent the respiratory disease type label of the child to be diagnosed. That is, the cross-modal semantic joint coding feature of the doctor's consultation record-auxiliary examination obtained by cross-modal combination of the global semantic aggregation feature vector of the doctor's consultation record information and the implicit semantic feature map of the auxiliary examination is classified and processed, so as to intelligently obtain the auxiliary diagnosis result of the respiratory disease type of the child to be diagnosed. In this way, the patient's implicit disease characteristics can be captured more comprehensively and accurately, and the complex relationship between symptoms and examination results can be discovered, ensuring the integrity and consistency of diagnostic information, thereby significantly improving the accuracy and comprehensiveness of auxiliary diagnosis of children's respiratory diseases.

[0057] In particular, in a specific embodiment of the present application, the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map is passed through a classifier-based children's respiratory disease auxiliary diagnostic device to obtain an auxiliary diagnosis result, including: expanding the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map into a doctor's consultation record-auxiliary examination cross-modal semantic joint encoding classification feature vector; using the fully connected layer of the classifier-based children's respiratory disease auxiliary diagnostic device to fully connect the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding classification feature vector to obtain a doctor's consultation record-auxiliary examination cross-modal semantic joint fully connected encoding classification feature vector; inputting the doctor's consultation record-auxiliary examination cross-modal semantic joint fully connected encoding classification feature vector into the Softmax classification function of the classifier-based children's respiratory disease auxiliary diagnostic device to obtain the probability value of the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map belonging to each classification label, and the classification label includes a classification label used to represent the type of respiratory disease of the child to be diagnosed; and determining the classification label corresponding to the largest of the probability values ​​as the auxiliary diagnosis result.

[0058] Specifically, the auxiliary diagnosis result is transmitted to the display terminal for display. It should be understood that in order to help doctors make diagnoses more efficiently and accurately, especially when facing complex pediatric respiratory diseases, in the technical solution of the present application, the auxiliary diagnosis result is transmitted to the display terminal for display, so as to improve the doctor's diagnostic ability and efficiency. In particular, in a specific embodiment of the present application, the display of the auxiliary diagnosis result is completed by a special module, which presents the processed information to the doctor in a clear and intuitive manner. In the display, the system lists all possible disease diagnoses and gives a corresponding diagnostic weight to each disease. This weight reflects the system's information level on the auxiliary diagnosis result (the doctor or the auxiliary diagnosis system's confidence level in the diagnosis made). For example, the display terminal may display "pneumonia + diagnostic weight" and list the main symptoms of the disease in detail. Among these symptoms, those that have been reflected in the current patient will be specially marked (such as blue highlight), while those that have not been reflected will be displayed in gray for doctors to quickly identify. In addition, if the diagnosis results include difficult and complicated cases, a red reminder will be displayed in the upper right corner of the display terminal to attract the doctor's attention, prompting them that further examinations or expert consultations may be needed. This design is intended to help doctors use auxiliary diagnostic information more effectively while staying alert to complex cases. Therefore, the entire display process is designed to provide a comprehensive and easy-to-understand diagnostic overview to support doctors in making more accurate clinical decisions.

[0059] It should be understandable that when the global semantic aggregation feature vector of the doctor's consultation record information and the auxiliary examination implicit semantic feature map respectively represent the semantic space domain short-range and long-range bidirectional association aggregation semantic features of the doctor's consultation record information and the image semantic features of the auxiliary examination information, when performing cross-modal joint encoding based on continuous prompt assistance, the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map will also have the distribution diversity of cross-modal information under the multi-dimensional feature distribution. Therefore, when the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map is classified by a classifier, it affects the accuracy of the classification result.

[0060] Preferably, the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map is passed through a classifier-based children's respiratory disease auxiliary diagnosis device to obtain an auxiliary diagnosis result, which includes:

[0061] Determine the feature mean and feature variance of the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map, and divide the feature mean by the feature variance to obtain the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding probability statistical value. This process can be expressed by the formula:

[0062] ;

[0063] in, is the feature mean of the cross-modal semantic joint encoding feature map of the doctor's consultation record-auxiliary examination, The feature variance of the cross-modal semantic joint encoding feature map of the doctor's consultation record-auxiliary examination, Probability statistics of the cross-modal semantic joint encoding of the doctor's consultation record-auxiliary examination;

[0064] The doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature map is multiplied by the inverse of the maximum eigenvalue in the doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature map to obtain the doctor's consultation record-auxiliary examination cross-modal semantic joint coding probability constraint map. The process can be expressed by the formula:

[0065] ;

[0066] in, The doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map, It means point multiplication by position. represents the inverse of the maximum eigenvalue in the cross-modal semantic joint encoding feature map of the doctor's consultation record-auxiliary examination, A probability constraint graph for cross-modal semantic joint encoding of the doctor's consultation record-auxiliary examination;

[0067] After dot-adding the doctor's consultation record-auxiliary examination cross-modal semantic joint coding probability constraint graph and the doctor's consultation record-auxiliary examination cross-modal semantic joint coding probability statistical value, the logarithmic value with base 2 is calculated to obtain the doctor's consultation record-auxiliary examination cross-modal semantic joint coding information interaction graph. This process can be expressed by the formula:

[0068] ;

[0069] in, The doctor's consultation record-auxiliary examination cross-modal semantic joint encoding probability constraint graph is: Indicates adding by position point. represents the probability statistic value of the cross-modal semantic joint encoding of the doctor's consultation record-auxiliary examination, represents the logarithmic function with base 2, The doctor's consultation record-auxiliary examination cross-modal semantic joint encoding information interaction diagram;

[0070] After performing point subtraction on the probability constraint graph of cross-modal semantic joint coding of doctor's consultation record and auxiliary examination with the probability statistics of cross-modal semantic joint coding of doctor's consultation record and auxiliary examination, the inverse of each eigenvalue is calculated to obtain the sequence constraint graph of cross-modal semantic joint coding of doctor's consultation record and auxiliary examination. The process can be expressed by the formula:

[0071] ;

[0072] in, The probability statistics of the cross-modal semantic joint encoding of the doctor's consultation record-auxiliary examination, It means to subtract by position point. The doctor's consultation record-auxiliary examination cross-modal semantic joint encoding probability constraint graph is: To calculate the inverse of each feature in the feature map, The doctor's consultation record-auxiliary examination cross-modal semantic joint encoding sequence constraint graph;

[0073] The doctor's consultation record-auxiliary examination cross-modal semantic joint coding information interaction graph and the doctor's consultation record-auxiliary examination cross-modal semantic joint coding sequence constraint graph are interpolated to obtain an optimized doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature graph;

[0074] The optimized doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map is passed through a classifier-based children's respiratory disease auxiliary diagnostic device to obtain an auxiliary diagnosis result.

[0075] That is, when the global semantic aggregation feature vector of the doctor's consultation record information and the auxiliary examination implicit semantic feature map respectively represent the semantic space domain short-range and long-range bidirectional association aggregation semantic features of the doctor's consultation record information and the image semantic features of the auxiliary examination information, when performing cross-modal joint encoding based on continuous prompt assistance, the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map will also have the distribution diversity of cross-modal information under the multi-dimensional feature distribution. Therefore, when the doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map is classified by a classifier, it affects the accuracy of the classification result.

[0076] Therefore, the present application takes into account that when the weight matrix of the classifier acts on the doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature vector after the doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature map is expanded, the distribution diversity of the doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature map causes the uncertainty of the weight stimulus parameters of the weight matrix of the classifier, resulting in the lack of posterior probability density inference of the doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature map through the weight matrix, which will affect the accuracy of the classification result.

[0077] In this case, the probability statistical characteristics of the cross-modal semantic joint coding feature map of the doctor's consultation record-auxiliary examination are used to simulate the mesoscale interaction structure under probability constraints between the feature value scale and the feature map scale of the cross-modal semantic joint coding feature map of the doctor's consultation record-auxiliary examination, so as to construct a bidirectional latent variable motif based on the mesoscale short sequence relative to the probability statistical value to perform mesoscale bidirectional migration, and perform posterior recovery based on the sequence constraints on the interactive information, so as to improve the convergence effect in the probability density domain, and improve the accuracy of the auxiliary diagnosis results obtained by the classifier-based auxiliary diagnosis device for children's respiratory diseases of the cross-modal semantic joint coding feature map of the doctor's consultation record-auxiliary examination. In this way, the implicit disease characteristics of the patient can be captured more comprehensively and accurately, and the complex relationship between symptoms and examination results can be discovered, ensuring the integrity and coherence of the diagnostic information, thereby significantly improving the accuracy and comprehensiveness of the auxiliary diagnosis of children's respiratory diseases.

[0078] In summary, the intelligent auxiliary system 100 for diagnosing children's respiratory diseases based on the embodiment of the present application is explained, which obtains the doctor's consultation record information of the child to be diagnosed, and in response to the existence of auxiliary examination items in the doctor's consultation record information, obtains the auxiliary examination information of the auxiliary examination items of the child to be diagnosed, and uses image recognition and data analysis technology based on artificial intelligence to segment the doctor's consultation record information and aggregate global semantic coding, and extracts implicit semantic features from the auxiliary examination information, so as to intelligently obtain the auxiliary diagnosis result of the respiratory disease type of the child to be diagnosed based on the cross-modal semantic joint features between the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination, and display the result. In this way, the patient's implicit disease characteristics can be captured more comprehensively and accurately, and the complex relationship between symptoms and examination results can be discovered, ensuring the integrity and coherence of the diagnostic information, thereby significantly improving the accuracy and comprehensiveness of the auxiliary diagnosis of children's respiratory diseases.

[0079] As described above, the intelligent assistance system 100 for diagnosing children's respiratory diseases according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an intelligent assistance algorithm for diagnosing children's respiratory diseases. In one possible implementation, the intelligent assistance system 100 for diagnosing children's respiratory diseases according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the intelligent assistance system 100 for diagnosing children's respiratory diseases can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the intelligent assistance system 100 for diagnosing children's respiratory diseases can also be one of the many hardware modules of the wireless terminal.

[0080] Alternatively, in another example, the intelligent assistance system 100 for diagnosing children's respiratory diseases and the wireless terminal may also be separate devices, and the intelligent assistance system 100 for diagnosing children's respiratory diseases may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0081] The above descriptions of various implementations of the present disclosure are exemplary and not exhaustive. They are not limited to the disclosed implementations, and many modifications and variations are obvious to those skilled in the art without departing from the scope and spirit of the described implementations.

Claims

1. An intelligent auxiliary system for diagnosing children's respiratory diseases, characterized in that: include: The diagnosis and consultation record acquisition module for children is used to obtain the doctor's consultation record information of the child to be diagnosed; An auxiliary examination information acquisition module, for acquiring auxiliary examination information of the auxiliary examination items of the child to be diagnosed in response to the existence of auxiliary examination items in the doctor's consultation record information; A consultation record information encoding module, used for performing segmentation processing and global semantic encoding aggregation on the doctor's consultation record information to obtain global semantic aggregation features of the doctor's consultation record information; An auxiliary inspection implicit feature extraction module is used to extract implicit features from the auxiliary inspection information to obtain auxiliary inspection implicit semantic features; A cross-modal semantic union module is used to perform cross-modal semantic union on the global semantic aggregation features of the doctor's consultation record information and the implicit semantic features of the auxiliary examination to obtain cross-modal semantic joint coding features of the doctor's consultation record-auxiliary examination, wherein the cross-modal semantic union module includes: an auxiliary feature decomposition unit, which is used to perform local feature decomposition on the implicit semantic features of the auxiliary examination to obtain a set of auxiliary examination implicit semantic local features; a prompt-guided feature union unit, which is used to perform prompt-guided feature union on the global semantic aggregation features of the doctor's consultation record information and the set of auxiliary examination implicit semantic local features to obtain the doctor's consultation record-auxiliary examination cross-modal semantic joint coding features; An auxiliary result generation and display module is used to obtain auxiliary diagnosis results based on the cross-modal semantic joint coding features of the doctor's consultation record-auxiliary examination, and display the auxiliary diagnosis results; Wherein, the medical consultation record information encoding module includes: A medical consultation record information segmentation encoding unit, used for segmenting the doctor's medical consultation record information according to the record category dimension, and then semantically encoding each segment of the doctor's medical consultation record information in the doctor's medical consultation record information to obtain a sequence of semantic feature vectors of the doctor's medical consultation record information segments; A global semantic aggregation unit for the consultation record information is used to pass the sequence of the semantic feature vectors of the doctor's consultation record information segments through a doctor's consultation record global semantic aggregator based on a bidirectional LTSM model to obtain a doctor's consultation record information global semantic aggregation feature vector as the doctor's consultation record information global semantic aggregation feature; The auxiliary inspection implicit feature extraction module is used to: pass the auxiliary inspection information through an auxiliary inspection implicit feature extractor based on a deep separable convolutional neural network model to obtain an auxiliary inspection implicit semantic feature map as the auxiliary inspection implicit semantic feature; The auxiliary feature decomposition unit is used to: perform local feature decomposition along the channel dimension on the auxiliary inspection latent semantic feature map to obtain a set of auxiliary inspection latent semantic local feature matrices as the set of auxiliary inspection latent semantic local features; Wherein, the prompt guidance feature combination unit includes: A doctor's consultation record-auxiliary examination query prompt subunit is used to use the global semantic aggregation feature vector of the doctor's consultation record information as a query vector and each auxiliary examination implicit semantic local feature matrix in the set of auxiliary examination implicit semantic local feature matrices as a key matrix, and input the query vector and the key matrix into a prompt learning network based on a converter structure to obtain a set of doctor's consultation record-auxiliary examination local query prompt semantic encoding vectors; A doctor's consultation record-auxiliary examination feature mask subunit, used for inputting each doctor's consultation record-auxiliary examination local query prompt semantic encoding vector in the set of the doctor's consultation record information global semantic aggregation feature vector and the doctor's consultation record-auxiliary examination local query prompt semantic encoding vector into a cross-modal mask weaving network based on prompt information to obtain a set of doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrices based on prompt information; A doctor's consultation record-auxiliary examination feature fine-grained interaction subunit, used to calculate the set of the doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrices based on prompt information and the set of the auxiliary examination implicit semantic local feature matrices, and to obtain a set of doctor's consultation record-auxiliary examination cross-modal local granular significant interaction matrices by multiplying the position points between each corresponding group of the doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrices based on prompt information and the auxiliary examination implicit semantic local feature matrices; The doctor's consultation record-auxiliary examination feature joint subunit is used to perform feature aggregation on the set of the doctor's consultation record-auxiliary examination cross-modal local granularity significant interaction matrices to obtain a doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature map as the doctor's consultation record-auxiliary examination cross-modal semantic joint coding feature.

2. The intelligent auxiliary system for diagnosing children's respiratory diseases according to claim 1, characterized in that: The doctor consultation record-auxiliary examination query prompt subunit includes: A secondary subunit for calculating the similarity score of doctor's consultation record-auxiliary examination is used to calculate the similarity score between the global semantic aggregation feature vector of the doctor's consultation record information and each row vector in the implicit semantic local feature matrix of the auxiliary examination to obtain a set of similarity scores of doctor's consultation record-auxiliary examination; A doctor's consultation record-auxiliary examination weight calculation secondary subunit is used to standardize each doctor's consultation record-auxiliary examination similarity score in the set of doctor's consultation record-auxiliary examination similarity scores to obtain a set of doctor's consultation record-auxiliary examination weight coefficients; The doctor's consultation record-auxiliary examination feature weighted secondary sub-unit is used to calculate the weighted sum of each row vector in the auxiliary examination implicit semantic local feature matrix based on the set of doctor's consultation record-auxiliary examination weight coefficients to obtain the doctor's consultation record-auxiliary examination local query prompt semantic encoding vector.

3. The intelligent auxiliary system for diagnosing children's respiratory diseases according to claim 2, characterized in that: The doctor consultation record-auxiliary examination similarity score calculation secondary subunit is used to: Calculating the binary norm of each row vector in the auxiliary inspection latent semantics local feature matrix to obtain a set of auxiliary inspection latent semantics local norm values; Calculating the binary norm of the global semantic aggregation feature vector of the doctor's consultation record information to obtain the global semantic norm value of the doctor's consultation record information; Multiplying the global semantic norm value of the doctor's consultation record information and the set of the auxiliary examination implicit semantic local norm values ​​by position to obtain a set of doctor's consultation record-auxiliary examination joint values; Respectively calculating the product between the global semantic aggregated feature vector of the doctor's consultation record information and the transposed vector of each row vector in the auxiliary examination implicit semantic local feature matrix to obtain a set of doctor's consultation record-auxiliary examination related values; Divide the set of the doctor's consultation record-auxiliary examination related values ​​and the set of the doctor's consultation record-auxiliary examination combined values ​​by the doctor's consultation record-auxiliary examination related values ​​and the doctor's consultation record-auxiliary examination combined values ​​corresponding to each group to obtain the set of the doctor's consultation record-auxiliary examination similarity scores.

4. The intelligent auxiliary system for diagnosing children's respiratory diseases according to claim 3, characterized in that: The doctor consultation record-auxiliary examination weight calculation secondary subunit is used to: Calculate the natural exponential function value with the natural constant e as the base and each doctor's consultation record-auxiliary examination similarity score in the set of doctor's consultation record-auxiliary examination similarity scores as the exponential power to obtain a set of doctor's consultation record-auxiliary examination similarity modulation scores; Calculating the set of doctor's consultation record-auxiliary examination similarity modulation scores point by point to obtain the total sum of doctor's consultation record-auxiliary examination similarity scores; The set of the doctor's consultation record-auxiliary examination similarity modulation scores is divided by the total sum of the doctor's consultation record-auxiliary examination similarity scores by position to obtain the set of the doctor's consultation record-auxiliary examination weight coefficients.

5. The intelligent auxiliary system for diagnosing children's respiratory diseases according to claim 4, characterized in that: The doctor consultation record-auxiliary examination feature mask subunit is used to: Multiplying the transposed vector of the doctor's consultation record-auxiliary examination local query prompt semantic encoding vector by the doctor's consultation record information global semantic aggregation feature vector to obtain a doctor's consultation record-auxiliary examination prompt association matrix; Dividing the doctor's consultation record-auxiliary examination prompt association matrix by the square root of the scale of the doctor's consultation record-auxiliary examination prompt association matrix by position to obtain a doctor's consultation record-auxiliary examination prompt association modulation matrix; The doctor's consultation record-auxiliary examination prompt association modulation matrix is ​​input into the softmax function to obtain the doctor's consultation record-auxiliary examination cross-modal local feature mask weight matrix based on the prompt information.

6. The intelligent auxiliary system for diagnosing children's respiratory diseases according to claim 5, characterized in that: The auxiliary result generation and display module is used to: The doctor's consultation record-auxiliary examination cross-modal semantic joint encoding feature map is passed through a classifier-based children's respiratory disease auxiliary diagnosis device to obtain the auxiliary diagnosis result, and the auxiliary diagnosis result is used to represent the respiratory disease type label of the child to be diagnosed; The auxiliary diagnosis result is transmitted to a display terminal for display.

Images