Disease condition detection device based on multi-modal image, disease condition detection method and medium

Abstract

The embodiment of the invention discloses illness state detection equipment based on a multi-modal image, an illness state detection method and a medium. A specific embodiment of the method comprises the following steps: acquiring a patient historical medical record information set and current patient physical state information; sensitive data desensitization processing is conducted on the historical medical record information set of the patient, and a desensitized historical medical record information set of the patient is obtained; inputting the current patient physical state information and the desensitized patient historical medical record information set into an illness state detection model to obtain current illness state information; performing causal relationship extraction on the desensitized patient historical medical record information set to obtain a disease condition time sequence causal diagram; inputting the disease time sequence causal diagram, the desensitized patient historical medical record information set and the current disease information into a disease prediction model to obtain disease development prediction information; and performing illness state grade evaluation on the illness state development prediction information, and sending illness state early warning information. According to the embodiment, the accuracy of illness state detection and prediction can be improved, and the accuracy of early warning information sending is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a disease detection device, a disease detection method, and a medium based on multimodal images. Background Art

[0002] At present, with the development of deep learning technology and the rapid accumulation of disease data, disease detection and disease prediction based on artificial intelligence have become the focus of people's attention. Disease detection and disease prediction based on artificial intelligence can assist doctors in detection, speed up detection efficiency and improve detection and prediction accuracy. For disease detection, the usual method is to detect the disease through a single historical disease image data to obtain the current disease information. Then, the current disease information and historical disease image data are input into the time series prediction model to obtain disease development information, and send warning information through the disease development information.

[0003] However, it is found in practice that when the above method is used to predict the condition, the following technical problems often occur: Since the content of a single disease image data is relatively simple and cannot fully display the patient's true condition, it is easy to miss or misjudge some key features related to the disease, resulting in low accuracy of disease detection. In addition, when predicting through a time series prediction model, there may be a large amount of interference information in the input disease information that is not related to the current disease, resulting in low accuracy of the predicted disease information, reducing the accuracy of sending early warning information, and wasting a lot of transmission and computing resources.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a disease condition detection device, a disease condition detection method, and a medium based on multimodal images to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a disease condition detection device based on multimodal images, the device comprising: a medical record data storage device, a data desensitization device, a disease condition detection main control chip, and a disease condition warning device, wherein: the above-mentioned disease condition detection main control chip is used to obtain the patient's historical medical record information set and the current patient's physical condition information from the above-mentioned medical record data storage device, wherein the historical medical record information in the above-mentioned patient's historical medical record information set comprises: historical multimodal disease condition image data, historical disease condition description text information corresponding to the above-mentioned historical multimodal disease condition image data, and the above-mentioned historical multimodal disease condition image data comprises at least one of the following: historical patient ultrasound images, historical patient magnetic resonance imaging images, and historical patient computer tomography images; the above-mentioned data desensitization device is used to perform sensitive data desensitization processing on the above-mentioned patient's historical medical record information set to obtain the patient's historical medical record information after desensitization. Information collection; the above-mentioned condition detection main control chip is used to input the above-mentioned current patient physical state information, the above-mentioned desensitized historical multimodal condition image data set and the above-mentioned desensitized historical condition description text information set included in the above-mentioned desensitized patient historical medical record information set into the condition detection model to obtain current condition information; the above-mentioned condition detection main control chip is used to extract the causal relationship of the above-mentioned desensitized patient historical medical record information set to obtain a condition time series causal graph; the above-mentioned condition detection main control chip is used to input the above-mentioned condition time series causal graph, the above-mentioned desensitized patient historical medical record information set and the above-mentioned current condition information into the condition prediction model to obtain condition development prediction information within the prediction time period; the above-mentioned condition detection main control chip is used to evaluate the condition level of the above-mentioned condition development prediction information to obtain condition level information; the above-mentioned condition warning device is used to send condition warning information of different levels according to the above-mentioned condition level information.

[0008] In a second aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0009] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0010] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the disease detection device based on multimodal images of some embodiments of the present disclosure can improve the accuracy of disease detection and prediction, and improve the accuracy of sending warning information. Specifically, the reasons for the low accuracy of the relevant predicted disease information, the low accuracy of sending warning information, and the waste of a large amount of transmission and computing resources are: because the content included in the single disease image data is relatively simple, it cannot fully display the patient's true condition, and it is easy to miss or misjudge some key features related to the disease, resulting in low accuracy of disease detection, and through the time series prediction model, the input disease information may contain a large amount of interference information that is irrelevant to the current disease, resulting in low accuracy of predicted disease information, reducing the accuracy of sending warning information, and wasting a lot of transmission and computing resources. Based on this, some embodiments of the present disclosure are based on multimodal image disease detection devices, including: medical record data storage device, data desensitization device, disease detection main control chip, disease warning device, wherein: the above-mentioned disease detection main control chip is used to obtain the patient's patient history medical record information set and the current patient's physical state information from the above-mentioned medical record data storage device, wherein the historical medical record information in the above-mentioned patient history medical record information set includes: historical multimodal disease image data, historical disease description text information corresponding to the above-mentioned historical multimodal disease image data, and the above-mentioned historical multimodal disease image data includes at least one of the following: historical patient ultrasound image, historical patient magnetic resonance imaging image, and historical patient computer tomography image. Here, the historical medical record information includes a variety of multimodal data related to the patient's disease, which can fully understand the patient's disease condition, and the current patient's physical state information is convenient for subsequent disease detection. The above-mentioned data desensitization device is used to perform sensitive data desensitization processing on the above-mentioned patient history medical record information set to obtain the patient's history medical record information set after desensitization. Here, the data security of the patient's history medical record information set can be improved, and the leakage of sensitive data can be reduced to a certain extent. The above-mentioned main control chip for disease detection is used to input the above-mentioned current patient's physical state information, the above-mentioned historical medical record information set after desensitization, the historical multimodal disease image data set after desensitization, and the historical disease description text information set after desensitization into the disease detection model to obtain the current disease information. Here, the accuracy of detecting the current disease information can be improved. The above-mentioned main control chip for disease detection is used to extract the causal relationship of the above-mentioned historical medical record information set after desensitization, and obtain the disease time series causal graph. Here, some interference information irrelevant to the patient's condition can be removed, the amount of data can be reduced, and the quality of the data subsequently input into the disease prediction model can be improved, reducing the waste of computing resources and improving the detection efficiency. The above-mentioned main control chip for disease detection is used to input the above-mentioned disease time series causal graph, the above-mentioned historical medical record information set after desensitization, and the above-mentioned current disease information into the disease prediction model to obtain the disease development prediction information within the prediction time period. Here, the accuracy of disease prediction can be improved.The above-mentioned disease detection main control chip is used to evaluate the disease level of the above-mentioned disease development prediction information to obtain the disease level information. Here, the severity of the patient's disease can be determined so that accurate prediction and judgment can be made later. The above-mentioned disease warning device is used to send different levels of disease warning information according to the above-mentioned disease level information. Here, different levels of warnings for patients can improve the accuracy of warning information and improve user experience. It can be obtained that the disease detection device based on multimodal images can improve the accuracy of disease detection and prediction, and improve the accuracy of sending warning information. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0012] Figure 1 is a schematic structural diagram of some embodiments of a disease condition detection device based on multimodal images according to the present disclosure;

[0013] Figure 2 is a schematic structural diagram of other embodiments of a disease condition detection device based on multimodal images according to the present disclosure;

[0014] Figure 3 is a flow chart of some embodiments of a disease condition detection method of a disease condition detection device based on multimodal images according to the present disclosure;

[0015] Figure 4 is a schematic diagram of historical multimodal disease condition image data of an illustrative internal test in some embodiments of a disease condition detection device based on multimodal images according to the present disclosure;

[0016] Figure 5 is a screenshot of an illustrative internal test for generating disease progression prediction information of a patient in some embodiments of the disease detection device based on multimodal images according to the present disclosure;

[0017] Figure 6 is a screenshot of an illustrative internal test for generating prediction information of the disease progression of another patient in some embodiments of the multimodal image-based disease detection device according to the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] First, see Figure 1 , Figure 1 A structural schematic diagram 100 of some embodiments of a disease detection device based on multimodal images according to the present disclosure is shown. The disease detection device based on multimodal images includes: a medical record data storage device 101, a data desensitization device 102, a disease detection main control chip 103, and a disease warning device 104.

[0025] The condition detection main control chip 103 is used to obtain the patient's historical medical record information set and the current patient's physical condition information from the above-mentioned medical record data storage device 101.

[0026] In some embodiments, the above-mentioned multimodal image-based disease condition detection device can obtain the patient's patient historical medical record information set and the current patient's physical condition information from the medical record data storage device 101 through a wired connection or a wireless connection, wherein the historical medical record information in the patient's historical medical record information set includes: historical multimodal disease condition image data, historical disease condition description text information corresponding to the above-mentioned historical multimodal disease condition image data, and the above-mentioned historical multimodal disease condition image data includes at least one of the following: historical patient ultrasound images, historical patient magnetic resonance imaging images, and historical patient computer tomography images. Among them, the above-mentioned disease condition detection main control chip can be a chip for processing the patient's historical medical record information set. For example, the above-mentioned disease condition detection main control chip can be but is not limited to one of the following: a central processing unit, a microprocessor unit. The above-mentioned medical record data storage device can be a memory for storing the patient's physical health status and disease treatment process. The patient's historical medical record information in the above-mentioned patient's historical medical record information set can be text information that is located before the current time and records the patient's disease treatment process. The above-mentioned historical multimodal disease condition image data can be different types of image data that display the patient's disease from different perspectives. The schematic internal test diagram of the above-mentioned historical multimodal disease condition image data is, for example, Figure 4 shown. Figure 4 The upper row of images are ultrasound images of the patient's thyroid nodules, and the lower row are contrast images of the patient's thyroid nodules. Figure 4 The images in each column may be, in order: images of the patient at the current moment, after treatment, one month after treatment, three months after treatment, and 12 months after treatment. Figure 4 The area indicated by the middle arrow may be the area where the thyroid nodules are located. The above-mentioned historical medical condition description text information may be information showing the patient's disease condition in the form of text. The above-mentioned historical patient nuclear magnetic resonance imaging image may be an imaging image obtained by scanning the patient's body through a nuclear magnetic resonance device. The above-mentioned historical patient computer tomography image may be an imaging image obtained by scanning the patient's body using optical coherence tomography technology before the current time. For example, the above-mentioned historical multimodal medical condition image data may be multimodal image data obtained by scanning and imaging the patient's skin. The above-mentioned current patient physical state information may be information representing the current state of the patient's body.

[0027] The data desensitizing device 102 is used to perform sensitive data desensitization processing on the patient's historical medical record information set to obtain a desensitized patient's historical medical record information set.

[0028] In some embodiments, the data desensitization device 102 is used to perform sensitive data desensitization processing on the patient historical medical record information set to obtain a desensitized patient historical medical record information set. The desensitized patient historical medical record information in the desensitized patient historical medical record information set may be historical medical record information obtained by replacing sensitive data in the patient historical medical record information with insensitive data.

[0029] Optionally, the data desensitization device is further configured to perform sensitive data desensitization processing on the patient historical medical record information set through the following steps to obtain a desensitized patient historical medical record information set:

[0030] The first step is to divide the patient's historical medical record information set into text sentences to obtain a historical medical record sentence set. The historical medical record sentences in the historical medical record sentence set can be sentences obtained by dividing the patient's historical medical record information with period punctuation marks.

[0031] In the second step, for each historical medical record sentence in the above historical medical record sentence set, the following steps of identifying the disease negation words are performed:

[0032] Sub-step 1, segment the above historical medical record sentences to obtain historical medical record segmentation groups. The historical medical record segmentation in the above historical medical record segmentation groups can be words related to the patient's medical record obtained by segmenting the above historical medical record sentences using the segmentation method based on string matching in the Jieba segmentation algorithm.

[0033] Sub-step 2, performing word matching processing on the above-mentioned historical medical record word groups and the preset standard medical word library to obtain a word matching result group. The above-mentioned preset standard medical word library can be an existing medical term library related to medical care. For example, the above-mentioned preset standard medical word library can be the ICD10 library in the super thesaurus included in the UMLS (Unified Medical Language System) and a medical term set with a specific semantic type. For example, the above-mentioned specific semantics can include but are not limited to at least one of the following: findings (disease symptoms, signs), diseases or comprehensive symptoms, mental or behavioral disorders.

[0034] Sub-step 3, constructing an index for at least one historical medical record segmentation corresponding to at least one word matching result in the word matching result group that represents a successful word matching, to obtain a condition index segmentation group. The condition index segmentation in the condition index segmentation group can be an index segmentation obtained by using a unique string identifier (SUI) in UMLS to establish a string index for the historical medical record segmentation of the word matching project.

[0035] Sub-step 4, performing negative word recognition on the above-mentioned historical medical record word groups to obtain a condition negation word group. Among them, the condition negation words in the above-mentioned condition negation word group can be words with negative meanings in semantics. In practice, the above-mentioned data desensitization device can perform word matching processing on the above-mentioned historical medical record word groups and the preset negative word library to obtain a word matching result group set. Then, at least one historical medical record word that successfully matches the corresponding representation word in the historical medical record word group is determined as the condition negation word group.

[0036] It should be noted that the disease symptom description text information with negative semantics in the patient's historical medical record information set is not the patient's disease, and there is no risk of exposing privacy. Therefore, identifying and removing disease symptom text with negative semantics can reduce the amount of sensitive data identification, improve the accuracy and efficiency of sensitive data identification, and reduce the time for sensitive data identification.

[0037] Sub-step 5, determining the negative word coverage of each condition negation word in the condition negation word group, and obtaining the negative word coverage group. The negative word coverage in the negative word coverage group can represent the semantic scope of the negation of the condition negation word. For example, the historical medical record sentence can be "He has never had a toothache." The condition negation word can be "no." The negative word coverage can be "tooth."

[0038] Sub-step 6, in response to determining that there is at least one condition index participle in the above-mentioned negative word coverage group, determining the above-mentioned at least one condition index participle as a negative historical medical record participle group.

[0039] Sub-step 7, removing the above-mentioned negative historical medical record segment phrases from the above-mentioned historical medical record segment phrases, and obtaining the removed historical medical record segment phrases as the sensitive disease segment phrases to be identified.

[0040] The third step is to identify the associated sensitive words on the obtained sensitive disease segmentation word set to be identified, and obtain a disease sensitive segmentation word set. The disease sensitive segmentation words in the above disease sensitive segmentation word set can represent the disease state.

[0041] As an example, the data desensitization device may first determine the semantic similarity value between each sensitive disease segmentation to be identified in the sensitive disease segmentation group set to be identified and each disease sensitive word included in the preset disease sensitive word library as the first semantic similarity, and obtain the first semantic similarity group set. Among them, the preset disease sensitive word library may be a SNOMED-CT (Systematized Nomenclature of Medicine-Clinical Terms) term library. Secondly, at least one sensitive disease segmentation to be identified corresponding to the first semantic similarity greater than or equal to the similarity threshold is selected from the sensitive disease segmentation group set to be identified as the first sensitive disease segmentation set. Among them, the similarity threshold may be a threshold obtained by the K-means clustering algorithm. Then, the first sensitive disease segmentation set is removed from the sensitive disease segmentation set to be identified to obtain the first removed sensitive disease segmentation set. Again, the semantic similarity between the first removed sensitive disease segmentation set and the first sensitive disease segmentation set is determined as the second semantic similarity, and the second semantic similarity group set is obtained. Subsequently, at least one removed sensitive disease segmentation corresponding to the second semantic similarity greater than or equal to the similarity threshold is selected from the above-mentioned removed sensitive disease segmentation set as the second sensitive disease segmentation set. After that, the above-mentioned second sensitive disease segmentation set is removed from the above-mentioned first removed sensitive disease segmentation set to obtain the second removed sensitive disease segmentation set. Then, the above-mentioned second removed sensitive disease segmentation set is subjected to segmentation combination processing to obtain the disease segmentation set to be identified. Among them, the above-mentioned segmentation combination processing includes the combination of two or three second removed sensitive disease segmentations. Then, the semantic similarity between the above-mentioned disease segmentation set to be identified and the above-mentioned first sensitive disease segmentation set is determined as the third semantic similarity to obtain the third semantic similarity group set. Finally, at least one disease segmentation to be identified corresponding to the third semantic similarity greater than or equal to the above-mentioned semantic similarity threshold is selected from the above-mentioned disease segmentation set to be identified as the third sensitive disease segmentation set. The above-mentioned first sensitive disease segmentation set, the above-mentioned second sensitive disease segmentation set and the above-mentioned third sensitive disease segmentation set are determined as disease sensitive segmentation sets. It should be noted that the recognition of the first sensitive disease condition word set can accurately identify disease condition sensitive words that do not have a regular structure. The recognition of the second sensitive disease condition word set can solve sensitive words that have strong semantic relevance to disease condition sensitive words, so as to prevent the inference of disease condition sensitive words through semantically related non-sensitive data that has not been desensitized. The third sensitive disease condition word set can avoid the inference of disease condition sensitive words due to combined terms to a certain extent. Thereby improving the recognition accuracy of disease condition sensitive words and the data security of the patient's historical medical record information set.

[0042] The fourth step is to perform semantic replacement on the above-mentioned disease-sensitive word set to obtain a replaced disease word set as the desensitized patient history information set. The above-mentioned semantic replacement can be a semantic replacement with non-sensitive words in the SNOMED-CT term library.

[0043] Optionally, the data desensitization device is further configured to determine the disease condition negation coverage of each disease condition negation word in the disease condition negation word group through the following steps to obtain a disease condition negation coverage group:

[0044] In the first step, for each condition negation word in the above condition negation word group, the following negation coverage determination steps are performed:

[0045] Sub-step 1, in response to determining that the above-mentioned condition negation word is a post-positioned condition negation word, the above-mentioned condition negation word and the preset negation structure are matched for sentence structure to obtain a sentence structure matching result. Among them, the above-mentioned post-positioned condition negation word can be a condition negation word that is located after the condition index participle in a historical medical record sentence and belongs to a preset post-positioned condition negation word library. The above-mentioned preset negation structure can be a structure in a historical medical record sentence, in which the negation structure is preceded by the condition index participle and the condition negation word is at the end. The above-mentioned preset post-positioned condition negation word library can be a pre-set word library formed by negation words with post-positioned negation semantics in the sentence. For example, the above-mentioned preset post-positioned condition negation word library can include but is not limited to at least one of the following: negative, unlikely.

[0046] Sub-step 2, in response to determining that the sentence structure matching result indicates that the sentence structure matching is successful, a preset number of historical medical record segmentation words before the above-mentioned condition negation word are determined as the post-negation word coverage range, as the negation word coverage range. The preset number can be the number of words included in the pre-set coverage range of the negation word. For example, the preset number can be 6.

[0047] Sub-step 3, in response to determining that the above-mentioned condition negation word is a pre-condition negation word, and there is no preset condition negation termination word in the preset condition negation termination word set in the above-mentioned historical medical record sentence, at least one historical medical record participle included between the above-mentioned condition negation word and the sentence termination word is determined as the first pre-condition negation word coverage range, as the negation word coverage range, wherein the above-mentioned sentence termination word is the historical medical record participle located at the termination position in the above-mentioned historical medical record sentence. Among them, the above-mentioned pre-condition negation word can be a condition negation word that is located before the condition index participle in a historical medical record sentence and belongs to the preset pre-condition negation word library. The above-mentioned preset pre-condition negation word library can be a word library formed by negation words with pre-condition negation semantics in the sentence. For example, the above-mentioned preset pre-condition negation word library can include but is not limited to at least one of the following: suspect, no, no, no indication, excluded. The above-mentioned preset condition negation termination word set can be a pre-set negation transition word with transition semantics. For example, the above-mentioned preset condition negation termination word set can include but is not limited to at least one of the following: except, involving, although, still, however.

[0048] Sub-step 4, in response to determining that the above-mentioned condition negation word is a preceding condition negation word, and any preset condition negation end word in the above-mentioned preset condition negation end word set exists in the above-mentioned historical medical record sentence, at least one historical medical record participle included between the above-mentioned condition negation word and the above-mentioned preset condition negation end word is determined as the second preceding condition negation word coverage range, as the negation word coverage range.

[0049] The condition detection main control chip 103 is used to input the above-mentioned current patient physical state information, the above-mentioned desensitized patient historical medical record information set including the desensitized historical multimodal condition image data set and the desensitized historical condition description text information set into the condition detection model to obtain the current condition information.

[0050] In some embodiments, the above-mentioned condition detection main control chip 103 is used to input the above-mentioned current patient physical state information, the above-mentioned desensitized historical multimodal condition image data set and the desensitized historical condition description text information set included in the above-mentioned desensitized patient historical medical record information set into the condition detection model to obtain the current condition information. Among them, the above-mentioned current condition information can be information after detection that characterizes the patient's current physical condition or the possibility of suffering from the disease. The above-mentioned condition detection model can be a deep neural network model that performs feature extraction, fusion and disease diagnosis on the input current patient physical state information, desensitized historical multimodal condition image data set, and desensitized historical condition description text information set, and outputs the current condition information, that is, the patient's diagnosis result.

[0051] The above-mentioned disease detection model may include: residual network, deep separable convolution layer, channel attention mechanism, manifold pyramid cross-modal feature fusion network, bidirectional attention cross-modal fusion mechanism and multi-classifier integration model. The above-mentioned residual network may be a network used to extract the symptom feature information, text feature information and image feature information of the above-mentioned current patient physical state information, the above-mentioned desensitized historical disease description text information set and the above-mentioned desensitized historical multimodal disease image data set. The above-mentioned deep separable convolution layer may be a network layer that fuses the features of symptom feature information, text feature information and image feature information at the cross-modal level, outputs the features after the first fusion, and alleviates the parameter amount to a certain extent. The above-mentioned channel attention mechanism may be a network that fuses the features after the first fusion in the channel dimension in a top-down manner and outputs the channel fusion feature. The above-mentioned manifold pyramid cross-modal feature fusion network may be a network that extracts the geometric features of the channel fusion features output by the channel attention mechanism through the manifold learning algorithm and gradually fuses the features of different modalities from the bottom to the top using a pyramid structure, and outputs the geometric fusion feature. The above-mentioned bidirectional attention cross-modal fusion mechanism can be a network that further fuses geometric fusion features using a top-down and bottom-up combined attention mechanism. The above-mentioned multi-classifier integration model can be a model that integrates Bagging classifiers, Adaboost (Adaptive Boosting) classifiers and SVM (Support Vector Machine) classifiers to perform disease diagnosis classification on the fused features output by the input bidirectional attention cross-modal fusion mechanism.

[0052] The condition detection main control chip 103 is used to extract the cause-effect relationship from the above-mentioned desensitized patient historical medical record information set to obtain a time-series cause-effect diagram of the condition.

[0053] In some embodiments, the above-mentioned condition detection main control chip 103 is used to extract the causal relationship of the patient's historical medical record information set after desensitization to obtain a causal graph of the condition time series. Among them, the above-mentioned condition time series causal graph can be a directed acyclic graph in the form of a graph, which represents the causal relationship between the development, transformation and interaction between the disease suffered by the above-mentioned patient and the cause of the disease. The above-mentioned condition time series causal graph can help medical workers and patients understand the nature of the disease and identify the cause of the disease. The above-mentioned condition time series causal graph can be composed of a set of nodes corresponding to the disease and the cause of the disease through directed arrows.

[0054] In the process of adopting technical solutions to solve the above technical problem 1, the following technical problem 2 is often accompanied: since there is a large amount of disease-affecting characteristic information that is irrelevant to the diagnosis of the patient's disease and redundant in the patient's historical medical record information set after desensitization, the prediction accuracy is low when predicting the patient's disease development, and it is impossible to accurately predict the patient's condition in a timely and effective manner, reducing the patient's experience and physical health. For the above technical problem 2, the conventional solution is generally: using the krill swarm algorithm to screen the disease impact indicator set included in the patient's historical medical record information set after desensitization, and obtain the screened indicator set. Then, construct an indicator causal graph for the screened indicator set. However, the above conventional solution still has the following problems: since the patient's historical medical record information after desensitization belongs to high-dimensional data, and there are a large number of irrelevant and repeated disease-affecting indicators, the search range and difficulty of the krill swarm algorithm are increased, and the efficiency of indicator screening is reduced, resulting in an extension of the time to generate the disease time series causal graph, reducing the accuracy of the disease time series causal graph and the timeliness of disease prediction. The inventors have considered the shortcomings of the above conventional solutions and combined with the advantages / technical status of the applicant and the entrusted cooperative enterprises in the generation of medical causal images in this field, and decided to adopt the following solution:

[0055] In some optional implementations of some embodiments, the above-mentioned condition detection main control chip is further configured to extract the causal relationship of the above-mentioned desensitized patient historical medical record information set through the following steps to obtain a condition time series causal graph:

[0056] The first step is to extract the disease impact indicator set of the disease type to which the current condition information belongs from the above-mentioned desensitized sensitive historical medical record information set. Among them, the disease impact indicators in the above-mentioned disease impact indicator set may be abnormal physical symptoms that cause the patient to suffer from the disease type to which the above-mentioned current condition information belongs. For example, the above-mentioned disease type may be lung cancer, then the above-mentioned disease impact indicator set may include but is not limited to at least one of the following: gender, age, nodule size, nodule stage, thyroid hormone, treatment method. The above-mentioned disease impact indicator set may include impact indicators related to the disease type required to cause the patient to suffer from the current condition, as well as irrelevant impact indicators.

[0057] The second step is to perform feature selection processing on the above disease impact indicator set to obtain a selected disease impact indicator set. In practice, the above disease detection main control chip can use the mRMR (Max-Relevance and Min-Redundancy) algorithm to perform feature selection processing on the above disease impact indicator set to obtain a selected disease impact indicator set.

[0058] The third step is to generate an initial indicator population according to the above-mentioned set of disease impact indicators after selection. Among them, the initial indicator individuals in the above-mentioned initial indicator population can be the various disease impact indicators after selection that directly cause the disease suffered by the patient, and the individuals with the smallest number of disease impact indicators after selection included in each disease impact indicator after selection. The initial indicator individuals in the above-mentioned initial indicator population include: initial position and individual upper and lower bounds, step scaling range of step scaling factor, maximum induced movement speed, maximum foraging movement speed, and maximum random diffusion movement speed. The above-mentioned initial position can represent a candidate solution for the disease impact indicator required to construct a disease time series causal graph. The above-mentioned compensation scaling factor can represent the ability to adjust the initial indicator individual in the mobile search process. The above-mentioned maximum induced movement speed can be the maximum movement speed that the initial indicator individual can reach when it is affected by other initial indicator individuals, and can affect the convergence speed and search efficiency of the initial indicator population, and improve the convergence speed. The above-mentioned maximum foraging movement speed can be the maximum movement speed that the above-mentioned initial indicator individual can reach in the process of exploring the global optimal solution, and can represent the sensitivity and exploration ability of the initial individual to the global optimal solution. The above maximum random diffusion moving speed can characterize the random moving range of the initial indicator individual in the search space, i.e., the upper and lower bounds of the individual, which can affect the diversity and exploration ability of the initial indicator individual and prevent the algorithm from falling into a local optimal solution.

[0059] As an example, the disease condition detection main control chip can initialize the selected disease impact indicator set to obtain an initial indicator population.

[0060] The fourth step is to discretize the initial indicator population to obtain a discretized initial indicator population. The discretized initial indicator individuals in the discretized initial population can be represented in binary form as to whether the disease impact indicator of the initial indicator individual is selected after each selection. 1 means selected, and 0 means not selected.

[0061] The fifth step is to generate an indicator screening fitness function for the above discretized initial indicator population. Among them, the above indicator screening fitness function can characterize the quality of each selected disease impact indicator corresponding to the discretized initial indicator individual. The above indicator screening fitness function can be a weighted sum of maximizing the classification accuracy and the selection rate of the selected disease impact indicator set. The weight of the above classification accuracy is 0.9, and the weight of the selection rate is 0.1. The above classification accuracy can characterize the accuracy of whether the disease type corresponding to the current condition information is determined based on the selected disease impact indicator set corresponding to the discretized indicator individual. The above selection rate can be the ratio of 1 to the number and total number of selected disease impact indicators included in the discretized indicator individual.

[0062] Step 6: Based on the discretized initial indicator population, perform the following screening steps:

[0063] Sub-step 1: for each discretized initial indicator individual in the discretized initial indicator population, perform the following input steps:

[0064] In the first sub-step, the discretized initial index population is input into the above index screening fitness function to obtain an initial screening fitness value set. The initial screening fitness value in the above initial screening fitness value set can characterize the quality of the discretized initial individuals. The larger the above initial screening fitness value, the higher the quality of the discretized initial individuals.

[0065] The second sub-step is to determine the initial adjacent indicator population and the initial target indicator individual of the discretized initial indicator individual, wherein the initial adjacent indicator individual in the initial adjacent indicator population is an individual whose distance to the initial indicator individual is less than or equal to a preset distance threshold, and the initial target indicator individual is an individual corresponding to the initial screening fitness value with the largest value in the initial screening fitness value set. The preset distance threshold may be the maximum value of a pre-set distance.

[0066] The third sub-step is to determine, based on the initial screening fitness value set, a position difference ratio set and a fitness difference ratio set between each initial neighboring indicator individual in the initial neighboring indicator population and the discretized initial indicator individual.

[0067] As an example, the above-mentioned disease detection main control chip can perform the following determination steps for each initial adjacent indicator individual and the initial indicator individual after discretization in the initial adjacent indicator population: First, determine the position difference between the individual position corresponding to the initial adjacent indicator individual and the individual position corresponding to the initial indicator individual after discretization. Secondly, determine the above-mentioned position difference and the ratio of the sum of the position difference and the preset positive minimum value. The above-mentioned preset integer minimum value can be a minimum value set to avoid the denominator being 0, which is not limited here. Subsequently, determine the maximum and minimum values ​​in the above-mentioned initial screening fitness value set as the maximum fitness value and the minimum fitness value. Then, determine the first fitness difference between the maximum fitness value and the minimum fitness value, and the initial screening fitness value corresponding to the initial adjacent indicator individual and the initial screening fitness value corresponding to the discretized initial indicator individual. Finally, determine the ratio of the above-mentioned second fitness difference to the above-mentioned first fitness difference as the fitness difference ratio.

[0068] In the fourth sub-step, each position difference ratio in the position difference ratio set is multiplied, accumulated and added with the corresponding fitness difference ratio in the fitness difference ratio set to obtain an adjacency induction coefficient.

[0069] The fifth sub-step is to determine the target position difference ratio and the target fitness difference ratio of the target position of the initial target indicator individual and the discretized initial indicator individual according to the initial screening fitness value set.

[0070] As an example, the above-mentioned disease detection main control chip can first determine the target position difference between the individual position corresponding to the initial target indicator individual and the individual position corresponding to the above-mentioned discretized initial indicator individual. Secondly, determine the ratio of the above-mentioned target position difference to the sum of the target position difference and the preset positive minimum value. Then, determine the target fitness difference between the initial screening fitness value corresponding to the initial target indicator individual and the initial screening fitness value corresponding to the discretized initial indicator individual. Finally, determine the ratio of the above-mentioned target fitness difference to the above-mentioned first fitness difference as the target fitness difference ratio.

[0071] The sixth sub-step is to determine the target induction coefficient according to the target position difference and the target fitness difference.

[0072] As an example, the above-mentioned disease detection main control chip can first determine the sum of the random value and the ratio of the number of times the screening step has been executed and the preset number of execution times as the first execution value. The random value can be a value between 0 and 1 generated by a random generator. Then, the product of the above-mentioned execution value and 2 is determined as the second execution value. Finally, the product of the above-mentioned second execution value, the above-mentioned target position difference and the above-mentioned target fitness difference is determined as the target induction coefficient.

[0073] The seventh sub-step is to determine the current induced moving speed of the discretized initial indicator individual according to the target induction coefficient, the adjacent induction coefficient and the maximum induced moving speed. The current induced moving speed may be the induced moving speed at the current moment after the discretized initial indicator individual is updated.

[0074] As an example, the above-mentioned disease detection main control chip can first determine the sum of the target induction coefficient and the adjacent induction coefficient as the induction coefficient. Secondly, determine the sum of the product of the above-mentioned maximum induced movement speed and the above-mentioned induction coefficient, and the product of the preset inertia induction coefficient and the historical induced movement speed corresponding to the previous screening step of the initial individual after discretization as the current induced movement speed. Among them, the above-mentioned preset inertia induction coefficient can be a preset value between 0 and 1.

[0075] The eighth sub-step is to determine the current foraging movement speed of the discretized initial indicator individual according to the initial screening fitness value set and the maximum foraging movement speed. The current foraging movement speed may be the foraging movement speed of the discretized initial individual at the current moment after update.

[0076] As an example, the above-mentioned disease detection main control chip can first determine the group food position corresponding to the current screening step. Among them, the above-mentioned individual food position is obtained by the following steps: determine the fitness ratio of 1 and the initial screening fitness value of the initial individual after discretization. Determine the ratio of the cumulative sum of the initial indicator population after discretization of the above-mentioned fitness ratio and the cumulative sum of the product of the above-mentioned fitness ratio and the individual position as the individual food position. Secondly, determine the food position difference between the above-mentioned group food position and the individual position corresponding to the above-mentioned initial indicator individual after discretization. Again, determine the ratio of the above-mentioned food position difference, the food position difference and the sum of the preset positive minimum value, as the food position difference ratio. Subsequently, determine the food fitness difference between the initial screening fitness value corresponding to the group food position and the initial screening fitness value corresponding to the initial indicator individual after discretization. Afterwards, determine the ratio of the above-mentioned food fitness difference and the above-mentioned first fitness difference as the food fitness difference ratio. Then, determine the product of the above-mentioned first execution value, the above-mentioned food fitness difference ratio and the above-mentioned food position difference ratio as the food foraging coefficient. Then, the sum of the food foraging coefficient, the product of the target position difference and the target fitness difference is determined as the target foraging coefficient. Finally, the sum of the maximum foraging movement speed, the product of the sum of the target foraging coefficient and the food foraging coefficient, the product of the preset inertial foraging coefficient and the historical foraging movement speed corresponding to the previous screening step of the initial individual after discretization is determined as the current foraging movement speed. Among them, the preset inertial foraging coefficient can be a preset value between 0 and 1.

[0077] The ninth sub-step is to determine the number of times the above input step has been executed.

[0078] The tenth sub-step is to determine the current random diffusion movement speed of the discretized initial indicator individual according to the number of executions, the preset execution number threshold and the maximum random diffusion movement speed. The current random diffusion movement speed may be the random diffusion movement speed at the current moment after the discretized initial indicator individual is updated.

[0079] As an example, the disease detection main control chip can first determine the difference between 1 and the ratio of the number of executions and the preset execution coefficient threshold as the execution difference. Then, the product of the maximum foraging movement speed, the execution difference and the random direction vector is determined as the current random diffusion movement speed. The random direction vector can be a random value between -1 and 1.

[0080] The eleventh sub-step is to determine the current step scaling factor of the initial indicator individual in terms of the number of times it has been executed, based on the above step scaling range.

[0081] As an example, the above-mentioned disease detection main control chip can first determine the difference between the maximum value and the minimum value included in the above-mentioned step scaling range as the step scaling difference. Secondly, determine an exponential function with e as the base and the inverse of the product of the preset adjustment coefficient and the square of the ratio of the number of executions to the preset execution number threshold as the scaling value. Among them, the above-mentioned preset adjustment coefficient can be a preset value between 0 and 1. Finally, determine the product of the step scaling difference and the above-mentioned scaling value and the sum of the minimum value included in the compensation scaling range as the current step scaling factor.

[0082] In the twelfth sub-step, the discretized initial indicator individuals are individually updated according to the induced movement speed, the foraging movement speed, the random diffusion movement speed and the current step size scaling factor to obtain the updated indicator individuals.

[0083] As an example, the disease detection main control chip can first determine the product of the current step size scaling factor and the difference between the upper and lower bounds of the individual value, and determine it as the updated compensation value. Then, the sum of the induced movement speed, foraging movement speed, and random diffusion movement speed, and the product of the updated compensation value are determined as the updated value. Finally, the sum of the updated value and the historical position of the initial indicator individual after discretization corresponding to the previous screening step is used as the more refined indicator individual.

[0084] The thirteenth sub-step is to perform crossover mutation processing on the updated indicator individuals to obtain mutated indicator individuals. The mutated indicator individuals may be individuals obtained by first performing a crossover operation on the updated indicator individuals and then performing a mutation operation on them.

[0085] As an example, the above-mentioned disease detection main control chip can first determine 0.2 and the maximum value in the above-mentioned initial screening fitness value set as the crossover probability value of the updated indicator individual. Secondly, a uniformly distributed random number between 0 and 1 is randomly generated for each dimension in the updated individual position of the updated indicator individual. Again, in response to determining that the random number is less than the above-mentioned crossover probability value, a discretized initial indicator individual and an updated indicator individual are randomly selected from the discretized initial indicator population for crossover to obtain a crossover indicator individual. Subsequently, 0.05 and the maximum value in the above-mentioned initial screening fitness value set are determined as the mutation probability value of the updated indicator individual. Afterwards, in response to determining that the above-mentioned mutation probability value is greater than the above-mentioned random number, the position difference of two discretized initial individuals randomly selected from the above-mentioned discretized initial indicator population, the product of the preset mutation value, and the sum of the crossover indicator individual are determined as the mutated indicator individual. Among them, the above-mentioned preset mutation value can be a random number between 0 and 1 set in advance.

[0086] In the fourteenth sub-step, the mutated indicator individual is input into the above indicator screening fitness function to obtain the updated fitness value.

[0087] Sub-step 2, screening the initial screening fitness value set and the updated fitness value set corresponding to the initial indicator population and the obtained mutated indicator population, to obtain the screened indicator population and the target mutated individual. Among them, the screened indicator individual in the above-mentioned screened indicator population can be the individual corresponding to the larger fitness value screened from the initial screening fitness value and the corresponding updated fitness value. The above-mentioned target mutated individual can be the individual with the largest fitness value corresponding to the above-mentioned screened indicator population.

[0088] Sub-step 3, in response to determining that the number of executions is greater than or equal to the preset execution number threshold, determining each selected disease impact indicator included in the target mutated individual as a target disease impact indicator set. The preset execution number may be a preset maximum execution number of the screening step. For example, the preset execution number threshold may be 100 times.

[0089] In the seventh step, in response to determining that the number of executions is less than the above target execution threshold, the filtered indicator population is determined as the initial indicator population after discretization, and the sum of the number of executions and the preset value is determined as the number of executions, so as to perform the above screening step again.

[0090] Step 8: Determine the causal direction of the indicators between the target disease impact indicators included in the target disease impact indicator set. The causal direction of the indicator can characterize the occurrence and development process of the disease or the causal relationship. The above determination can be made by the IGCI (Information-theoretic Graph-based Causal Inference, causal inference based on information entropy) algorithm.

[0091] The ninth step is to construct an indicator causal diagram for the above target disease impact indicator set according to the causal direction of the above indicators to obtain a time series causal diagram of the disease condition.

[0092] As an example, the execution subject may causally connect each target disease impact indicator included in the target disease impact indicator set through the indicator causal direction to obtain a disease condition time series causal diagram.

[0093] The above technical solution and its related contents, combined with the disease prediction model, as an inventive point of the embodiment of the present disclosure, solve the second technical problem mentioned in the background technology: "Since the patient's historical medical record information after desensitization belongs to high-dimensional data, and there are a large number of irrelevant and repeated disease influencing indicators, the search range and difficulty of the krill swarm algorithm are increased, and the efficiency of indicator screening is reduced, resulting in the extension of the time to generate the disease time series causal graph, and the reduction of the accuracy of the disease time series causal graph and the timeliness of disease prediction." The factors that lead to the extension of the time to generate the disease time series causal graph and the reduction of the accuracy of the disease time series causal graph and the timeliness of disease prediction are often as follows: Since the patient's historical medical record information after desensitization belongs to high-dimensional data, and there are a large number of irrelevant and repeated disease influencing indicators, the search range and difficulty of the krill swarm algorithm are increased, and the efficiency of indicator screening is reduced. If the above factors are solved, the effect of shortening the time to generate the disease time series causal graph and improving the accuracy of the disease time series causal graph and the timeliness of disease prediction can be achieved. In order to achieve this effect, the present invention first extracts the disease impact indicator set from the desensitized sensitive historical medical record information set and then performs the first feature selection. Through the correlation and redundancy between the indicators, the search range and search quantity of the subsequent indicator screening can be narrowed. Secondly, the initial indicator population is generated and discretized, which is more conducive to the subsequent screening of strongly correlated indicators in binary form. Subsequently, each discretized initial indicator individual in the discretized initial indicator population is iterated cyclically. In the first step, the current induced movement speed, the current foraging movement speed, the current random diffusion movement speed are updated, and the nonlinear decreasing adjustment of the current step length scaling factor of the number of executions can be made to expand the scope of the early search for a wide search, shorten the scope of the later search for a refined search, and search locally and globally to improve the search and avoid falling into the local optimal value to a certain extent. In the second step, the updated indicator individuals are cross-mutated, which can improve the diversity and global search ability of the individuals and increase the probability of approaching the global optimal solution. Finally, the causal direction is identified and the causal graph is constructed for the target disease influencing indicator set to obtain a time-series causal graph of the disease. This can improve the accuracy of causal graph generation and shorten the generation time, as well as improve the accuracy and timeliness of the disease development prediction.

[0094] The condition detection main control chip 103 is used to input the above-mentioned condition time series cause-effect diagram, the above-mentioned patient historical medical record information set after desensitization and the above-mentioned current condition information into the condition prediction model to obtain the condition development prediction information within the prediction time period.

[0095] In some embodiments, the above-mentioned condition detection main control chip 103 is used to input the above-mentioned condition time series cause and effect diagram, the above-mentioned patient historical medical record information set after desensitization, and the above-mentioned current condition information into the condition prediction model to obtain the prediction information of the condition development within the prediction time period. Among them, the above-mentioned condition development prediction information can be the information of the predicted change of the patient's condition in the future period of time. The above-mentioned condition prediction model can be a model that predicts the condition of the input condition time series cause and effect diagram, the patient's historical medical record information set after desensitization, and the current condition information, and outputs the prediction information of the patient's condition development in the future time period. Figure 5 Information on the prediction of disease progression in patients with stage II benign thyroid nodules is shown. Figure 5 The medium blue part indicates that the thyroid nodule is soft. The hardness of the thyroid nodule increases from blue to red. Figure 5 This is shown in the vertical color map in the lower left image. Figure 5 The upper row may be an ultrasound image of a patient's thyroid nodule, and the lower row may be an elastic imaging image of a patient's thyroid nodule. Figure 5 The leftmost image column may be an ultrasound image and an elastic imaging image representing the condition information of the patient with benign lesions of stage II thyroid nodules at the current moment. Figure 5 The second column may be an ultrasonic image illustrating internal testing during the prediction period, which indicates that the patient's thyroid nodules have low hardness, necrotic cells of the thyroid nodules are absorbed quickly by the body, and the patient is suitable for ablation surgery. Figure 5 The third column may be ultrasound images and elastic imaging images of the patient after receiving treatment surgery. Figure 5 The fourth, fifth and sixth columns may be image columns showing the reduction of thyroid nodules after 1 month, 7 months and 13 months of the patient, respectively, which are consistent with the predicted results. Figure 6 Information on the predicted disease progression for patients with stage II thyroid nodules and hard, benign lesions is displayed. Figure 6 The first column may be an ultrasound image and an elastic imaging image representing the patient's condition information of stage II thyroid nodule benign lesions and hard nodules at the current moment. Figure 6 The second column may be an ultrasonic image illustrating internal testing during the prediction period that indicates that the patient's thyroid nodules have a higher hardness, necrotic cells of the thyroid nodules are absorbed slowly by the body, and ablation surgery is not the most suitable treatment option. Figure 6 The third column may be ultrasound images and elastic imaging images of the patient after receiving treatment surgery. Figure 6 The fourth, fifth and sixth columns may be image columns showing that the thyroid nodules of the patients have not changed much and are consistent with the predicted results after 1 month, 12 months and 13 months respectively.

[0096] The above-mentioned disease prediction model may include: a feature extraction network, a sequence variational autoencoder, a decoding network and a classifier. The above-mentioned feature extraction network may be a network that performs text and image encoding on the input disease time series causal graph, the patient's historical medical record information set after desensitization and the current disease information. The above-mentioned feature extraction network may be a convolutional neural network. The above-mentioned variational autoencoder may be a network that encodes the feature vector output by the input feature extraction network through a gated recurrent unit and two fully connected layers to obtain a time series hidden vector. The above-mentioned decoding network may be a network that reconstructs the time series hidden vector to obtain a reconstructed time series vector. The above-mentioned classifier may be a network that predicts and classifies the input reconstructed time series vector to obtain disease progression prediction information.

[0097] The above-mentioned disease condition detection main control chip 103 is used to evaluate the disease condition level of the above-mentioned disease condition development prediction information to obtain disease condition level information.

[0098] In some embodiments, the above-mentioned condition detection main control chip 103 is used to evaluate the condition level of the above-mentioned condition development prediction information to obtain the condition level information. Among them, the above-mentioned condition level information can characterize the severity of the patient's condition. In practice, the above-mentioned condition detection main control chip can first quantify the various condition index data included in the above-mentioned condition development prediction information to obtain a set of quantitative values ​​of the condition index. Among them, the above-mentioned condition index data can be indicator data used to measure the severity of the patient's illness. The above-mentioned condition index data can include but is not limited to at least one of the following: heart rate, respiratory rate, systolic blood pressure, body temperature. Then, the above-mentioned condition index quantization value set is normalized to obtain a normalized condition index quantization value set. After that, the above-mentioned normalized condition index quantization value set is weighted summed to obtain a comprehensive condition value. Finally, by querying the preset condition severity level table, the condition level information corresponding to the above-mentioned comprehensive condition value is determined. Among them, the above-mentioned preset condition severity level table can be a data table of different severity levels corresponding to different values ​​set in advance.

[0099] The above-mentioned disease warning device is used to send disease warning information of different levels according to the above-mentioned disease level information.

[0100] In some embodiments, the above-mentioned condition warning device is used to send different levels of condition warning information according to the above-mentioned condition level information. Among them, the above-mentioned condition warning device can be a server for associating with the patient's communication device to send the condition warning information to the patient's communication device. The above-mentioned different levels of condition warning information can be different levels of condition warning information distinguished by different colors. For example, green represents a first-level warning, that is, the lowest level warning, red represents a second-level warning, orange represents a third-level warning and red represents a fourth-level warning, that is, the highest level warning.

[0101] Next, combine Figure 2 and Figure 1 The above-mentioned disease detection device based on multimodal images is further explained. Figure 2 Schematic diagram of the structure of some other embodiments of the disease detection device based on multimodal images according to the present disclosure. Figure 2 As shown, the above-mentioned disease detection device also includes: personalized disease treatment auxiliary equipment 105, emotion recognition equipment 106, detection value recognition equipment 107, and target terminal 108.

[0102] In some embodiments, the personalized disease condition treatment auxiliary device 105 is used to filter out the disease treatment information corresponding to the above-mentioned current patient's physical state information from the disease detection information set included in the disease condition knowledge graph corresponding to the above-mentioned current disease condition information, and obtain the target disease treatment information set. Among them, the above-mentioned personalized disease condition treatment auxiliary device can be a server for determining the treatment method for treating the patient's disease. The above-mentioned disease condition knowledge graph can be a knowledge graph corresponding to the disease of the same type as the patient's current disease. The disease detection information in the above-mentioned disease detection information set can be information on the disease treatment plan corresponding to the patient's current disease.

[0103] In some embodiments, the emotion recognition device 106 is used to determine the patient's emotion value for the treatment plan of each target disease treatment information in the target disease treatment information set, and obtain a treatment plan emotion value set. The treatment plan emotion value can characterize the patient's emotional tendency towards the target disease treatment information. In practice, the emotion recognition device can first obtain the patient's comment text information for each target disease treatment information in the target disease treatment information set, and obtain a patient comment text information set. The patient comment text information can be text information of the patient's opinion and evaluation of the target disease treatment information. Then, the LDA (Latent Dirichlet Allocation) topic model is used to perform emotion recognition on the patient comment text information set to obtain a treatment plan emotion information set.

[0104] In some embodiments, the detection value identification device 107 is used to determine the treatment value value of each target disease treatment information in the target disease treatment information set according to the current patient physical state information and the desensitized patient historical medical record information set, and obtain a treatment value value set. The detection value identification device can be a server for evaluating the effectiveness of the treatment of the current disease of the patient treated by the target disease treatment information. The treatment value value can characterize the degree of treatment effect of the target disease treatment information on the patient's current condition information.

[0105] As an example, the detection value recognition device can use the DQN (Deep Q-Networks) model to determine the treatment value of each target disease treatment information in the target disease treatment information set according to the current patient physical state information and the desensitized patient historical medical record information set, and obtain a treatment value value set. Among them, the state space of the DQN model can be the current patient physical state information and the desensitized patient historical medical record information set. The action space of the DQN model can be the target disease treatment information set. The reward mechanism of the DQN model can be to give positive rewards for the relief of the patient's disease symptoms. The better the degree of relief of the disease symptoms, the higher the positive reward, and negative rewards if the disease symptoms are not relieved and worsen or side effects occur.

[0106] In some embodiments, the personalized disease treatment auxiliary device 105 is used to perform weighted summation on each treatment value value in the treatment value value set and the corresponding treatment scheme emotional value in the treatment scheme emotional value set to generate a target value value and obtain a target value value set. The weighted summation may be a weighted summation of the product of the first preset weight and the treatment value value and the product of the second preset weight and the corresponding treatment scheme emotional value. The first preset weight and the second preset weight may be pre-set values ​​less than 1. For example, the first preset weight and the second preset weight are both 0.5.

[0107] In some embodiments, the personalized disease treatment auxiliary device 105 is used to filter out the target disease treatment information with the largest corresponding target value from the target disease treatment information set as the user disease treatment plan, and send it to the target terminal 108 for display.

[0108] Optionally, the disease condition knowledge graph is generated by the personalized disease condition treatment auxiliary device through the following steps:

[0109] The first step is to obtain multi-source heterogeneous disease detection data of the same disease type as the current disease information, wherein the multi-source heterogeneous disease detection data includes: structured disease detection data, semi-structured disease detection data and unstructured disease detection data. The multi-source heterogeneous disease detection data may be data of the same disease type as the patient, from different data sources, and recording the disease treatment process of different patients. The structured disease detection data may be disease detection data in the form of a two-dimensional table structure. The semi-structured disease detection data may be disease detection data stored in Json format. The unstructured disease detection data may be medical record text information or medical images of different patients.

[0110] The second step is to preprocess the multi-source heterogeneous disease detection data to obtain preprocessed multi-source heterogeneous disease detection data. The preprocessing may include but is not limited to at least one of the following: image enhancement processing, text standardization, and word segmentation processing.

[0111] The third step is to extract entity relationships from the preprocessed multi-source heterogeneous disease detection data to obtain a set of disease entity triples. The disease entity triples in the disease entity triple set can be a data combination of entities, relationships, and entity structures that represent the topological relationship between any two entities.

[0112] The fourth step is to input the above disease entity triple set into a preset graph database to obtain an initial disease condition knowledge graph. The above initial disease knowledge graph may be a knowledge graph that only displays the network structure formed by the above disease entity triple set. The above preset graph database may be a Neo4J graph database.

[0113] The fifth step is to perform knowledge reasoning on the initial disease condition knowledge graph to obtain a disease condition knowledge graph. The disease condition knowledge graph may be a knowledge graph obtained by reasoning new entity relationship information from the entity relationship information already in the initial disease condition knowledge graph and adding the new entity relationship information to the initial disease condition knowledge graph.

[0114] Optionally, the personalized disease treatment auxiliary device is further configured to perform entity relationship extraction processing on the pre-processed multi-source heterogeneous disease detection data through the following steps to obtain a set of disease entity triples:

[0115] The first step is to perform word segmentation processing on the multi-source heterogeneous disease detection data after the preprocessing to obtain a disease detection word segmentation set. The disease detection word segmentation in the disease detection word segmentation set can be words related to the disease condition obtained based on the word segmentation processing of Jieba word segmentation.

[0116] The second step is to represent the above disease detection word segmentation set with word vectors to obtain the disease detection word segmentation vector set. Among them, the disease detection word segmentation vectors in the above disease detection word segmentation vector set can represent the vector representation of the semantic information, grammatical information and position information of the disease detection word segmentation. In practice, the above personalized disease treatment auxiliary device can use the GloVe (Global Vectors for Word Representation) model to represent the above disease detection word segmentation set with word vectors to obtain the disease detection word segmentation vector set.

[0117] The third step is to determine the disease detection word part-of-speech information of each disease detection word in the disease detection word set to obtain the disease detection word part-of-speech information set. The disease detection word part-of-speech information in the disease detection word part-of-speech information set can represent the part-of-speech information of the disease detection word.

[0118] The fourth step is to perform an embedded representation on the disease detection word segmentation vector set according to the disease detection word part-of-speech information set to obtain a disease detection word segmentation feature vector set. The disease detection word segmentation feature vector in the disease detection word segmentation feature vector set can represent the semantic information and part-of-speech information of the disease detection word segmentation.

[0119] As an example, the personalized disease treatment auxiliary device can encode the disease detection word segmentation part-of-speech information set to obtain a detection part-of-speech vector set. Then, the detection part-of-speech vector set and the disease detection word segmentation vector set are concatenated to obtain a disease detection word segmentation feature vector set.

[0120] The fifth step is to extract semantic features from the disease detection word segmentation feature vector set to obtain a disease detection word segmentation semantic feature vector set. The disease detection word segmentation semantic feature vector in the disease detection word segmentation semantic feature vector set can represent the semantic sequence information of the disease detection word segmentation feature vector. The semantic feature extraction can be performed by inputting the disease detection word segmentation feature vector set into a bidirectional long short-term memory neural network for semantic feature extraction.

[0121] The sixth step is to perform triple joint extraction processing on the above disease detection word segmentation semantic feature vector set to obtain the disease entity triple set.

[0122] Optionally, the personalized disease treatment auxiliary device is further configured to perform triple joint extraction processing on the disease detection word segmentation semantic feature vector set to obtain a disease entity triple set through the following steps:

[0123] The first step is to perform entity boundary detection on each disease detection word segmentation semantic feature vector in the above disease detection word segmentation semantic feature vector set to obtain a disease detection word segmentation boundary feature vector set. Among them, the disease detection word segmentation boundary feature vector in the above disease detection word segmentation boundary feature vector set can represent the position information of a single Chinese character corresponding to the disease detection word segmentation temporal feature vector in the disease detection word segmentation temporal feature vector. In practice, the above personalized disease treatment auxiliary device can use the BIOES standard algorithm to perform entity boundary detection on each disease detection word segmentation temporal feature vector in the above disease detection word segmentation temporal feature vector set to obtain a disease detection word segmentation boundary feature vector set.

[0124] The second step is to perform a linear transformation on the disease detection word segmentation boundary feature vector set to obtain a disease detection word segmentation linear feature vector set. The disease detection word segmentation linear feature vector in the disease detection word segmentation linear feature vector set can represent the linear information and weight of the disease detection word segmentation boundary feature vector.

[0125] The third step is to perform syntactic analysis on the pre-processed multi-source heterogeneous disease detection data to obtain a disease detection syntactic dependency graph. The disease detection syntactic dependency graph may be a dependency graph that represents the grammatical dependency relationship between the disease detection segmentations included in the disease detection segmentation set in a graph-like manner. The grammatical dependency relationship may include but is not limited to at least one of the following: subject-predicate relationship, verb-object relationship, and adverbial modification relationship.

[0126] The fourth step is to transform the above disease detection syntactic dependency graph to obtain a disease detection syntactic dependency matrix. The above disease detection syntactic dependency matrix can be a matrix that represents the grammatical dependency relationship between disease detection word segments in the form of a matrix.

[0127] The fifth step is to perform graph attention weighted processing on the above-mentioned disease detection word segmentation linear feature vector set and the above-mentioned disease detection syntactic dependency matrix to obtain a sentence sequence feature vector set and a word segmentation dependency feature vector set. Among them, the sentence sequence feature vectors in the above-mentioned sentence sequence feature vector set can represent the sequence information of each sentence included in the multi-source heterogeneous disease detection data after the above-mentioned preprocessing. The word segmentation dependency feature vectors in the above-mentioned word segmentation dependency feature vector set can represent the semantic and positional dependency information between disease detection word segmentations. The above-mentioned graph attention weighted processing can be the attention weighted processing performed by inputting the above-mentioned disease detection word segmentation linear feature vector set and the above-mentioned disease detection syntactic dependency matrix into the graph attention network.

[0128] In the sixth step, the sentence sequence feature vector set and the word segmentation dependency feature vector set are predicted to obtain a disease entity triple set. In practice, the personalized disease treatment auxiliary device can input the sentence sequence feature vector set and the word segmentation dependency feature vector set into a Sigmoid activation function to obtain a disease entity triple set.

[0129] Continue to refer Figure 3 , Figure 3 The process 300 of some embodiments of the application of the disease detection device based on multimodal images according to the present disclosure is shown. The disease detection method includes the following steps:

[0130] Step 301, obtaining a patient's historical medical record information set and current patient physical condition information from a medical record data storage device.

[0131] In some embodiments, the above-mentioned condition detection device can obtain the patient's historical medical record information set and current patient physical condition information from a medical record data storage device, wherein the historical medical record information in the above-mentioned patient's historical medical record information set includes: historical multimodal condition image data, historical condition description text information corresponding to the above-mentioned multimodal condition image data, and the above-mentioned historical multimodal condition image data includes at least one of the following: historical ultrasound image set, historical magnetic resonance imaging image set, historical computer tomography imaging image.

[0132] Step 302, desensitizing the sensitive data of the patient's historical medical record information set to obtain a desensitized patient's historical medical record information set.

[0133] In some embodiments, the above-mentioned condition detection device can perform sensitive data desensitization processing on the above-mentioned patient historical medical record information set to obtain a desensitized patient historical medical record information set.

[0134] Step 303, input the current patient's physical state information, the desensitized historical medical record information set of the patient, including the desensitized historical multimodal disease image data set and the desensitized historical disease description text information set, into the disease detection model to obtain the current disease information.

[0135] In some embodiments, the above-mentioned condition detection device can input the above-mentioned current patient physical state information, the above-mentioned desensitized patient historical medical record information set including the desensitized historical multimodal condition image data set and the desensitized historical condition description text information set into the condition detection model to obtain the current condition information.

[0136] Step 304, extract the causal relationship from the patient's historical medical record information set after desensitization to obtain a time series causal graph of the disease condition.

[0137] In some embodiments, the above-mentioned disease condition detection device can extract causal relationships from the above-mentioned desensitized patient historical medical record information set to obtain a disease condition time series causal diagram.

[0138] Step 305, input the disease time series cause-effect diagram, the patient's historical medical record information set after desensitization and the current disease information into the disease prediction model to obtain the disease development prediction information within the prediction time period.

[0139] In some embodiments, the above-mentioned disease detection device can input the above-mentioned disease time series cause and effect diagram, the above-mentioned desensitized patient historical medical record information set and the above-mentioned current disease information into the disease prediction model to obtain the disease development prediction information within the prediction time period.

[0140] Step 306, evaluating the disease progression prediction information to obtain disease level information.

[0141] In some embodiments, the disease condition detection device may perform disease condition level assessment on the disease condition progression prediction information to obtain disease condition level information.

[0142] Step 307, according to the disease level information, control the disease warning device to send disease warning information of different levels.

[0143] In some embodiments, the above-mentioned condition detection device can control the above-mentioned condition warning device to send condition warning information of different levels according to the above-mentioned condition level information.

[0144] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0145] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0146] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the patient's historical medical record information set and the current patient's physical condition information from the above-mentioned medical record data storage device, wherein the historical medical record information in the above-mentioned patient's historical medical record information set includes: historical multimodal medical condition image data, historical medical condition description text information corresponding to the above-mentioned historical multimodal medical condition image data, and the above-mentioned historical multimodal medical condition image data includes at least one of the following: historical patient ultrasound images, historical patient magnetic resonance imaging images, and historical patient computer tomography images; perform sensitive data desensitization processing on the above-mentioned patient's historical medical record information set to obtain the patient's historical medical condition after desensitization. historical information set; input the above-mentioned current patient physical state information, the above-mentioned desensitized historical multimodal disease image data set and the above-mentioned desensitized historical disease description text information set included in the above-mentioned desensitized patient historical medical record information set into the disease detection model to obtain the current disease information; extract the causal relationship of the above-mentioned desensitized patient historical medical record information set to obtain the disease time series causal graph; input the above-mentioned disease time series causal graph, the above-mentioned desensitized patient historical medical record information set and the above-mentioned current disease information into the disease prediction model to obtain the disease development prediction information within the prediction time period; perform disease level assessment on the above-mentioned disease development prediction information to obtain disease level information; according to the above-mentioned disease level information, control the above-mentioned disease warning device to send disease warning information of different levels.

[0147] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0149] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0150] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A disease detection device based on multimodal images, wherein: The multimodal image disease detection device includes: a medical record data storage device, a data desensitization device, a disease detection main control chip, and a disease warning device, wherein: The condition detection main control chip is used to obtain the patient's historical medical record information set and the current patient's physical condition information from the medical record data storage device, wherein the historical medical record information in the patient's historical medical record information set includes: historical multimodal condition image data, historical condition description text information corresponding to the historical multimodal condition image data, and the historical multimodal condition image data includes at least one of the following: historical patient ultrasound images, historical patient magnetic resonance imaging images, and historical patient computer tomography images; The data desensitizing device is used to perform sensitive data desensitization processing on the patient's historical medical record information set to obtain a desensitized patient's historical medical record information set; The condition detection main control chip is used to input the current patient's physical state information, the desensitized historical multimodal condition image data set and the desensitized historical condition description text information set included in the desensitized patient historical medical record information set into the condition detection model to obtain the current condition information; The disease condition detection main control chip is used to extract the cause-effect relationship of the patient's historical medical record information set after desensitization to obtain a disease condition time series cause-effect diagram; The disease condition detection main control chip is used to input the disease condition time series cause-effect diagram, the patient's historical medical record information set after desensitization and the current disease condition information into the disease condition prediction model to obtain disease condition development prediction information within the prediction time period; The disease condition detection main control chip is used to evaluate the disease condition level of the disease condition development prediction information to obtain disease condition level information; The disease warning device is used to send disease warning information of different levels according to the disease level information.

2. The disease condition detection device according to claim 1, wherein: The disease detection device also includes: personalized disease treatment auxiliary equipment, emotion recognition equipment, detection value recognition equipment, and target terminal, wherein: The personalized disease treatment auxiliary device is used to filter out disease treatment information corresponding to the current patient's physical condition information from the disease detection information set included in the disease condition knowledge graph corresponding to the current disease condition information, and obtain a target disease treatment information set; The emotion recognition device is used to determine the patient's emotion value for the treatment plan of each target disease treatment information in the target disease treatment information set, and obtain a treatment plan emotion value set; The detection value identification device is used to determine the treatment value value of each target disease treatment information in the target disease treatment information set according to the current patient physical state information and the desensitized patient historical medical record information set to obtain a treatment value value set; The personalized disease treatment auxiliary device is used to perform weighted summation on each treatment value value in the treatment value value set and the corresponding treatment scheme emotion value in the treatment scheme emotion value set to generate a target value value and obtain a target value value set; The personalized disease treatment auxiliary device is used to filter out the target disease treatment information with the largest corresponding target value from the target disease treatment information set as the user disease treatment plan, and send it to the target terminal for display.

3. The disease condition detection device according to claim 2, wherein: The disease condition knowledge graph is generated by the personalized disease condition treatment auxiliary device through the following steps: Acquire multi-source heterogeneous disease detection data of the same disease type as the current disease condition information, wherein the multi-source heterogeneous disease detection data includes: structured disease detection data, semi-structured disease detection data and unstructured disease detection data; Preprocessing the multi-source heterogeneous disease detection data to obtain preprocessed multi-source heterogeneous disease detection data; Performing entity relationship extraction on the pre-processed multi-source heterogeneous disease detection data to obtain a disease entity triple set; Inputting the disease entity triple set into a preset graph database to obtain an initial disease condition knowledge graph; Knowledge reasoning is performed on the initial disease condition knowledge graph to obtain a disease condition knowledge graph.

4. The disease condition detection device according to claim 3, wherein: The personalized disease treatment auxiliary device is further configured to perform entity relationship extraction processing on the pre-processed multi-source heterogeneous disease detection data through the following steps to obtain a disease entity triple set: Performing word segmentation processing on the preprocessed multi-source heterogeneous disease detection data to obtain a disease detection word segmentation set; Performing word vector representation on the disease detection word segmentation set to obtain a disease detection word segmentation vector set; Determine the disease detection word part-of-speech information of each disease detection word in the disease detection word set to obtain a disease detection word part-of-speech information set; According to the disease detection word segmentation part-of-speech information set, the disease detection word segmentation vector set is embeddedly represented to obtain a disease detection word segmentation feature vector set; Extracting semantic features from the disease detection word segmentation feature vector set to obtain a disease detection word segmentation semantic feature vector set; The disease detection word segmentation semantic feature vector set is subjected to triple joint extraction processing to obtain a disease entity triple set.

5. The disease condition detection device according to claim 4, wherein: The personalized disease treatment auxiliary device is further configured to perform triple joint extraction processing on the disease detection word segmentation semantic feature vector set through the following steps to obtain a disease entity triple set: Performing entity boundary detection on each disease detection word segmentation semantic feature vector in the disease detection word segmentation semantic feature vector set to obtain a disease detection word segmentation boundary feature vector set; Performing a linear transformation on the disease detection word segmentation boundary feature vector set to obtain a disease detection word segmentation linear feature vector set; Performing syntactic analysis on the preprocessed multi-source heterogeneous disease detection data to obtain a disease detection syntactic dependency graph; Transforming the disease detection syntactic dependency graph to obtain a disease detection syntactic dependency matrix; Performing graph attention weighted processing on the disease detection word segmentation linear feature vector set and the disease detection syntactic dependency matrix to obtain a sentence sequence feature vector set and a word segmentation dependency feature vector set; The sentence sequence feature vector set and the word segmentation dependency feature vector set are used to predict word segmentation relationships to obtain a disease entity triple set.

6. The disease condition detection device according to claim 1, wherein: The data desensitization device is further configured to perform sensitive data desensitization processing on the patient historical medical record information set through the following steps to obtain a desensitized patient historical medical record information set: Dividing the patient's historical medical record information set into text sentences to obtain a historical medical record sentence set; For each historical medical record sentence in the historical medical record sentence set, the following steps of identifying the negative words of the disease condition are performed: Performing word segmentation processing on the historical medical record sentences to obtain historical medical record word segments; Performing word matching processing on the historical medical record word groups and a preset standard medical word library to obtain a word matching result group; Constructing an index for at least one historical medical record word segment corresponding to at least one word matching result in the word matching result group indicating successful word matching, to obtain a disease index word segment group; Performing negative word recognition on the historical medical record word groups to obtain a disease condition negative word group; Determine the negative word coverage of each condition negation word in the condition negation word group to obtain a negative word coverage group; In response to determining that at least one condition index participle exists in the negative word coverage group, determining the at least one condition index participle as a negative historical medical record participle group; Removing the negated historical medical record segmentation group from the historical medical record segmentation group to obtain the removed historical medical record segmentation group as the sensitive condition segmentation group to be identified; Relative sensitive words are identified on the obtained sensitive disease segmentation word group set to be identified to obtain a disease sensitive segmentation word set; The condition-sensitive word segmentation set is semantically replaced to obtain a replaced condition segmentation set as the patient's historical medical record information set after desensitization.

7. The disease condition detection device according to claim 6, wherein: The data desensitization device is further configured to determine the negative word coverage of each condition negation word in the condition negation word group through the following steps to obtain a negative word coverage group: For each condition negation word in the condition negation word group, the following negation coverage determination steps are performed: In response to determining that the condition negation word is a postpositioned condition negation word, performing sentence structure matching on the condition negation word and a preset negation structure to obtain a sentence structure matching result; In response to determining that the sentence structure matching result indicates that the sentence structure matching is successful, a preset number of historical medical record participles located before the condition negation word are determined as a post-negation word coverage range as the negation word coverage range; In response to determining that the condition negation word is a preceding condition negation word, and that any preset condition negation termination word in a preset condition negation termination word set does not exist in the historical medical record sentence, at least one historical medical record participle included between the condition negation word and the sentence termination word is determined as a first preceding condition negation word coverage range as a negation word coverage range, wherein the sentence termination word is a historical medical record participle located at a termination position in the historical medical record sentence; In response to determining that the condition negation word is a preceding condition negation word, and any preset condition negation terminus in the preset condition negation terminus set exists in the historical medical record sentence, at least one historical medical record participle included between the condition negation word and the preset condition negation terminus is determined as the second preceding condition negation word coverage range, as the negation word coverage range.

8. A disease condition detection method, applied to the disease condition detection device according to any one of claims 1 to 7, the method comprising: Acquire a patient's historical medical record information set and current patient physical condition information from the medical record data storage device, wherein the historical medical record information in the patient's historical medical record information set includes: historical multimodal disease image data, historical disease description text information corresponding to the historical multimodal disease image data, and the historical multimodal disease image data includes at least one of the following: historical patient ultrasound images, historical patient magnetic resonance imaging images, and historical patient computer tomography images; Performing sensitive data desensitization processing on the patient's historical medical record information set to obtain a desensitized patient's historical medical record information set; Input the current patient's physical state information, the desensitized historical multimodal condition image data set and the desensitized historical condition description text information set included in the desensitized patient historical medical record information set into the condition detection model to obtain the current condition information; Extracting causal relationships from the patient's historical medical record information set after desensitization to obtain a time-series causal graph of the disease condition; Input the disease time series cause-effect diagram, the patient's historical medical record information set after desensitization, and the current disease information into the disease prediction model to obtain disease progression prediction information within the prediction time period; Performing a disease level assessment on the disease progression prediction information to obtain disease level information; According to the disease level information, the disease warning device is controlled to send disease warning information of different levels.

9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claim 8.

10. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to claim 8 is implemented.

Images