Disease early warning method and device, electronic equipment and medium

Abstract

The invention relates to a disease early warning method and device, electronic equipment and a medium, and belongs to the technical field of medical decision, and the method comprises the steps: obtaining image data and physiological signal data of a patient; inputting the image data into a fully trained convolutional neural network model to obtain a first feature, and inputting the physiological signal data into a fully trained long-short-term memory network model to obtain a second feature; inputting the first feature and the second feature into a fully trained dynamic weighted graph attention network model to obtain a comprehensive feature; the comprehensive features are input into a decision model which is completely trained, a disease attack risk value is obtained, early warning is carried out based on the disease attack risk value, and the decision model is constructed based on a neural network. The structural features of the image data are extracted through the neural network, the time sequence features of the physiological signal data are extracted through the long-short-term memory network, multi-modal medical data deep fusion is achieved, and the diagnosis precision is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of medical decision-making technology, and in particular to a disease early warning method, device, electronic equipment and medium. Background Art

[0002] With the rapid development of information technology and the growing demand in the medical field, medical decision warning systems have become a hot research area today.

[0003] Traditional medical decision-making and early warning systems process data in various modalities relatively independently. Imaging data is collected by specialized imaging diagnostic equipment and then processed by image analysis software; physiological signals are collected and analyzed by independent monitoring instruments, lacking an effective fusion mechanism. Therefore, traditional medical decision-making and early warning systems are mostly based on single-modal data, such as relying solely on imaging data (CT, MRI, etc.) or a single type of physiological signal (such as ECG) using simple statistical analysis or rule-based engines for analysis.

[0004] Single-modal data cannot fully reflect the patient's condition. Different diseases have different physiological and imaging manifestations in many aspects. Relying solely on a single information source can easily lead to missing key diagnostic clues, resulting in high misdiagnosis and missed diagnosis rates and poor early warning accuracy. Summary of the Invention

[0005] In view of this, it is necessary to provide a disease warning method, device, electronic device and medium to solve the technical problem of poor warning accuracy relying on single-modal data in the existing technology.

[0006] In order to solve the above problems, in a first aspect, the present invention provides a disease early warning method, comprising: Acquire patient imaging data and physiological signal data; Inputting the image data into a well-trained convolutional neural network model to obtain a first feature, and inputting the physiological signal data into a well-trained long short-term memory network model to obtain a second feature; Inputting the first feature and the second feature into a well-trained dynamic weighted graph attention network model to obtain a comprehensive feature; The comprehensive features are input into a well-trained decision model to obtain a disease onset risk value, and an early warning is issued based on the disease onset risk value. The decision model is constructed based on a neural network.

[0007] In a possible implementation, after acquiring the patient's image data and physiological signal data, the method further includes: De-noising and standardization processing is performed on the image data and physiological signal data to obtain pre-processed image data and physiological signal data.

[0008] In one possible implementation, after inputting the image data into a well-trained convolutional neural network model to obtain the first feature and inputting the physiological signal data into a well-trained long short-term memory network model to obtain the second feature, the method further includes: Based on a timing alignment algorithm, the image data and the physiological signal data are time-synchronized to obtain time-aligned image data and physiological signal data.

[0009] In a possible implementation, the timing alignment algorithm adopts a dynamic time warping algorithm.

[0010] In a possible implementation, the decision model adopts a dual deep Q network.

[0011] In one possible implementation, the method is applied to an edge device, and initial model parameter data of the convolutional neural network model, the long short-term memory network model, and the decision model are all trained on the edge device. The method further includes: Uploading the initial model parameter data to the cloud, where the cloud is used to perform weighted averaging on the initial model parameter data of multiple edge devices to obtain updated model parameter data; Receive updated model parameter data from the cloud, and update the model parameters of the convolutional neural network model, the long short-term memory network model, and the decision model based on the updated model parameter data.

[0012] In one possible implementation, the cloud performs weighted averaging on the initial model parameter data of multiple edge devices based on a federated learning framework based on PySyft.

[0013] In a second aspect, the present invention further provides a disease early warning device, comprising: A data acquisition unit, used to acquire imaging data and physiological signal data of the patient; a feature extraction unit, configured to input the image data into a well-trained convolutional neural network model to obtain a first feature, and input the physiological signal data into a well-trained long short-term memory network model to obtain a second feature; a feature fusion unit, configured to input the first feature and the second feature into a well-trained dynamic weighted graph attention network model to obtain a comprehensive feature; A decision-making and early warning unit is used to input the comprehensive features into a well-trained decision model to obtain a disease onset risk value, and to issue an early warning based on the disease onset risk value. The decision model is constructed based on a neural network.

[0014] In a third aspect, the present invention further provides an electronic device comprising a memory and a processor; The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the disease early warning method described above.

[0015] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the disease warning method described above are implemented.

[0016] The beneficial effects of the present invention are as follows: the disease warning method provided by the present invention extracts the structural features of image data through a neural network, extracts the temporal features of physiological signal data through a long short-term memory network, and performs weighted fusion on the features extracted from different modal data through a dynamic weighted graph attention network. The obtained comprehensive features are sent to the final decision warning model for judgment and prediction, thereby realizing deep fusion of multimodal medical data, fully mining the correlation information between data, effectively reducing the risk of misdiagnosis and missed diagnosis, improving diagnostic accuracy, and ensuring the safety of clinical decision-making.

[0017] Furthermore, traditional systems mostly perform post-analysis or monitor data at fixed time intervals, making it difficult to respond immediately to sudden changes in the patient's condition. The rule engine needs to traverse all preset rules, which is complex and time-consuming to calculate. In the scenario of rapid progression of acute diseases, decision delays seriously affect the treatment effect. Fixed rules cannot flexibly adjust decision-making strategies according to individual differences of patients and dynamic changes in their condition. The present invention extracts trained models to quickly adjust diagnosis and treatment decisions according to the patient's real-time status, thereby enhancing the real-time performance and flexibility of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic flow chart of an embodiment of the disease early warning method provided by the present invention; Figure 2 A flow chart of another embodiment of the disease early warning method provided by the present invention; Figure 3 A schematic structural diagram of an embodiment of the disease early warning device provided by the present invention; Figure 4 This is a schematic structural diagram of an embodiment of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] It should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present invention illustrate operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps that have no logical contextual relationship can be reversed in order or implemented simultaneously. In addition, those skilled in the art, guided by the content of the present invention, can add one or more other operations to the flowcharts or remove one or more operations from the flowcharts. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0021] The terms "first" and "second" in the embodiments of the present invention are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, technical features specified as "first" or "second" may explicitly or implicitly include at least one of these features. "And / or" describes the association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone.

[0022] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] The present invention provides a disease early warning method, device, electronic device and medium, which are described below respectively.

[0024] Figure 1 A flow chart of an embodiment of the disease early warning method provided by the present invention is shown as follows: Figure 1 As shown, disease early warning methods include: S101, obtaining imaging data and physiological signal data of a patient; It should be noted that imaging data usually refers to ultrasound images, CT (Computed Tomography) images, and MRI (Magnetic Resonance Imaging) images, etc., and physiological signal data usually refers to ECG (Electrocardiogram, reflecting heart activity) signals, PPG (Photoplethysmography) heart rate signals, and EEG (Electroencephalogram) brain wave signals. Imaging data and physiological signal data need to be selected according to the type of disease to be warned. Taking the epilepsy warning scenario as an example, imaging data includes brain acoustic wave image data, and physiological signal data includes PPG heart rate signals and EEG brain wave signal data. The above data are all collected in real time.

[0025] In order to obtain higher quality image data and physiological signal data, in some embodiments of the present invention, after S101, the method further includes: The image data and physiological signal data are subjected to denoising and standardization processing to obtain preprocessed image data and physiological signal data.

[0026] It should be noted that, taking the epilepsy warning scenario as an example, the preprocessing steps include: removing interference such as baseline drift and high-frequency noise in the signal, normalizing the heart rate signal to the [0,1] interval, decomposing and calculating the energy of the EEG signal by frequency band (such as δ waves, θ waves, α waves, β waves, etc.), and adjusting the resolution and contrast of the ultrasound image.

[0027] S102. Input the image data into a well-trained convolutional neural network model to obtain a first feature, and input the physiological signal data into a well-trained long short-term memory network model to obtain a second feature; It should be noted that the 3D-CNN (Convolutional Neural Networks) model can better extract the primary features of image data (such as hippocampal morphology and ventricular size), and the Long Short-Term Memory (LSTM) model can better extract the secondary features of physiological signal data (such as the RR interval change pattern of heart rate signals and the rhythmic changes of EEG signals).

[0028] In order to ensure the consistency of different modal data in time and space dimensions and lay the foundation for subsequent fusion, in some embodiments of the present invention, after step S102, the method further includes: Based on the timing alignment algorithm, the image data and physiological signal data are time-synchronized to obtain the time-aligned image data and physiological signal data. The timing alignment algorithm adopts the dynamic time warping algorithm.

[0029] It should be noted that the timing alignment algorithm uses the dynamic time warping (DTW) algorithm to better align the first feature and the second feature (such as accurately matching the moment of abnormal heart rate fluctuation with the period of epileptic-like discharge of brain waves).

[0030] S103, inputting the first feature and the second feature into a well-trained dynamic weighted graph attention network model to obtain a comprehensive feature; It should be noted that the Dynamic-weighted Graph Attention Network (DGA-NET) dynamically weights different modal features (such as heart rate variability, EEG α-wave spectrum, organ scan structural features, etc.) according to the importance of each modal data feature to generate a high-dimensional joint representation, highlight key diagnostic information, and suppress irrelevant interference. Taking the epilepsy warning scenario as an example, it can better dynamically weight heart rate variability features, EEG θ wave and γ wave spectrum features, brain structure features, etc. to generate comprehensive features.

[0031] S104. Input the comprehensive features into a well-trained decision model to obtain a disease onset risk value, and issue an early warning based on the disease onset risk value. The decision model is constructed based on a neural network.

[0032] In order to make better decisions, in some embodiments of the present invention, the decision model adopts a Double Deep Q-Network (DDQN).

[0033] It should be noted that the ‌DDQN main network generates diagnosis and treatment decisions in real time based on fused features. If the risk exceeds a preset threshold (such as 0.8), a warning signal is immediately issued through the headphones, and preliminary diagnosis and treatment recommendations are pushed (such as keeping the patient in a side-lying position, avoiding strong light stimulation, etc.), while recording the decision log. The ‌DDQN target network evaluates decision quality from the perspective of long-term clinical benefits, and continuously optimizes the stability of the decision-making strategy through KL divergence constraints, so that the system gradually approaches the optimal strategy in the dynamic decision-making process. The lightweight DDQN inference module (model size <5MB) is based on an optimized neural network architecture. After training with a large amount of clinical data, it can quickly process data at the edge and generate preliminary decisions, with a response time of <200ms.

[0034] It should also be noted that the system continuously monitors the patient's real-time status (such as blood oxygen saturation, blood pressure, heart rate and other vital signs). Once the status indicator triggers the preset threshold (such as blood oxygen saturation <92%), the dynamic action space clipping mechanism is immediately activated. The set of optional diagnosis and treatment actions is limited according to the patient's current condition, and urgent and appropriate treatment measures (such as oxygen supply therapy) are recommended first to ensure that the decision is timely and meets the patient's actual needs. With the help of dynamic action space clipping and the optimized DDQN algorithm, the system can respond quickly according to the patient's real-time status, and the decision delay is shortened from the traditional 500ms to 180ms, quickly providing accurate diagnosis and treatment recommendations, effectively reducing clinical risks, and ensuring that patients receive timely treatment during the critical period of disease changes.

[0035] Compared with the existing technology, this application extracts the structural features of image data through neural networks, extracts the temporal features of physiological signal data through long-short-term memory networks, and performs weighted fusion of features extracted from different modal data through dynamic weighted graph attention networks. The obtained comprehensive features are sent to the final decision-making and early warning model for judgment and prediction, realizing deep fusion of multimodal medical data, fully mining the correlation information between data, effectively reducing the risk of misdiagnosis and missed diagnosis, improving diagnostic accuracy, and ensuring the safety of clinical decision-making.

[0036] Furthermore, traditional systems mostly perform post-analysis or monitor data at fixed time intervals, making it difficult to respond immediately to sudden changes in the patient's condition. The rule engine needs to traverse all preset rules, which is complex and time-consuming to calculate. In the scenario of rapid progression of acute diseases, decision delays seriously affect the treatment effect. Fixed rules cannot flexibly adjust decision-making strategies according to individual differences of patients and dynamic changes in their condition. The present invention extracts trained models to quickly adjust diagnosis and treatment decisions according to the patient's real-time status, thereby enhancing the real-time performance and flexibility of the system.

[0037] Considering the difficulties in data sharing and collaboration in multi-center medical data applications, the data of each medical institution is stored in isolation and there is a lack of a unified and secure sharing mechanism. Traditional data transmission is prone to privacy leakage and cannot effectively integrate multi-source data to improve the generalization ability of the model, which limits the performance improvement of the medical decision-making system driven by large-scale data. To solve the above problems, in some embodiments of the present invention, the method is applied to the edge device of the hospital, and the initial model parameter data of the convolutional neural network model, the long short-term memory network model and the decision model are all trained on the edge device, such as Figure 2 As shown, the method further includes: S201: Upload the initial model parameter data to the cloud through an encrypted channel. The cloud is used to perform weighted averaging on the initial model parameter data of multiple edge devices to obtain updated model parameter data. In order to better obtain updated model parameter data, in some embodiments of the present invention, the cloud performs weighted averaging on the initial model parameter data of multiple edge devices based on the PySyft federated learning framework.

[0038] It should be noted that the use of the federated learning framework and blockchain technology can meet the requirements of strict regulations such as GDPR / HIPAA, realize multi-center data sharing and collaborative training while ensuring data privacy and security, improve the model generalization ability, overcome the data island problem, and open up a safe path for medical big data applications.

[0039] S202. Receive updated model parameter data from the cloud, and update model parameters of the convolutional neural network model, the long short-term memory network model, and the decision model based on the updated model parameter data.

[0040] It should be noted that the blockchain evidence storage module records the time, source, content hash value and other information of each data upload and model update to ensure that the data is traceable and cannot be tampered with.

[0041] It should also be noted that with the help of edge-cloud collaborative computing and blockchain technologies, data silos are broken, data privacy and security are ensured, multi-center data collaborative training and efficient utilization are achieved, and the system's generalization capabilities in different clinical scenarios are improved; through continuous iterative training and optimization on multi-center, large-scale patient data, the system performance continues to improve, playing an increasingly accurate and reliable decision-making support role in epilepsy warning and other acute disease diagnosis scenarios, providing strong assistance to medical personnel and ensuring the health and safety of patients.

[0042] In order to better implement a disease early warning method in an embodiment of the present invention, based on a disease early warning method, correspondingly, Figure 3 As shown, the embodiment of the present invention further provides a disease early warning device 300, comprising: The data acquisition unit 301 is used to acquire the patient's image data and physiological signal data; A feature extraction unit 302 is configured to input the image data into a well-trained convolutional neural network model to obtain a first feature, and input the physiological signal data into a well-trained long short-term memory network model to obtain a second feature; A feature fusion unit 303 is configured to input the first feature and the second feature into a well-trained dynamic weighted graph attention network model to obtain a comprehensive feature; The decision warning unit 304 is used to input the comprehensive features into a well-trained decision model to obtain a disease onset risk value, and issue a warning based on the disease onset risk value. The decision model is constructed based on a neural network.

[0043] The disease warning device 305 provided in the above embodiment can implement the technical solution described in the above disease warning method embodiment. The specific implementation principles of the above units can refer to the corresponding contents in the above disease warning method embodiment, which will not be repeated here.

[0044] like Figure 4 As shown, the present invention also provides an electronic device 400. The electronic device 400 includes a processor 401, a memory 402 and a display 403. Figure 4 Only some of the components of the electronic device 400 are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0045] In some embodiments, the memory 402 may be an internal storage unit of the electronic device 400, such as a hard disk or memory of the electronic device 400. In other embodiments, the memory 402 may also be an external storage device of the electronic device 400, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 400.

[0046] Furthermore, the memory 402 may include both an internal storage unit of the electronic device 400 and an external storage device. The memory 402 is used to store application software installed in the electronic device 400 and various data.

[0047] In some embodiments, the processor 401 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 402, such as the disease early warning method of the present invention.

[0048] In some embodiments, display 403 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 403 is used to display information on electronic device 400 and to display a visual user interface. Components 401-403 of electronic device 400 communicate with each other via a system bus.

[0049] In some embodiments of the present invention, when the processor 401 executes the disease early warning program in the memory 402, the following steps may be implemented: Acquire patient imaging data and physiological signal data; The image data is fed into a well-trained convolutional neural network model to obtain the first feature, and the physiological signal data is fed into a well-trained long short-term memory network model to obtain the second feature; Input the first and second features into the well-trained dynamic weighted graph attention network model to obtain the comprehensive features; The comprehensive features are input into a well-trained decision-making model to obtain the disease onset risk value, and an early warning is issued based on the disease onset risk value. The decision-making model is built based on a neural network.

[0050] It should be understood that, when the processor 401 executes the disease warning program in the memory 402 , in addition to the above functions, it can also implement other functions. For details, please refer to the description of the corresponding method embodiment above.

[0051] Furthermore, the embodiment of the present invention does not specifically limit the type of the electronic device 400 mentioned. The electronic device 400 may be a portable electronic device such as a mobile phone, a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, or the like. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices equipped with IOS, Android, Microsoft, or other operating systems. The above-mentioned portable electronic devices may also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 400 may not be a portable electronic device, but a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0052] Accordingly, an embodiment of the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps or functions of the disease warning method provided in the above-mentioned method embodiments can be implemented.

[0053] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the above-described program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0054] The above is a detailed introduction to a disease warning method provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

[0055] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A disease early warning method, characterized in that: include: Acquire patient imaging data and physiological signal data; Inputting the image data into a well-trained convolutional neural network model to obtain a first feature, and inputting the physiological signal data into a well-trained long short-term memory network model to obtain a second feature; Inputting the first feature and the second feature into a well-trained dynamic weighted graph attention network model to obtain a comprehensive feature; The comprehensive features are input into a well-trained decision model to obtain a disease onset risk value, and an early warning is issued based on the disease onset risk value. The decision model is constructed based on a neural network.

2. The disease early warning method according to claim 1, characterized in that: After acquiring the patient's image data and physiological signal data, the method further includes: De-noising and standardization processing is performed on the image data and physiological signal data to obtain pre-processed image data and physiological signal data.

3. The disease early warning method according to claim 1, characterized in that: After inputting the image data into a well-trained convolutional neural network model to obtain the first feature, and inputting the physiological signal data into a well-trained long short-term memory network model to obtain the second feature, the method further includes: Based on a timing alignment algorithm, the image data and the physiological signal data are time-synchronized to obtain time-aligned image data and physiological signal data.

4. The disease early warning method according to claim 3, characterized in that: The timing alignment algorithm adopts a dynamic time warping algorithm.

5. The disease early warning method according to claim 1, characterized in that: The decision model adopts a dual deep Q network.

6. The disease early warning method according to claim 1, characterized in that: The method is applied to an edge device, wherein initial model parameter data of the convolutional neural network model, the long short-term memory network model, and the decision model are all trained on the edge device, and the method further includes: Uploading the initial model parameter data to the cloud, where the cloud is used to perform weighted averaging on the initial model parameter data of multiple edge devices to obtain updated model parameter data; Receive updated model parameter data from the cloud, and update the model parameters of the convolutional neural network model, the long short-term memory network model, and the decision model based on the updated model parameter data.

7. The disease early warning method according to claim 6, characterized in that: The cloud performs weighted averaging of initial model parameter data of multiple edge devices based on the PySyft-based federated learning framework.

8. A disease early warning device, characterized in that: include: A data acquisition unit, used to acquire imaging data and physiological signal data of the patient; a feature extraction unit, configured to input the image data into a well-trained convolutional neural network model to obtain a first feature, and input the physiological signal data into a well-trained long short-term memory network model to obtain a second feature; a feature fusion unit, configured to input the first feature and the second feature into a well-trained dynamic weighted graph attention network model to obtain a comprehensive feature; A decision-making and early warning unit is used to input the comprehensive features into a well-trained decision model to obtain a disease onset risk value, and to issue an early warning based on the disease onset risk value. The decision model is constructed based on a neural network.

9. An electronic device, characterized in that: including memory and processor; The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps of the disease early warning method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the disease early warning method described in any one of claims 1 to 7 are implemented.

Images