Remote medical aid decision-making method and system based on AI large model

Abstract

The invention provides a remote medical aid decision-making method and system based on an AI large model. The method comprises the following steps: acquiring a first data result and a second data result; obtaining the difference between the first data result and the second data result based on a joint latent variable model, and taking the difference between the first data result and the second data result as a third data result; comprehensively analyzing the first data result, the second data result and the third data result based on an AI large model, and obtaining an influence factor generating the third data result as a fourth data result; based on the classification level, obtaining the current state information value of the patient; the state interval of the patient is obtained based on the state information value and the fourth data result, different remote medical aid decisions are started based on the different state intervals of the patient, the patient and a doctor do not need to go to a hospital or a clinic personally through remote medical aid, effective utilization of medical resources can be achieved, and the coverage range of medical services can be widened.

Description

Technical Field

[0001] The present application relates to the technical field of remote medical decision-making assistance, and specifically to a remote medical decision-making assistance method and system based on an AI large model. Background Art

[0002] With the rapid development of artificial intelligence and big data technology, telemedicine, as an innovative medical service method, is gradually becoming a hot spot in the medical industry. In telemedicine, the use of large models (large artificial intelligence models) for auxiliary decision-making has become an important trend.

[0003] Telemedicine decision-making assistance refers to the use of artificial intelligence technology to assist doctors in making diagnosis and treatment decisions by analyzing patients' clinical data, imaging materials and other information. In recent years, with the breakthroughs in deep learning and natural language processing technology, large models (such as GPT-3, BERT, etc.) have been increasingly used in the medical field and have become one of the important tools for telemedicine decision-making assistance. Although there are many studies on telemedicine using artificial intelligence, there are fewer studies on the use of large models to connect patients with contracted doctors. This makes it impossible for patients to directly contact relevant doctors when their physical condition changes, and then provide timely reminders to patients. This will result in patients being unable to make timely changes to their medication or seek medical treatment when their condition changes, which will lead to more serious consequences.

[0004] Therefore, how to overcome the above-mentioned technical problems and defects becomes a key issue that needs to be addressed. Summary of the invention

[0005] In order to overcome the problems existing in the above-mentioned prior art, the present application provides a remote medical decision-making assistance method and system based on an AI large model, which adopts the following technical solutions:

[0006] In the first aspect, the present application provides a remote medical decision-making assistance method based on an AI large model, comprising:

[0007] Acquiring first multimodal data information and second multimodal data information of a patient;

[0008] A multimodal fusion model using an attention mechanism is used to perform feature fusion on the first multimodal data information and the second multimodal data information respectively to obtain a first data result and a second data result;

[0009] Obtaining the difference between the first data result and the second data result based on the joint latent variable model, and using the difference between the first data result and the second data result as the third data result;

[0010] Perform a comprehensive analysis on the first data result, the second data result and the third data result based on the AI ​​big model to obtain the influencing factors of the third data result;

[0011] Using a binary classifier to classify the third data result, obtaining a classification level of the third data result, and obtaining a current state information value of the patient based on the classification level;

[0012] The patient's state interval is obtained based on the state information value and the fourth data result, and different remote medical auxiliary decisions are enabled based on the patient's different state intervals.

[0013] Furthermore, the first multimodal data information and the second multimodal data information of the patient are obtained, wherein the first multimodal data information and the second multimodal data information include text data information, image data information, and voice data information.

[0014] Furthermore, the first multimodal data information and the second multimodal data information of the patient are obtained, wherein the first multimodal data information is historical multimodal data information, and the second multimodal data information is real-time multimodal data information.

[0015] Furthermore, the multimodal fusion model using the attention mechanism performs feature fusion on the first multimodal data information and the second multimodal data information respectively to obtain a first data result and a second data result, wherein the first data result is a data result of historical multimodal data information, and the second data result is a data result of real-time multimodal data information.

[0016] Furthermore, the multimodal fusion model using the attention mechanism performs feature fusion on the first multimodal data information and the second multimodal data information respectively to obtain a first data result and a second data result, including:

[0017] Step S201, performing feature extraction on each modal data in the first multimodal data information and the second multimodal data information respectively to obtain a feature representation of each modal data;

[0018] Step S202, obtaining the attention weight of each modal data through an attention weight calculation module for the feature representation of each modal data;

[0019] Step S203, multiplying the attention weight of each modal data by the feature of each modal data to obtain a weighted feature representation of each modal data;

[0020] Step S204, concatenating and summing the weighted feature representations of each modal data to obtain final multimodal feature representations of the first multimodal data information and the second multimodal data information, that is, a first data result and a second data result.

[0021] Further, the obtaining the difference between the first data result and the second data result based on the joint latent variable model, and taking the difference between the first data result and the second data result as the third data result, includes:

[0022] Step S301, jointly modeling the first multimodal data information and the second multimodal data information to obtain shared latent variables of the first data result and the second data result;

[0023] Step S302, obtaining representations of the first data result and the second data result in a shared latent variable by maximizing the joint likelihood learning latent variable model of the data;

[0024] Step S303: compare the representations of the first data result and the second data result in the shared latent variable to obtain the difference between the first data result and the second data result.

[0025] Furthermore, the obtaining of the patient's state interval based on the state information value and the fourth data result, and enabling different telemedicine auxiliary decision making based on different state intervals of the patient, include:

[0026] The state interval includes a first state interval, a second state interval and a third state interval; wherein the first state interval is a state in which the patient's health state is in a worsening trend, the second state interval is a state in which the patient's health state is in a mitigating trend, and the third state interval is a state in which the patient's health state is in a stable state.

[0027] In the second aspect, the present application also provides a remote medical decision-making assistance system based on an AI large model, including:

[0028] A multimodal data information acquisition module, used to acquire first multimodal data information and second multimodal data information of a patient;

[0029] A multimodal feature fusion module, configured to use a multimodal fusion model with an attention mechanism to perform feature fusion on the first multimodal data information and the second multimodal data information respectively, to obtain a first data result and a second data result;

[0030] A third data result acquisition module, configured to acquire the difference between the first data result and the second data result based on a joint latent variable model, and use the difference between the first data result and the second data result as a third data result;

[0031] A comprehensive analysis module, used to perform a comprehensive analysis on the first data result, the second data result and the third data result based on the AI ​​big model, and obtain the influencing factors of the third data result as the fourth data result;

[0032] A state information value acquisition module, used to classify the third data result using a binary classifier, obtain the classification level of the third data result, and obtain the current state information value of the patient based on the classification level;

[0033] The status information value judgment module is used to obtain the patient's status interval based on the status information value and the fourth data result, and enable different remote medical auxiliary decision-making based on the patient's different status intervals.

[0034] In a third aspect, the present application provides an electronic device, including:

[0035] One or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, cause the device to perform the method as described in the first aspect.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.

[0037] In a fifth aspect, the present application provides a computer program, which, when executed by a computer, is used to execute the method described in the first aspect.

[0038] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor.

[0039] This application has the following beneficial effects:

[0040] 1. This application receives multimodal medical data information of patients based on a remote communication unit, uses a binary classifier to analyze the multimodal medical data information, and obtains a first data result, which helps to achieve remote communication and information sharing between doctors and patients, improve the efficiency and quality of medical services, and also promote the digital transformation and intelligent development of the medical industry; helps medical staff to understand the patient's condition more comprehensively, and adjust the treatment plan according to real-time data, thereby improving the quality and efficiency of medical care; at the same time, it can also enable patients to receive medical services conveniently at home, reducing the cost and time cost of medical treatment for patients;

[0041] 2. This application obtains historical medical data information through the hospital's electronic medical record system, laboratory information system, and imaging database, and obtains real-time medical data information through real-time monitoring by medical equipment, remote monitoring equipment, wearable devices, etc.; the text data information, imaging data information, and voice data information in the historical medical data information and real-time medical data information are of great significance for medical decision-making and patient monitoring, and help improve the quality and efficiency of medical services;

[0042] 3. This application uses the strong learning and generalization capabilities of large models to learn potential patterns and rules from a large amount of medical data; and telemedicine does not require patients and doctors to visit hospitals or clinics in person, which can achieve effective use of medical resources and improve the coverage of medical services;

[0043] 4. This application analyzes the patient's medical data and condition information through the decision-making support of a large model to assist doctors in making diagnosis and treatment decisions; the large model can use its learned knowledge and pattern recognition capabilities to help doctors accurately diagnose patients, develop personalized treatment plans, and provide real-time monitoring and feedback to support the implementation and optimization of telemedicine services.

[0044] 5. This application obtains the differences between historical medical data and real-time medical data, and obtains the influencing factors of the differences based on the AI ​​big model, so as to obtain the patient's health status change data information in a timely manner, so that the patient can be given early warning information in a timely manner. At the same time, based on the patient's early warning information, the doctor can make timely decisions for the patient, and can intervene in the patient in time or in advance to reduce the occurrence of dangerous events. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0046] Figure 2 A flow chart of a method according to an embodiment of the present application;

[0047] Figure 3 This is a flow chart of a multimodal feature fusion module according to an embodiment of the present application;

[0048] Figure 4 A flowchart of obtaining the third data result of an embodiment of the present application;

[0049] Figure 5 This is a flow chart of the device according to an embodiment of the present application;

[0050] Figure 6 It is a schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0052] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0053] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0054] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0055] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0056] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Group Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) players, laptop computers and desktop computers, etc.

[0057] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0058] It should be noted that the remote medical decision-making assistance method based on the AI ​​big model provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the remote medical decision-making assistance system based on the AI ​​big model is generally set in the server / terminal device.

[0059] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0060] Continue to refer Figure 2 , the figure shows a flow chart of a remote medical decision-making assistance method based on an AI large model of the present application, the method comprising the following steps:

[0061] Step S1, obtaining first multimodal data information and second multimodal data information of a patient;

[0062] In a possible implementation, the first multimodal data information and the second multimodal data information of the patient are obtained, wherein the first multimodal data information and the second multimodal data information include text data information, image data information, and voice data information.

[0063] In a possible implementation, the first multimodal data information and the second multimodal data information of the patient are acquired, wherein the first multimodal data information is historical multimodal data information, and the second multimodal data information is real-time multimodal data information.

[0064] Step S2, using a multimodal fusion model with an attention mechanism to perform feature fusion on the first multimodal data information and the second multimodal data information respectively, to obtain a first data result and a second data result;

[0065] In a possible implementation, the multimodal fusion model using the attention mechanism performs feature fusion on the first multimodal data information and the second multimodal data information respectively to obtain a first data result and a second data result, wherein the first data result is a data result of historical multimodal data information, and the second data result is a data result of real-time multimodal data information.

[0066] In a possible trial method, the multimodal fusion model using the attention mechanism performs feature fusion on the first multimodal data information and the second multimodal data information respectively. To obtain the first data result and the second data result, please refer to Figure 3 , the specific steps include:

[0067] Step S201, performing feature extraction on each modal data in the first multimodal data information and the second multimodal data information respectively to obtain a feature representation of each modal data;

[0068] Step S202, obtaining the attention weight of each modal data through an attention weight calculation module for the feature representation of each modal data;

[0069] Step S203, multiplying the attention weight of each modal data by the feature of each modal data to obtain a weighted feature representation of each modal data;

[0070] Step S204, concatenating and summing the weighted feature representations of each modal data to obtain final multimodal feature representations of the first multimodal data information and the second multimodal data information, that is, a first data result and a second data result.

[0071] Step S3, obtaining the difference between the first data result and the second data result based on the joint latent variable model, and taking the difference between the first data result and the second data result as the third data result;

[0072] The difference between the first data result and the second data result is obtained based on the joint latent variable model, and the difference between the first data result and the second data result is used as the third data result. Please refer to Figure 4 , the specific steps include:

[0073] Step S301, jointly modeling the first multimodal data information and the second multimodal data information to obtain shared latent variables of the first data result and the second data result;

[0074] Step S302, obtaining representations of the first data result and the second data result in a shared latent variable by maximizing the joint likelihood learning latent variable model of the data;

[0075] Step S303: compare the representations of the first data result and the second data result in the shared latent variable to obtain the difference between the first data result and the second data result.

[0076] Step S4, performing a comprehensive analysis on the first data result, the second data result and the third data result based on the AI ​​big model, and obtaining the influencing factors of the third data result as the fourth data result;

[0077] Step S5, using a binary classifier to classify the third data result, obtaining a classification level of the third data result, and obtaining a current state information value of the patient based on the classification level;

[0078] Step S6, obtaining the patient's state interval based on the state information value and the fourth data result, and enabling different remote medical auxiliary decision-making based on the patient's different state intervals.

[0079] In a possible implementation, the state interval includes a first state interval, a second state interval and a third state interval; wherein the first state interval is a state in which the patient's health state is in a worsening trend, the second state interval is a state in which the patient's health state is in a mitigating trend, and the third state interval is a state in which the patient's health state is in a stable state.

[0080] In a possible implementation, the patient's status interval is obtained based on the status information value and the fourth data result, and different remote medical auxiliary decision-making is enabled based on the patient's different status intervals, including: judging and analyzing the patient's current status information value, if the status information value is in the first status interval, the patient's health status is in a worsening trend state, and the remote communication unit sends a first warning message to the patient and the patient's contracted doctor, reminding the patient to seek medical treatment in time, and reminding the patient's contracted doctor to re-diagnose the patient in time; if the status information value is in the second status interval, the patient's health status is in a mitigating trend state, and the remote communication unit sends a second warning message to the patient and the patient's contracted doctor, reminding the patient's contracted doctor to re-diagnose the patient in time; if the patient is taking medication, the patient is reminded to go to the hospital to replace the medication; if the status information value is in the third status interval, the patient's health status is in a stable state, and the remote communication unit does not send a warning message to the patient and the patient's contracted doctor.

[0081] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0082] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0083] Continue to refer Figure 5 The remote medical decision-making assistance system based on the AI ​​big model described in this embodiment includes:

[0084] A multimodal data information acquisition module 501 is used to acquire first multimodal data information and second multimodal data information of a patient;

[0085] A multimodal feature fusion module 502, configured to use a multimodal fusion model with an attention mechanism to perform feature fusion on the first multimodal data information and the second multimodal data information respectively, to obtain a first data result and a second data result;

[0086] A third data result acquisition module 503 is used to acquire the difference between the first data result and the second data result based on the joint latent variable model, and use the difference between the first data result and the second data result as the third data result;

[0087] A comprehensive analysis module 504 is used to perform a comprehensive analysis on the first data result, the second data result and the third data result based on the AI ​​big model, and obtain the influencing factors of the third data result as the fourth data result;

[0088] The state information value acquisition module 505 is used to classify the third data result using a binary classifier, obtain the classification level of the third data result, and obtain the current state information value of the patient based on the classification level;

[0089] The state information value judgment module 506 is used to obtain the state interval of the patient based on the state information value and the fourth data result, and enable different remote medical auxiliary decision-making based on different state intervals of the patient.

[0090] To solve the above technical problems, the present application also provides a computer device. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.

[0091] The computer device 6 includes a memory 6a, a processor 6b, and a network interface 6c that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 6a-6c, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0092] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0093] The memory 6a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 6a can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 6a can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 6a can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 6a is generally used to store the operating system and various application software installed on the computer device 6, such as the program code of the remote medical auxiliary decision-making method based on the AI ​​large model, etc. In addition, the memory 6a can also be used to temporarily store various types of data that have been output or are to be output.

[0094] The processor 6b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 6b is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 6b is used to run the program code stored in the memory 6a or process data, such as running the program code of the remote medical auxiliary decision-making method based on the AI ​​large model.

[0095] The network interface 6c may include a wireless network interface or a wired network interface. The network interface 6c is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0096] The present application also provides another embodiment, namely, providing a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a program of a remote medical decision-making assistance method based on an AI big model, and the remote medical decision-making assistance program based on an AI big model can be executed by at least one processor so that the at least one processor performs the steps of the remote medical decision-making assistance method based on an AI big model as described above.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0098] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A remote medical decision-making assistance method based on an AI large model, characterized in that: include: Acquiring first multimodal data information and second multimodal data information of a patient; A multimodal fusion model using an attention mechanism is used to perform feature fusion on the first multimodal data information and the second multimodal data information respectively to obtain a first data result and a second data result; Obtaining the difference between the first data result and the second data result based on the joint latent variable model, and using the difference between the first data result and the second data result as the third data result; Perform a comprehensive analysis on the first data result, the second data result, and the third data result based on the AI ​​big model to obtain an influencing factor of the third data result as a fourth data result; Using a binary classifier to classify the third data result, obtaining a classification level of the third data result, and obtaining a current state information value of the patient based on the classification level; The patient's state interval is obtained based on the state information value and the fourth data result, and different remote medical auxiliary decisions are enabled based on the patient's different state intervals.

2. The remote medical decision-making assistance method based on AI big model according to claim 1 is characterized in that: The step of obtaining first multimodal data information and second multimodal data information of the patient, wherein the first multimodal data information and the second multimodal data information include text data information, image data information, and voice data information.

3. The remote medical decision-making assistance method based on AI big model according to claim 1 is characterized in that: The first multimodal data information and the second multimodal data information of the patient are obtained, wherein the first multimodal data information is historical multimodal data information, and the second multimodal data information is real-time multimodal data information.

4. The remote medical decision-making assistance method based on AI big model according to claim 1 is characterized in that: The multimodal fusion model using the attention mechanism performs feature fusion on the first multimodal data information and the second multimodal data information respectively to obtain a first data result and a second data result, wherein the first data result is a data result of historical multimodal data information, and the second data result is a data result of real-time multimodal data information.

5. The remote medical decision-making assistance method based on AI big model according to claim 2 is characterized in that: The multimodal fusion model using the attention mechanism performs feature fusion on the first multimodal data information and the second multimodal data information respectively to obtain a first data result and a second data result, including: Step S201, performing feature extraction on each modal data in the first multimodal data information and the second multimodal data information respectively to obtain a feature representation of each modal data; Step S202, obtaining the attention weight of each modal data through an attention weight calculation module for the feature representation of each modal data; Step S203, multiplying the attention weight of each modal data by the feature of each modal data to obtain a weighted feature representation of each modal data; Step S204, concatenating and summing the weighted feature representations of each modal data to obtain final multimodal feature representations of the first multimodal data information and the second multimodal data information, that is, a first data result and a second data result.

6. The remote medical decision-making assistance method based on AI big model according to claim 1 is characterized in that: The obtaining the difference between the first data result and the second data result based on the joint latent variable model, and taking the difference between the first data result and the second data result as the third data result, includes: Step S301, jointly modeling the first multimodal data information and the second multimodal data information to obtain shared latent variables of the first data result and the second data result; Step S302, obtaining representations of the first data result and the second data result in a shared latent variable by maximizing the joint likelihood learning latent variable model of the data; Step S303: compare the representations of the first data result and the second data result in the shared latent variable to obtain the difference between the first data result and the second data result.

7. A remote medical decision-making assistance system based on an AI large model, characterized in that: include: A multimodal data information acquisition module, used to acquire first multimodal data information and second multimodal data information of a patient; A multimodal feature fusion module, configured to use a multimodal fusion model with an attention mechanism to perform feature fusion on the first multimodal data information and the second multimodal data information respectively, to obtain a first data result and a second data result; A third data result acquisition module, configured to acquire the difference between the first data result and the second data result based on a joint latent variable model, and use the difference between the first data result and the second data result as a third data result; A comprehensive analysis module, used to perform a comprehensive analysis on the first data result, the second data result and the third data result based on the AI ​​big model, and obtain the influencing factors of the third data result as the fourth data result; A state information value acquisition module, used to classify the third data result using a binary classifier, obtain the classification level of the third data result, and obtain the current state information value of the patient based on the classification level; The status information value judgment module is used to obtain the patient's status interval based on the status information value and the fourth data result, and enable different remote medical auxiliary decision-making based on the patient's different status intervals.

8. An electronic device, characterized in that: include: one or more processors; Memory; And one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the device, enable the device to perform the steps of the remote medical decision-making assistance method based on the AI ​​large model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the steps of the remote medical decision-making assistance method based on an AI large model as described in any one of claims 1 to 6.

Images