Multi-modal information integration and rebroadcasting meta universe medical digital space construction method

Abstract

The invention relates to the technical field of medical information, and discloses a multi-modal information integration and rebroadcasting meta-universe medical digital space construction method, which comprises the following steps of S1, determining specific requirements and targets of medical application, including data integration and rebroadcasting requirements of an operating room, an I CU, a ward and the like; s2, making a digital transformation plan, and determining the type of multi-modal information needing to be integrated; real-time acquisition and processing of multi-modal information are realized by defining medical requirements, formulating a digital transformation plan and installing equipment and sensors with high compatibility, and data from different equipment and sensors are seamlessly integrated into a unified platform by using a standardized data format and a communication protocol, so that the consistency and the real-time performance of the data are ensured, and meanwhile, the real-time performance of the system is improved. And through Fourier transform and other data processing algorithms, key features are extracted for data cleaning, so that the efficiency and accuracy of data processing are improved, and the integration and processing efficiency of medical data is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical information technology, and specifically to a method for constructing a metaverse medical digital space for multimodal information integration and relaying. Background Art

[0002] With the development of modern medical technology, medical institutions generate and process multimodal information in their daily operations, such as medical images, physiological data, laboratory data, electronic medical records, etc. The data is scattered across different devices and systems, lacking unified integration and management, resulting in serious data silo problems, low data processing efficiency, and inability to achieve effective information sharing and real-time decision support.

[0003] The existing technologies have the following deficiencies in the integration and processing of multimodal information:

[0004] The data formats and communication protocols between different devices and systems are not unified, which makes it impossible to achieve seamless data integration, resulting in the inability to ensure data consistency and real-time performance, affecting the overall efficiency of data utilization;

[0005] Due to the lack of effective data processing and analysis tools, existing technologies have obvious deficiencies in real-time processing and analysis of multimodal information. Delays and inaccuracies in data processing affect the timeliness and accuracy of medical decision-making.

[0006] Existing technologies have limitations in the visualization of medical data and are unable to provide highly visualized three-dimensional digital scenes, making it impossible for medical personnel to intuitively understand and operate complex medical data, limiting the application value of the data.

[0007] Therefore, those skilled in the art provide a method for constructing a metaverse medical digital space for multimodal information integration and broadcasting to solve the problems raised in the above-mentioned background technology. Summary of the invention

[0008] In view of the shortcomings of the prior art, the present invention provides a method for constructing a metaverse medical digital space for multimodal information integration and relaying, so as to solve the problem that data in modern medical technology is scattered across different devices and systems, lacks unified integration and management, leads to serious data island problems, low data processing efficiency, and cannot achieve effective information sharing and real-time decision support.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for constructing a metaverse medical digital space for multimodal information integration and relaying, comprising the following steps: S1, determining specific requirements and goals of medical applications, including data integration and relaying requirements of operating rooms, I CUs, wards, etc.;

[0010] S2. Develop a digital transformation plan and identify the types of multimodal information that need to be integrated;

[0011] S3. Install compatible and stable devices and sensors to ensure full compatibility between the devices and the cloud platform and computing power platform;

[0012] S4, real-time collection of multimodal information;

[0013] S5. Process and analyze the collected data in real time, extract key features, and perform data cleaning;

[0014] S6. Integrate data from various devices and sensors into a unified platform to ensure data consistency and real-time performance;

[0015] S7. Ensure interoperability and seamless data integration between devices through standardized data formats and communication protocols;

[0016] S8. Based on 3D digital modeling technology, the collected data is converted into a highly visualized metaverse scene;

[0017] S9. Establish a virtual mirror of the medical environment corresponding to the real world;

[0018] S10. Provide multi-point and multi-domain mobile login support, allowing medical personnel to access and interact with the Metaverse medical space in real time at different locations;

[0019] S11. Support real-time doctor-patient communication, remote diagnosis and surgical guidance, and realize remote medical services through the efficient support of cloud communication and computing power platform.

[0020] S1 clarifies the specific needs of different medical scenarios, formulates targeted integration plans, and improves integration efficiency; S2 ensures system compatibility and stability, identifies system bottlenecks, and formulates detailed implementation plans and risk response measures; S4 and S5 ensure the real-time and accuracy of data and improve data processing efficiency; S8 and S9 provide intuitive three-dimensional visualization scenes to facilitate medical personnel to operate and drill, and improve the operability and practicality of data; S10 and S11 support multi-point and multi-domain mobile login, combined with cloud communication and computing power platforms, to achieve real-time doctor-patient communication, remote diagnosis and surgical guidance, improve the coverage and efficiency of telemedicine services, ensure priority transmission of important data, and improve the quality of telemedicine services.

[0021] Preferably, in the step S2, the steps of formulating the digital transformation plan are as follows: S21, discussing with relevant departments of the medical institution to determine the type of multimodal information to be integrated;

[0022] S22. Evaluate the compatibility, stability and connectivity of devices and systems with cloud platforms and computing platforms to identify system bottlenecks;

[0023] S23. Determine the types of data to be integrated and develop data standards and formats;

[0024] S24. Determine the communication protocols and data formats used to ensure interoperability between devices and seamless integration of data;

[0025] S25. Develop an implementation schedule, clarify the tasks and time nodes of each stage, and arrange training for relevant personnel to ensure that the trained personnel can operate the equipment and system proficiently;

[0026] S26. Identify risks, develop appropriate response measures, and ensure that there is a comprehensive backup and recovery plan to deal with data loss or system failure;

[0027] S27. Carry out comprehensive testing before actual application to ensure the stability and reliability of the system;

[0028] S28. Collect user feedback and make corresponding adjustments based on the feedback.

[0029] S21 Ensure the comprehensiveness and accuracy of demand analysis, and formulate more targeted and effective integration solutions to meet the specific needs of different departments and scenarios; S22 Identify system bottlenecks in advance, ensure the compatibility and stability of equipment and platforms, and improve the overall performance and reliability of the system; S23 Ensure the unification of data formats and communication protocols, improve the interoperability between devices, achieve seamless integration of data, and improve data consistency and real-time performance; S24 Through standardized communication protocols and data formats, achieve seamless integration of data from different devices and systems, and improve the coordination of the overall system and the continuity of data flow; S25 Ensure the orderly implementation of the project, improve the proficiency of personnel in the operation of equipment and systems, and reduce problems and delays in the implementation process; S26 Improve the security and reliability of the system, ensure rapid recovery in the event of data loss or system failure, and reduce the impact on medical services; S27 Through sufficient testing, discover and solve potential problems, ensure the stable operation of the system in actual applications, and improve the reliability of the system; S28 Discover problems and improve the system in a timely manner through user feedback to ensure that the system better meets user needs and improve user satisfaction and system practicality.

[0030] Preferably, the digital transformation plan is based on data collection, and its function expression is: D(t) = {d1(t), d2(t), ..., d n (t)}, where D(t) represents the data set collected at time point t, d n (t) represents the data collected by different devices or sensors at time point t.

[0031] Preferably, in step S4, the multimodal information includes imaging data, physiological data, laboratory data, medical record data, patient monitoring data, equipment data, and environmental data;

[0032] The image data: medical images, ultrasound images, endoscopic images, digital pathology images;

[0033] The physiological data: electrocardiogram, electroencephalogram, blood oxygen saturation, blood pressure, respiratory rate, body temperature;

[0034] The laboratory data: blood test results, urine analysis results, pathological test results;

[0035] The medical record data: electronic medical records, surgical records, diagnosis reports, treatment plans;

[0036] The patient monitoring data: ICU monitoring data, intraoperative monitoring data;

[0037] The equipment data: medical equipment operating status and parameters, equipment alarm and fault information;

[0038] The environmental data include: temperature, humidity, cleanliness, air quality, and light intensity.

[0039] Image data: By integrating multiple image data, it provides more comprehensive image information of the disease, assists doctors to make more accurate diagnoses, and improves the accuracy and comprehensiveness of diagnosis; Physiological data: Real-time collection and analysis of multiple physiological data can comprehensively monitor the patient's physiological state, provide important basis for clinical decision-making, and improve the effect of patient monitoring and treatment; Laboratory data: By integrating laboratory data, the patient's test results can be quickly obtained and analyzed to assist clinical diagnosis and treatment, shorten the diagnosis and treatment time, and improve the efficiency of diagnosis and treatment; Medical record data: Comprehensive medical record data integration facilitates doctors to quickly review the patient's medical history and treatment plan, improve the continuity and personalization of diagnosis and treatment, and reduce misdiagnosis and missed diagnosis; Patient monitoring data: Real-time collection and monitoring of patient monitoring data can timely detect abnormal conditions of patients, provide timely intervention measures, and improve patient safety and treatment effects; Equipment data: By real-time monitoring of equipment data, the normal operation of medical equipment is ensured, equipment failures are handled in a timely manner, and the smooth progress of the diagnosis and treatment process is guaranteed; Environmental data: By monitoring medical environment data, medical environment conditions can be optimized, the safety and comfort of the diagnosis and treatment process can be improved, and the impact of the environment on the treatment effect of patients can be reduced.

[0040] Preferably, in the step S5, the key features of the collected data are extracted based on Fourier transform, which is used to convert the time domain signal into the frequency domain and extract the frequency features of the signal. The Fourier transform formula is: Among them, X(f) is the frequency domain signal, x(n) is the time domain signal, N is the number of sampling points of the signal, and f is the frequency.

[0041] Preferably, in the step S8, the specific implementation steps of converting the collected data into a highly visualized metaverse scene based on three-dimensional digital modeling technology are as follows: S81, collecting raw data from various sensors and devices, cleaning the data to remove noise and errors, and correcting the data;

[0042] S82, converting the data into a format that can be processed by the 3D modeling software, and extracting the region by an image segmentation algorithm;

[0043] S83. Convert volume data into polygonal meshes using the Marching Cubes algorithm or other mesh generation techniques;

[0044] S84, remove the noise and irregularities on the model surface through the smoothing algorithm, the smoothing algorithm formula is: V i ′=v i +λ∑ j∈N(i) (v j -V i ), where v i ′ is the vertex position after smoothing, V i is the original vertex position, N(i) is the neighboring vertex of vertex i, and λ is the smoothing factor;

[0045] S85. Reduce the number of polygons in the mesh. The formula for mesh simplification is:

[0046] Among them, v i ′ is the simplified vertex position, V i is the original vertex position;

[0047] S86, mapping the collected image data to the surface of the three-dimensional model, then combining the three-dimensional model into the virtual scene, and adding scene elements;

[0048] S87. Use ray tracing, rasterization and other technologies to render the scene, generate an image, and then display the rendered result on the screen or present it through a VR / AR device.

[0049] S81 removes noise and errors through effective data cleaning and correction, improves data accuracy, and provides a reliable data foundation for subsequent 3D modeling; S82 ensures the compatibility of data formats, uses efficient image segmentation algorithms to accurately extract required areas, and improves modeling accuracy and efficiency; S83 quickly generates high-quality polygon meshes through the Marching Cubes algorithm, improves modeling efficiency, and ensures the fineness and coherence of the model; S84 uses smoothing algorithms to effectively remove noise and irregularities on the model surface, optimizes the model surface quality, and makes it smoother and more accurate; S85 uses mesh simplification technology to reduce the number of polygons, optimize the complexity of the model, improve rendering and display efficiency, ensure model accuracy, and reduce the computing burden; S86 realizes accurate mapping of image data and 3D models, constructs highly realistic virtual scenes, provides intuitive visual references for medical personnel, and improves data visualization effects; S87 uses advanced rendering technology to generate high-quality images and presents them through screens or VR / AR devices, providing users with an immersive visual experience and enhancing the interactivity and operability of data.

[0050] Preferably, in the step S9, the specific implementation steps of establishing a virtual image of the medical environment are as follows: S91, data collection and analysis: real-time data from medical environments such as operating rooms, ICUs, and wards are collected through sensors and monitoring equipment, and the collected data is cleaned, corrected, and analyzed to ensure the accuracy and consistency of the data;

[0051] S92. Create a virtual environment model: Based on the architectural plan and layout of the actual medical environment, construct a structural model of the virtual environment through 3D modeling software, and add detailed 3D models of medical equipment, furniture, medical staff and patients in the virtual environment;

[0052] S93, dynamic update of virtual environment: integrating real-time collected environmental data into the virtual environment, reflecting real-time changes in the environment, and dynamically adjusting environmental parameters such as lighting, temperature, and humidity in the virtual environment based on real-time data;

[0053] S94. Interaction design of virtual environment: Design user interface and interaction mode to enable medical personnel to quickly browse and operate the virtual environment, simulate actual operation and drill in the virtual environment, and verify the accuracy and practicality of the virtual environment;

[0054] S95. Integrate virtual and real environments: Establish a data feedback loop to feed back operations and changes in the virtual environment to the real environment to provide decision support.

[0055] S91 collects medical environment data comprehensively and in real time to ensure the accuracy and consistency of the data, providing a reliable data foundation for subsequent virtual environment modeling; S92 builds an accurate virtual environment model to truly reflect the layout and equipment conditions of the actual medical environment, provide highly simulated virtual scenes, and improve the practicality and accuracy of the virtual environment; S93 realizes dynamic updates of the virtual environment, reflects environmental changes in real time, provides dynamic response capabilities, and ensures the synchronization and real-time nature of the virtual environment and the actual environment; S94 provides an intuitive and easy-to-use user interface through perfect interactive design, enabling medical personnel to efficiently operate the virtual environment, conduct simulated operations and drills, and improve the operating experience and practicality of the virtual environment; S95 realizes the close integration of the virtual environment and the real environment, and applies the results of virtual operations to the real environment through a data feedback mechanism, providing effective decision support and improving the accuracy and efficiency of medical decision-making.

[0056] Preferably, in the step S10, based on network bandwidth management and a bandwidth allocation algorithm, priority transmission of important data is ensured, and its function expression is: Among them, B i is the bandwidth allocation for class i data, W i is the importance weight of the i-type data, B total is the total bandwidth.

[0057] Preferably, in the step S11, based on medical image processing technology and machine learning model prediction technology;

[0058] The medical image processing technology formula is: Among them, F i,j，k is the feature map, I i，j is the input image, K m，n，k is the convolution kernel;

[0059] The machine learning model prediction technology formula is: Among them, α i is the Lagrange multiplier, y i is the label, K(x i , x) is the kernel function, and b is the bias.

[0060] The present invention provides a method for constructing a metaverse medical digital space for multimodal information integration and relaying, which has the following beneficial effects:

[0061] 1. The present invention clarifies medical needs, develops a digital transformation plan, installs highly compatible equipment and sensors, realizes real-time collection and processing of multimodal information, and uses standardized data formats and communication protocols to seamlessly integrate data from different devices and sensors into a unified platform to ensure data consistency and real-time performance. At the same time, through Fourier transform and other data processing algorithms, key features are extracted for data cleaning, thereby improving the efficiency and accuracy of data processing, thereby improving the integration and processing efficiency of medical data.

[0062] 2. Based on three-dimensional digital modeling technology and virtual mirror technology, the present invention converts the collected data into a highly visualized metaverse scene, establishes a virtual environment corresponding to the real medical environment, and dynamically updates various parameters in the virtual environment and designs the user interface to enable medical personnel to quickly browse and operate the virtual environment, simulate actual operations and drills. In addition, it provides multi-point and multi-domain mobile login support, combines cloud communication and computing power platforms, realizes real-time doctor-patient communication, remote diagnosis and surgical guidance, and significantly improves the quality and efficiency of telemedicine services. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0064] In order to make the technical personnel in the technical field understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be clearly and completely described below in combination with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a partial embodiment of the present invention, not a complete embodiment. Based on the embodiment of the present invention, other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0065] The present invention is described in detail below in conjunction with the accompanying drawings:

[0066] Embodiment 1:

[0067] Please refer to the attached Figure 1 ,The embodiment of the present invention provides a method for constructing a metaverse medical digital space for multimodal information integration and relay, including the following steps: S1, discussing with the operating room and I CU team to determine their data integration and relay requirements;

[0068] S21. Discuss with the operating room and ICU related departments to determine the imaging data, physiological data and patient monitoring data to be integrated;

[0069] S22. Evaluate the compatibility and stability of existing equipment and check its connectivity with the cloud platform;

[0070] S23. Determine the type of data to be integrated and develop data standards and formats;

[0071] S24. Ensure interoperability between devices through communication protocols and data formats;

[0072] S25. Develop an implementation schedule and arrange training for relevant personnel;

[0073] S26. Identify risks and develop backup and recovery plans;

[0074] S27. Conduct comprehensive testing before actual application;

[0075] S28. Collect user feedback and make adjustments;

[0076] S3. Install compatible and stable equipment and sensors in the operating room and I CU to ensure full compatibility with the cloud platform and computing platform;

[0077] S4, collect imaging data, physiological data and patient monitoring data (I CU monitoring data);

[0078] S5. Clean, correct and analyze the collected data, extract key features and perform data cleaning;

[0079] S6. Integrate the data of various devices and sensors into a unified platform to ensure data consistency and real-time performance;

[0080] S7. Use standardized data formats and communication protocols to ensure interoperability between devices and seamless data integration;

[0081] S81. Collect and clean the raw data;

[0082] S82, converting the data into a three-dimensional modeling format and extracting an image region;

[0083] S83. Generate polygonal mesh using Marching Cubes algorithm;

[0084] S84, applying a smoothing algorithm to remove noise on the surface of the model;

[0085] S85, perform mesh simplification;

[0086] S86, mapping the image data to the surface of the three-dimensional model and combining it into a virtual scene;

[0087] S87. Use ray tracing and other techniques to render scenes and display them on screen or through VR devices;

[0088] S91, collect environmental data of operating room and I CU in real time, and perform cleaning and analysis;

[0089] S92. Build a virtual environment model based on the actual environment and add detailed equipment and personnel models;

[0090] S93, dynamically updating parameters such as lighting and temperature in the virtual environment;

[0091] S94. Design interactive interfaces to enable medical personnel to simulate operations and drills;

[0092] S95. Establish a data feedback loop to feed back the virtual operation results to the real environment;

[0093] S10, provides multi-point and multi-domain mobile login support, combined with cloud communication and computing power platform, to support real-time doctor-patient communication, remote diagnosis and surgical guidance.

[0094] Summary: Example 1 realizes multimodal information integration and relay of operating rooms and I CU through meticulous step planning and comprehensive data integration, and establishes a highly visualized virtual medical environment. This example improves data processing efficiency, visualization effects, and the quality of telemedicine services, and provides a powerful auxiliary tool for medical personnel.

[0095] Embodiment 2:

[0096] Please see attached Figure 1 ,The embodiment of the present invention provides a method for constructing a metaverse medical digital space for multimodal information integration and relay, including the following steps: S1, discussing with the ward and laboratory teams to determine data integration and relay requirements;

[0097] S21. Discuss with relevant departments of wards and laboratories to determine the laboratory data, medical record data and environmental data to be integrated;

[0098] S22. Evaluate the compatibility and stability of existing equipment and check its connectivity with the cloud platform;

[0099] S23. Determine the type of data to be integrated and develop data standards and formats;

[0100] S24, communication protocols and data formats to ensure interoperability between devices;

[0101] S25. Develop an implementation schedule and arrange training for relevant personnel;

[0102] S26. Identify risks and develop backup and recovery plans;

[0103] S27. Conduct comprehensive testing before actual application;

[0104] S28. Collect user feedback and make adjustments;

[0105] S3. Install compatible and stable equipment and sensors in wards and laboratories to ensure full compatibility with cloud platforms and computing platforms;

[0106] S4, collect laboratory data, medical record data and environmental data;

[0107] S5. Clean, correct and analyze the collected data, extract key features and perform data cleaning;

[0108] S6. Integrate the data of various devices and sensors into a unified platform to ensure data consistency and real-time performance;

[0109] S7. Use standardized data formats and communication protocols to ensure interoperability between devices and seamless data integration;

[0110] S81. Collect and clean the raw data;

[0111] S82, converting the data into a three-dimensional modeling format and extracting an image region;

[0112] S83. Generate polygonal mesh using Marching Cubes algorithm;

[0113] S84, applying a smoothing algorithm to remove noise on the surface of the model;

[0114] S85, perform mesh simplification;

[0115] S86, mapping the image data to the surface of the three-dimensional model and combining it into a virtual scene;

[0116] S87. Use ray tracing and other techniques to render scenes and display them on screen or through VR devices;

[0117] S91, collect environmental data of wards and laboratories in real time, and perform cleaning and analysis;

[0118] S92. Build a virtual environment model based on the actual environment and add detailed equipment and personnel models;

[0119] S93, dynamically updating parameters such as lighting and temperature in the virtual environment;

[0120] S94. Design interactive interfaces to enable medical personnel to simulate operations and drills;

[0121] S95. Establish a data feedback loop to feed back the virtual operation results to the real environment;

[0122] S10, provides multi-point and multi-domain mobile login support, combined with cloud communication and computing power platform, to support real-time doctor-patient communication, remote diagnosis and surgical guidance.

[0123] Summary: Example 2 successfully realizes multimodal information integration and relay of wards and laboratories through comprehensive data integration and detailed virtual environment construction, and establishes a highly visualized virtual medical environment. This example improves the efficiency of laboratory and ward data processing, enhances data visualization and the quality of telemedicine services, and provides powerful support and auxiliary tools for medical personnel.

[0124] Embodiment three:

[0125] Please refer to the attached Figure 1 The embodiment of the present invention provides a method for constructing a metaverse medical digital space for multimodal information integration and relaying, comprising the following steps:

[0126] S1. Discuss with the emergency department and radiology teams to determine data integration and broadcasting requirements;

[0127] S21. Discuss with the relevant departments of the emergency department and the imaging department to determine the imaging data, real-time monitoring data and medical record data to be integrated;

[0128] S22. Evaluate the compatibility and stability of existing equipment and check its connectivity with the cloud platform;

[0129] S23. Determine the type of data to be integrated and develop data standards and formats;

[0130] S24, communication protocols and data formats to ensure interoperability between devices;

[0131] S25. Develop an implementation schedule and arrange training for relevant personnel;

[0132] S26. Identify risks and develop backup and recovery plans;

[0133] S27. Conduct comprehensive testing before actual application;

[0134] S28. Collect user feedback and make adjustments;

[0135] S3. Install compatible and stable equipment and sensors in the emergency department and imaging department to ensure full compatibility with the cloud platform and computing platform;

[0136] S4, collect imaging data, real-time monitoring data and medical record data;

[0137] S5. Clean, correct and analyze the collected data, extract key features and perform data cleaning;

[0138] S6. Integrate the data of various devices and sensors into a unified platform to ensure data consistency and real-time performance;

[0139] S7. Use standardized data formats and communication protocols to ensure interoperability between devices and seamless data integration;

[0140] S81. Collect and clean the raw data;

[0141] S82, converting the data into a three-dimensional modeling format and extracting an image region;

[0142] S83. Generate polygonal mesh using Marching Cubes algorithm;

[0143] S84, applying a smoothing algorithm to remove noise on the surface of the model;

[0144] S85, perform mesh simplification;

[0145] S86, mapping the image data to the surface of the three-dimensional model and combining it into a virtual scene;

[0146] S87. Use ray tracing and other techniques to render scenes and display them on screen or through VR devices;

[0147] S91, collect environmental data from the emergency department and imaging department in real time, and perform cleaning and analysis;

[0148] S92. Build a virtual environment model based on the actual environment and add detailed equipment and personnel models;

[0149] S93, dynamically updating parameters such as lighting and temperature in the virtual environment;

[0150] S94. Design interactive interfaces to enable medical personnel to simulate operations and drills;

[0151] S95. Establish a data feedback loop to feed back the virtual operation results to the real environment;

[0152] S10, providing multi-point and multi-domain mobile login support, combined with cloud communication and computing power platform, supporting real-time doctor-patient communication, remote diagnosis and surgical guidance;

[0153] Summary: Example 3 demonstrates data integration and relay between the emergency department and the imaging department, combines imaging data and real-time monitoring data to create a highly visualized virtual emergency and imaging environment. This example improves the efficiency of emergency treatment and the accuracy of imaging diagnosis, and provides strong support for medical personnel in the emergency and imaging departments.

[0154] Embodiment 4:

[0155] Please see attached Figure 1 ,The embodiment of the present invention provides a method for constructing a metaverse medical digital space for multimodal information integration and relay, including the following steps: S1, discussing with the pharmacy and clinical department teams to determine data integration and relay requirements;

[0156] S21. Discuss with relevant departments of the pharmacy and clinical departments to determine the drug inventory data, patient medication records and clinical treatment data to be integrated;

[0157] S22. Evaluate the compatibility and stability of existing equipment and check its connectivity with the cloud platform;

[0158] S23. Determine the type of data to be integrated and develop data standards and formats;

[0159] S24. Ensure interoperability between devices through communication protocols and data formats;

[0160] S25. Develop an implementation schedule and arrange training for relevant personnel;

[0161] S26. Identify risks and develop backup and recovery plans;

[0162] S27. Conduct comprehensive testing before actual application;

[0163] S28. Collect user feedback and make adjustments;

[0164] S3. Install compatible and stable equipment and sensors in pharmacies and clinical departments to ensure full compatibility with cloud platforms and computing platforms;

[0165] S4, collect drug inventory data, patient medication records and clinical treatment data;

[0166] S5. Clean, correct and analyze the collected data, extract key features and perform data cleaning;

[0167] S6. Integrate the data of various devices and sensors into a unified platform to ensure data consistency and real-time performance;

[0168] S7. Use standardized data formats and communication protocols to ensure interoperability between devices and seamless data integration;

[0169] S81. Collect and clean the raw data;

[0170] S82, converting the data into a three-dimensional modeling format and extracting an image region;

[0171] S83. Generate polygonal mesh using Marching Cubes algorithm;

[0172] S84, applying a smoothing algorithm to remove noise on the surface of the model;

[0173] S85, perform mesh simplification;

[0174] S86, mapping the image data to the surface of the three-dimensional model and combining it into a virtual scene;

[0175] S87. Use ray tracing and other techniques to render scenes and display them on screen or through VR devices;

[0176] S91, collect environmental data from pharmacies and clinical departments in real time, and perform cleaning and analysis;

[0177] S92. Build a virtual environment model based on the actual environment and add detailed equipment and personnel models;

[0178] S93, dynamically updating parameters such as lighting and temperature in the virtual environment;

[0179] S94. Design interactive interfaces to enable medical personnel to simulate operations and drills;

[0180] S95. Establish a data feedback loop to feed back the virtual operation results to the real environment;

[0181] S10, provides multi-point and multi-domain mobile login support, combined with cloud communication and computing power platform, to support real-time doctor-patient communication, remote diagnosis and surgical guidance.

[0182] Summary: Example 4 realizes multimodal information integration and broadcasting of pharmacies and clinical departments through comprehensive data integration and detailed virtual environment construction, and creates a highly visual virtual pharmacy and clinical environment. This example improves the efficiency of pharmacy management and the accuracy of clinical treatment, and provides a powerful support tool for medical staff in pharmacies and clinical departments.

[0183] Experimental design:

[0184] Comparative experiment 1: data processing efficiency comparison;

[0185] Experimental purpose: To compare the data processing efficiency of the metaverse medical digital space construction method (method A) for multimodal information integration and broadcasting and the traditional data processing method (method B);

[0186] Experimental parameters:

[0187] Data types: imaging data, physiological data, medical record data, environmental data;

[0188] Data volume: 1000 records of each data type;

[0189] Processing time (seconds);

[0190] Experimental steps:

[0191] Collect the same data set;

[0192] Use method A and method B to process the data;

[0193] Record the data processing time for various data types.

[0194] Experimental data table:

[0195] Data Types Data volume (items) Method A processing time (S) Method A processing time (S) Image data 1000 120 200 Physiological data 1000 100 180 Medical record data 1000 80 150 Environmental data 1000 60 100 Total time 4000 360 630

[0196] Comparative experiment 2: data consistency and real-time comparison;

[0197] Experimental purpose: To compare the performance of the metaverse medical digital space construction method (method A) for multimodal information integration and broadcasting with the traditional data processing method (method B) in terms of data consistency and real-time performance;

[0198] Experimental parameters:

[0199] Data types: imaging data, physiological data, medical record data, environmental data;

[0200] Experimental steps:

[0201] Collect the same data set;

[0202] Integrate and process the data using Method A and Method B;

[0203] Document consistency and processing times for various data types.

[0204] Experimental data table:

[0205] Data Types Data volume Method A consistency Method B consistency Method A: Real-time Method B: Real-time Image data 1000 items 98％ 90％ 1.2s 2.0s Physiological data 1000 items 99％ 92％ 1.0s 1.8s Medical record data 1000 items 97％ 89％ 0.8s 1.5s Environmental data 1000 items 99％ 91％ 0.6s 1.0s average value 4000 98.25％ 90.5％ 0.9s 1.575s

[0206] Comparative experiment 3: Comparison of telemedicine service quality;

[0207] Experimental purpose: To compare the performance of the metaverse medical digital space construction method of multimodal information integration and relay (method A) and the traditional telemedicine method (method B) in terms of telemedicine service quality;

[0208] Experimental parameters:

[0209] Data types: imaging data, physiological data, medical record data, environmental data;

[0210] Experimental steps:

[0211] Collect the same data set;

[0212] Use Method A and Method B for telemedicine services;

[0213] Record telesurgery response time and success rate.

[0214] Experimental data table:

[0215]

[0216]

[0217] Experimental summary:

[0218] Comparative experiment 1: Method A significantly improves the processing efficiency of various types of data, and the processing time is less than that of method B, with the total time reduced by about 43%.

[0219] Comparative experiment 2: Method A performs better in terms of data consistency and real-time performance. The average data consistency is 7.75% higher and the real-time performance is improved by about 43%.

[0220] Comparative experiment three: Method A is superior to Method B in terms of remote diagnosis accuracy, surgical guidance response time and doctor-patient communication success rate. The response time is shortened by about 66% and the communication success rate is increased by about 9%.

[0221] The experimental results show that the metaverse medical digital space construction method of multimodal information integration and broadcasting has obvious advantages in data processing efficiency, consistency, real-time and telemedicine service quality.

[0222] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a metaverse medical digital space for multimodal information integration and transmission, characterized in that: The steps include: S1. Determine the specific needs and goals of medical applications, including data integration and broadcasting requirements in operating rooms, ICUs, wards, etc.; S2. Develop a digital transformation plan and identify the types of multimodal information that need to be integrated; S3. Install compatible and stable devices and sensors to ensure full compatibility between the devices and the cloud platform and computing power platform; S4, real-time collection of multimodal information; S5. Process and analyze the collected data in real time, extract key features, and perform data cleaning; S6. Integrate data from various devices and sensors into a unified platform to ensure data consistency and real-time performance; S7. Ensure interoperability and seamless data integration between devices through standardized data formats and communication protocols; S8. Based on 3D digital modeling technology, the collected data is converted into a highly visualized metaverse scene; S9. Establish a virtual mirror of the medical environment corresponding to the real world; S10. Provide multi-point and multi-domain mobile login support, allowing medical personnel to access and interact with the Metaverse medical space in real time at different locations; S11. Support real-time doctor-patient communication, remote diagnosis and surgical guidance, and realize remote medical services through the efficient support of cloud communication and computing power platform.

2. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: In step S2, the steps for formulating a digital transformation plan are as follows: S21. Discuss with relevant departments of medical institutions to determine the types of multimodal information to be integrated; S22. Evaluate the compatibility, stability and connectivity of devices and systems with cloud platforms and computing platforms to identify system bottlenecks; S23. Determine the types of data to be integrated and develop data standards and formats; S24. Determine the communication protocols and data formats used to ensure interoperability between devices and seamless integration of data; S25. Develop an implementation schedule, clarify the tasks and time nodes of each stage, and arrange training for relevant personnel to ensure that the trained personnel can operate the equipment and system proficiently; S26. Identify risks, develop appropriate response measures, and ensure that there is a comprehensive backup and recovery plan to deal with data loss or system failure; S27. Carry out comprehensive testing before actual application to ensure the stability and reliability of the system; S28. Collect user feedback and make corresponding adjustments based on the feedback.

3. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: The digital transformation plan is based on data collection, and its function expression is: D(t) = {d1(t), d2(t), ..., d n (t)}, where D(t) represents the data set collected at time point t, d n (t) represents the data collected by different devices or sensors at time point t.

4. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: In step S4, the multimodal information includes imaging data, physiological data, laboratory data, medical record data, patient monitoring data, equipment data, and environmental data; The image data: medical images, ultrasound images, endoscopic images, digital pathology images; The physiological data: electrocardiogram, electroencephalogram, blood oxygen saturation, blood pressure, respiratory rate, body temperature; The laboratory data: blood test results, urine analysis results, pathological test results; The medical record data: electronic medical records, surgical records, diagnosis reports, treatment plans; The patient monitoring data: ICU monitoring data, intraoperative monitoring data; The equipment data: medical equipment operating status and parameters, equipment alarm and fault information; The environmental data include: temperature, humidity, cleanliness, air quality, and light intensity.

5. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: In the step S5, the key features of the collected data are extracted based on Fourier transform, which is used to convert the time domain signal into the frequency domain and extract the frequency features of the signal. The Fourier transform formula is: Among them, X(f) is the frequency domain signal, x(n) is the time domain signal, N is the number of sampling points of the signal, and f is the frequency.

6. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: In the step S8, based on the three-dimensional digital modeling technology, the specific implementation steps of converting the collected data into a highly visualized metaverse scene are as follows: S81, collecting raw data from various sensors and devices, cleaning the data to remove noise and errors, and correcting the data; S82, converting the data into a format that can be processed by the 3D modeling software, and extracting the region by an image segmentation algorithm; S83. Convert volume data into polygonal meshes using the Marching Cubes algorithm or other mesh generation techniques; S84, remove the noise and irregularities on the model surface through the smoothing algorithm, the smoothing algorithm formula is: i ′=v i +λ∑ j∈N(i) (v j -v i ), where v i ′ is the vertex position after smoothing, v i is the original vertex position, N(i) is the neighboring vertex of vertex i, and λ is the smoothing factor; S85. Reduce the number of polygons in the mesh. The formula for mesh simplification is: Among them, v i ′ is the simplified vertex position, v i is the original vertex position; S86, mapping the collected image data to the surface of the three-dimensional model, then combining the three-dimensional model into the virtual scene, and adding scene elements; S87. Use ray tracing, rasterization and other technologies to render the scene, generate an image, and then display the rendering result on the screen or present it through a VR / AR device.

7. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: In the step S9, the specific implementation steps of establishing a virtual image of the medical environment are as follows: S91, data collection and analysis: using sensors and monitoring equipment to collect data in medical environments such as operating rooms, ICUs, and wards in real time, and cleaning, correcting, and analyzing the collected data to ensure the accuracy and consistency of the data; S92. Create a virtual environment model: Based on the architectural plan and layout of the actual medical environment, construct a structural model of the virtual environment through 3D modeling software, and add detailed 3D models of medical equipment, furniture, medical staff and patients in the virtual environment; S93, dynamic update of virtual environment: integrating real-time collected environmental data into the virtual environment, reflecting real-time changes in the environment, and dynamically adjusting environmental parameters such as lighting, temperature, and humidity in the virtual environment based on real-time data; S94. Interaction design of virtual environment: Design user interface and interaction mode to enable medical personnel to quickly browse and operate the virtual environment, simulate actual operation and drill in the virtual environment, and verify the accuracy and practicality of the virtual environment; S95. Integrate virtual and real environments: Establish a data feedback loop to feed back operations and changes in the virtual environment to the real environment to provide decision support.

8. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: In the step S10, based on the network bandwidth management and the bandwidth allocation algorithm, priority transmission of important data is ensured, and its function expression is: Among them, B i is the bandwidth allocation for class i data, W i is the importance weight of the i-type data, B total is the total bandwidth.

9. The method for constructing a metaverse medical digital space for multimodal information integration and relaying according to claim 1, characterized in that: In the step S11, based on medical image processing technology and machine learning model prediction technology; The medical image processing technology formula is: Among them, F i,j,k is the feature map, I i,j is the input image, K m,n,k is the convolution kernel; The machine learning model prediction technology formula is: Among them, α i is the Lagrange multiplier, y i is the label, K(x i ,x) is the kernel function and b is the bias.

Images