Remote consultation and live broadcast director method and system for multi-source medical images

Abstract

The invention discloses a remote consultation and live broadcast director method and system for a multi-source medical image, and belongs to the technical field of medical information, and the system comprises a multi-source image collection and adaptation module, a streaming media processing and live broadcast director module, and an intelligent consultation cooperation module. The multi-source image acquisition and adaptation module is used for acquiring original image data streams from different medical image devices in real time through at least two different interface protocols, and performing standardization processing on the original image data streams to generate multiple paths of image data streams; the streaming media processing and live broadcast director module is in communication connection with the multi-source image acquisition and adaptation module and is used for receiving multiple paths of image data streams and carrying out parallel high-speed hardware coding on each path of image data streams; and the intelligent consultation cooperation module is in communication connection with the streaming media processing and live broadcast director module and is used for providing a consultation interaction interface. According to the invention, by integrating the multi-source heterogeneous image data, cross-device and cross-region real-time collaborative consultation is realized.

Description

Technical Field

[0001] This application relates to the field of medical information technology, specifically to a method and system for remote consultation and live broadcasting of multi-source medical images. Background Technology

[0002] Telemedicine consultations are an important means of overcoming geographical limitations and optimizing the allocation of medical resources. With the rapid development of medical imaging technology, the diagnosis of a single disease often requires reference to imaging data from different modalities. For example, tumor diagnosis requires the integration of information from CT, MRI, and even PET-CT; in interventional surgery, it is necessary to simultaneously observe real-time DSA images and preoperative CT three-dimensional models.

[0003] Existing remote consultation systems have several limitations: First, multi-source image access is difficult, as different manufacturers and devices with different modes use varying interface protocols, making it difficult to achieve unified and stable data acquisition and low-latency transmission. Second, existing systems mostly use simple video conferencing modes, which cannot achieve professional-grade screen switching and compositing, resulting in a single consultation perspective and a lack of focus. Furthermore, the interactivity during consultations is poor, making it difficult for experts to perform real-time and accurate annotation and measurement on shared images, and the annotation information suffers from significant delays, failing to guarantee synchronization of perspectives from all parties. In addition, due to the sensitivity of medical data, existing systems lack sufficient security in data transmission and storage, as well as the traceability of operations, posing risks of leakage and tampering.

[0004] Therefore, it is necessary to provide a method and system for remote consultation and live broadcasting of multi-source medical images to solve the above problems.

[0005] It should be noted that the information disclosed in this background section is only for understanding the background technology of this application concept, and therefore may include information that does not constitute prior art. Summary of the Invention

[0006] Based on the aforementioned problems in the existing technology, the problem to be solved by this application is to provide a method and system for remote consultation and live broadcasting of multi-source medical images, which integrates multi-source heterogeneous image data to achieve real-time collaborative consultation across devices and regions, significantly improving the diagnostic efficiency and accuracy of complex cases.

[0007] The technical solution adopted by this application to solve its technical problem is: a remote consultation and live broadcast system for multi-source medical images, the system including a multi-source image acquisition and adaptation module, a streaming media processing and live broadcast module, an intelligent consultation and collaboration module, and a data security and access control module;

[0008] The multi-source image acquisition and adaptation module is used to acquire raw image data streams from different medical imaging devices in real time through at least two different interface protocols, and to standardize the raw image data streams to generate multiple image data streams.

[0009] The streaming media processing and live broadcasting module is communicatively connected to the multi-source image acquisition and adaptation module. It is used to receive the multi-channel image data streams and perform parallel high-speed hardware encoding on each video data stream. The module also integrates a software-defined virtual broadcasting station. The virtual broadcasting station combines and switches at least two encoded video streams in real time according to a preset consultation scenario template or the received real-time broadcasting instructions to generate one or more composite video streams for live streaming.

[0010] The intelligent consultation collaboration module is communicatively connected to the streaming media processing and live broadcasting module to provide a consultation interaction interface. The consultation interaction interface includes at least a live video display area, a patient information area, an interactive annotation tool area, and a consultation member list area. The module also establishes a low-latency data synchronization channel independent of the video stream transmission channel to synchronize interactive operation commands in real time between multiple consultation terminals. The interactive operation commands include annotation information on video frames, reconstruction parameters of three-dimensional images, perspective transformation commands, and measurement data.

[0011] The data security and access control module is integrated with other modules in the system to perform two-way authentication and authorization for all users and devices accessing the system, encrypt all data streams in transmission end-to-end, and store audio and video streams, operation logs, and generated reports throughout the consultation process on the blockchain.

[0012] Furthermore, the streaming media processing and live broadcasting module also establishes a low-latency data synchronization channel independent of the video stream transmission channel, which is used to synchronize interactive operation instructions in real time between multiple consultation terminals. The interactive operation instructions include, but are not limited to, annotation information on video frames, reconstruction parameters of three-dimensional images, perspective transformation instructions, and measurement data.

[0013] Furthermore, the multi-source image acquisition and adaptation module further includes a protocol conversion submodule, a data parsing submodule, and a synchronization control submodule;

[0014] The protocol conversion submodule is configured with multiple physical interfaces and protocol stacks, which are used to establish physical connections and communication sessions with heterogeneous medical imaging devices, and convert non-standard signals or proprietary protocol data output by the devices into a unified intermediate data format.

[0015] The data parsing submodule is used to parse the converted intermediate data format, extract valid image frame data, device parameters, acquisition time information, and strip away control signaling unrelated to image display;

[0016] The synchronization control submodule has a built-in high-precision clock, which is used to stamp each frame of image data acquired from different devices with a unified timestamp, and to synchronize the clock source based on the network time protocol or the global positioning system clock. By calculating the time difference of data packet arrival and applying a dynamic buffering mechanism, it performs audio-visual synchronization and frame-level alignment on multiple image data streams to ensure that images from different sources remain consistent on the timeline.

[0017] Furthermore, the high-speed hardware encoding in the streaming media processing and live broadcasting module adopts a dedicated encoder array based on FPGA or ASIC. The encoder array supports parallel processing of multiple high-resolution, high-frame-rate medical image sequences and optimizes the configuration of key parameters in the encoding process, including setting the GOP structure to a single-frame I-frame or an ultra-short GOP length to reduce decoding latency, adaptively adjusting quantization parameters and bit rate according to network conditions, and preprocessing the image data before encoding to enhance the details of key diagnostic areas.

[0018] The virtual control console runs as a software application on a general-purpose server or a dedicated workstation. It provides a graphical user interface for the director to operate, including a source signal preview window, a program output window, a transition effects library, and a scene template library. The virtual control console also integrates an artificial intelligence-based scene recognition unit, which can analyze the video stream content in real time, automatically identify surgical instruments, lesion areas, or specific operations, and recommend or automatically perform perspective switching operations to the director accordingly.

[0019] Furthermore, the low-latency data synchronization channel in the intelligent consultation collaboration module adopts a WebRTC data channel or a private reliable transmission protocol based on UDP. The interactive operation instructions are encapsulated into lightweight structured data packets. The data packet structure includes an operation type field, a timestamp field, an operation target identifier field, and a set of operation parameters. When a consultation terminal generates an interactive operation instruction, the instruction is sent to the server instantly through the data synchronization channel and broadcast by the server to all other consultation terminals. After receiving the instruction, each terminal applies the instruction locally to reproduce the operation effect.

[0020] Furthermore, the interactive operation instructions specifically include annotation instructions, 3D operation instructions, and measurement instructions;

[0021] The annotation instructions include pen type, color, line width, starting coordinate sequence, pressure information, and the binding relationship between the annotation object and a specific video frame or 3D model vertex;

[0022] The three-dimensional operation commands include rotation, scaling, translation angle and displacement of the three-dimensional volume rendering model, window width and window level adjustment parameters, and the position and orientation of the virtual cutting surface;

[0023] The measurement instructions include the coordinates of the start and end points in length measurement, the vertex and two side vectors in angle measurement, and the range coordinates of the region of interest.

[0024] Furthermore, the implementation of the data security and access control module includes:

[0025] Use a public key infrastructure system based on digital certificates to authenticate the identities of consultation expert terminals and medical imaging equipment;

[0026] Use the national cryptographic algorithm SM4 or AES-256-GCM to encrypt the audio and video streams and synchronization data packets during transmission;

[0027] A hash function is used to generate data fingerprints for key data generated during the consultation process, including keyframes of the video stream, annotation information, and diagnostic opinions. The data fingerprints are then uploaded to a blockchain network for storage. This blockchain network is a permissioned blockchain, where only authorized nodes can participate in consensus.

[0028] Furthermore, the system also integrates an intelligent image analysis engine, which is connected to the streaming media processing and live broadcasting module and the intelligent consultation collaboration module;

[0029] The intelligent image analysis engine is loaded with a trained deep learning model for online analysis of real-time transmitted image streams. The analysis tasks include automatic lesion detection and delineation, anatomical structure recognition, and hemodynamic parameter calculation.

[0030] The analysis results are displayed in real time on the live video stream in the form of an overlay, or sent to the consultation expert terminal through the low-latency data synchronization channel to provide auxiliary information for expert decision-making.

[0031] On the other hand, a method for remote consultation and live broadcasting of multi-source medical images is provided, the method comprising the following steps:

[0032] Step S101: Through the multi-source image acquisition and adaptation module, real-time image data streams from at least two different medical imaging devices are simultaneously accessed. Protocol parsing, format conversion, and time synchronization are performed on each data stream to output standardized, frame-aligned multi-channel image data streams.

[0033] Step S102: The streaming media processing and live broadcasting module receives the standardized multi-channel image data streams, uses a hardware encoder to efficiently compress and encode them, and at the same time, the virtual broadcasting station selects one or more encoded streams to synthesize them according to the consultation requirements to generate the main live stream and at least one auxiliary live stream.

[0034] Step S103: The intelligent consultation collaboration module distributes the live stream address to the authorized expert terminal, and the expert accesses the consultation through the terminal; during the consultation, any annotation or operation performed by any expert on the local terminal generates an interactive instruction, which is sent to the server through an independent data synchronization channel and broadcast to other terminals to ensure that the interfaces of all participants are synchronized;

[0035] Step S104: Throughout the consultation process, the data security and access control module monitors data access and transmission in real time, performs encryption and authentication, and stores key consultation events and data in the blockchain after hashing.

[0036] Furthermore, the method also includes an interactive synchronization step for the three-dimensional image:

[0037] When a consulting expert rotates, scales, or adjusts the rendering parameters of a 3D reconstruction model, the terminal records the changes in the model view matrix, projection matrix, and rendering parameters before and after the operation.

[0038] The changes are encapsulated into three-dimensional operation instructions, which are then sent and broadcast through a data synchronization channel.

[0039] After receiving the instruction, other terminals apply the same transformation to the same 3D model loaded locally, thereby presenting a consistent 3D perspective and rendering effect on all terminals and realizing collaborative operation in 3D space.

[0040] The beneficial effects of this application are: The remote consultation and live broadcasting method and system for multi-source medical images provided by this application integrates multi-source heterogeneous image data to realize real-time collaborative consultation across devices and regions, which significantly improves the diagnostic efficiency and accuracy of complex cases, especially in emergency care, consultation of difficult and complicated diseases and remote surgical guidance.

[0041] In addition to the purposes, features, and advantages described above, this application has other purposes, features, and advantages. A further detailed description of this application will be provided below with reference to the figures. Attached Figure Description

[0042] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0043] Figure 1This is an overall schematic diagram of a remote consultation and live broadcast system for multi-source medical images according to this application;

[0044] Figure 2 This is a flowchart illustrating a remote consultation and live broadcast method for multi-source medical images according to this application. Detailed Implementation

[0045] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0046] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0047] like Figures 1 to 2 As shown, this application provides a remote consultation and live broadcast system for multi-source medical images. The system includes a multi-source image acquisition and adaptation module, a streaming media processing and live broadcast module, an intelligent consultation collaboration module, and a data security and access control module.

[0048] The multi-source image acquisition and adaptation module is used to acquire raw image data streams from different medical imaging devices in real time through at least two different interface protocols, and to standardize the raw image data streams to generate multiple image data streams.

[0049] In the process of remote medical image consultation, different medical imaging equipment, such as CT, MRI, and ultrasound, are often involved. Their data formats and transmission protocols are heterogeneous, making it difficult to synchronize and access multi-source images in real time. Therefore, a multi-source image acquisition and adaptation module is used to analyze and convert data streams of different protocols in real time, realize the standardized encapsulation and timestamp alignment of heterogeneous image data, ensure the spatiotemporal consistency of multi-source images in subsequent processing, and provide a synchronous, clear, and interactive image foundation for remote consultation.

[0050] For example, the interface protocols include DICOM protocol, HD-SDI interface, HDMI interface and RTSP / RTMP streaming protocol, through the compatibility support of the above multiple interface protocols.

[0051] Furthermore, after acquiring the raw image data stream, it is necessary to perform standardization processing. This is because the image data output by different devices differs in resolution, frame rate, color space, and encoding format. Direct use of these data can lead to inconsistent display or parsing errors. Standardization processing includes format unification, resolution standardization, and timestamp synchronization of the raw image data stream to ensure that all video data streams have a consistent spatiotemporal reference during subsequent fusion and playback.

[0052] The streaming media processing and live broadcasting module is communicatively connected to the multi-source image acquisition and adaptation module. It is used to receive the above-mentioned multiple image data streams and perform parallel high-speed hardware encoding on each image data stream. At the same time, the module also integrates a software-defined virtual broadcasting station. The virtual broadcasting station is used to synthesize and switch at least two encoded video streams in real time according to the preset consultation scene template or the received real-time broadcasting instructions, to generate one or more composite video streams for live streaming.

[0053] The intelligent consultation collaboration module communicates with the streaming media processing and live broadcasting module to provide a consultation interaction interface. This consultation interaction interface includes at least a live video display area, a patient information area, an interactive annotation tool area, and a consultation member list area. The module also establishes a low-latency data synchronization channel independent of the video stream transmission channel to synchronize interactive operation commands in real time between multiple consultation terminals. The interactive operation commands include, but are not limited to, annotation information on video frames, reconstruction parameters of three-dimensional images, viewpoint transformation commands, and measurement data.

[0054] For example, the encoding formats include H.264, H.265 and AV1, and the composition and switching include picture-in-picture, left-right split screen, four-screen split screen and focus-following switching modes to meet the visual presentation needs of different consultation scenarios.

[0055] High-speed encoding can employ a dedicated encoder array based on FPGA or ASIC. This encoder array supports parallel processing of multiple high-resolution, high-frame-rate medical image sequences and optimizes key parameters during the encoding process. This includes setting the GOP structure to a single-frame I-frame or an ultra-short GOP length to reduce decoding latency, adaptively adjusting quantization parameters and bit rate according to network conditions, and preprocessing the image data before encoding to enhance the details of key diagnostic areas.

[0056] The virtual production console runs as a software application on a general-purpose server or a dedicated workstation. It provides a graphical user interface for the production director to operate, including a source signal preview window, a program output window, a transition effects library, and a scene template library. The virtual production console also integrates an artificial intelligence-based scene recognition unit, which can analyze the video stream content in real time, automatically identify surgical instruments, lesion areas, or specific operations, and recommend or automatically perform perspective switching operations to the production director accordingly.

[0057] The multi-source image acquisition and adaptation module further includes a protocol conversion submodule, a data parsing submodule, and a synchronization control submodule. The protocol conversion submodule is equipped with multiple physical interfaces and protocol stacks for establishing physical connections and communication sessions with heterogeneous medical imaging devices, converting non-standard signals or proprietary protocol data output by the devices into a unified intermediate data format.

[0058] The data parsing submodule is used to parse the converted intermediate data format, extract valid image frame data, device parameters, acquisition time information, and remove control signaling that is not related to image display;

[0059] The synchronization control submodule has a built-in high-precision clock, which is used to stamp each frame of image data acquired from different devices with a unified timestamp. It performs clock source synchronization based on the network time protocol or the global positioning system clock. By calculating the time difference of data packet arrival and applying a dynamic buffering mechanism, it performs audio-visual synchronization and frame-level alignment on multiple image data streams to ensure that images from different sources remain consistent on the timeline.

[0060] Furthermore, the low-latency data synchronization channel in the intelligent consultation collaboration module adopts a WebRTC data channel or a private reliable transmission protocol based on UDP, and the interactive operation instructions are encapsulated into lightweight structured data packets. The data packet structure includes an operation type field, a timestamp field, an operation target identifier field, and a set of operation parameters. When a consultation terminal generates an interactive operation instruction, the interactive operation instruction is sent to the server in real time through the data synchronization channel, and the server broadcasts it to all other consultation terminals. After receiving the instruction, each terminal applies the instruction locally to reproduce the operation effect, realizing seamless synchronization of multi-party collaborative annotation, perspective linkage, and real-time annotation.

[0061] For example, the interactive operation instructions specifically include annotation instructions, 3D operation instructions, and measurement instructions. Annotation instructions include pen type, color, line width, starting coordinate sequence, pressure information, and the binding relationship between the annotation object and specific video frames or 3D model vertices. 3D operation instructions include rotation, scaling, translation angles and displacements of the 3D volumetric rendering model, window width and window level adjustment parameters, and the position and orientation of virtual cutting planes. Measurement instructions include the starting and ending coordinates in length measurement, the vertex and two side vectors in angle measurement, and the range coordinates of the region of interest, and are associated with the timestamp and spatial coordinate system of the corresponding image frame to ensure consistency and traceability of measurement results across multiple terminals. All operation instructions can be scheduled through a priority queue. High-priority instructions, such as emergency annotation or viewpoint reset, can be executed in the queue, while low-priority instructions are transmitted when bandwidth is idle to ensure real-time response to critical operations. Instruction data packets support incremental updates and differential compression to reduce network load.

[0062] The implementation methods for the data security and access control module include:

[0063] Use a public key infrastructure system based on digital certificates to authenticate the identities of consultation expert terminals and medical imaging equipment;

[0064] Use the national cryptographic algorithm SM4 or AES-256-GCM to encrypt the audio and video streams and synchronization data packets during transmission;

[0065] A hash function is used to generate data fingerprints for key data generated during the consultation process, including key frames of the video stream, annotation information, and diagnostic opinions. The data fingerprints are then uploaded to a blockchain network for storage. The blockchain network is a permissioned blockchain, where only authorized nodes can participate in consensus.

[0066] It should be noted that data fingerprints are bound to timestamps and operator identification and then uploaded to the blockchain to ensure that the operation is non-repudiable and auditable. Access control is based on the RBAC model and combined with role configuration in dynamic consultation scenarios to achieve fine-grained access control. For example, the attending physician has the authority to execute instructions and confirm conclusions, consultation members can only submit annotation and measurement results, and image data is only decrypted and displayed on authorized terminals to prevent unauthorized access.

[0067] Based on the above embodiments, another embodiment of the present invention is that the system also integrates an intelligent image analysis engine, which is connected to the streaming media processing and live broadcasting module and the intelligent consultation collaboration module. The intelligent image analysis engine is loaded with a trained deep learning model for online analysis of the real-time transmitted image stream. The analysis tasks include automatic lesion detection and delineation, anatomical structure recognition, and hemodynamic parameter calculation.

[0068] The analysis results are displayed in real time on the live video stream in the form of an overlay, or sent to the consultation expert terminal through the low-latency data synchronization channel to provide auxiliary information for expert decision-making. At the same time, it supports the fusion and verification of analysis results with expert annotations to improve diagnostic accuracy.

[0069] For example, in emergency consultations for stroke, the intelligent image analysis engine can detect intracranial large vessel occlusion in real time and automatically generate perfusion mismatch maps to help experts quickly assess the extent of the ischemic penumbra. The engine supports multi-task parallel reasoning, loads corresponding models according to different diseases, such as lung nodule classification models and liver tumor segmentation models, and transmits the analysis confidence along with the models for clinical reference.

[0070] Based on the above embodiments, another embodiment of the present invention provides a method for remote consultation and live broadcasting of multi-source medical images, the method comprising the following steps:

[0071] Step S101: Multi-source image access and standardization processing: Through the multi-source image acquisition and adaptation module, real-time image data streams from at least two different medical imaging devices are simultaneously accessed. Protocol parsing, format conversion and time synchronization are performed on each data stream, and standardized, frame-aligned multi-source image data streams are output.

[0072] Step S102: Low-latency encoding and live stream generation: The streaming media processing and live broadcasting module receives the standardized multi-channel image data streams and uses a hardware encoder to efficiently compress and encode them. At the same time, the virtual broadcasting station selects one or more encoded streams to synthesize according to the consultation requirements to generate the main live stream and at least one auxiliary live stream.

[0073] Step S103: Consultation Collaboration and Interaction Synchronization: The intelligent consultation collaboration module distributes the live stream address to the authorized expert terminal, and the expert accesses the consultation through the terminal; during the consultation, any annotation or operation performed by any expert on the local terminal generates an interaction instruction, which is sent to the server through an independent data synchronization channel and broadcast to other terminals to ensure that the interfaces of all participants are synchronized;

[0074] Step S104: Security Monitoring and Data Storage: Throughout the consultation process, the data security and access control module monitors data access and transmission in real time, performs encryption and authentication, and stores key consultation events and data in the blockchain after hashing.

[0075] The logic for stream synthesis in the virtual broadcast control station includes:

[0076] Step S102a: Predefine multiple consultation scenario templates. The templates specify the layout rules of video sources in different scenarios. For example, in a live surgery scenario, laparoscopic images are the main screen, and vital sign monitoring images are picture-in-picture; in a multidisciplinary consultation scenario, CT, MRI, and PET-CT images are displayed side-by-side in a four-screen format.

[0077] Step S102b: The director or the system AI can manually or automatically switch between different scene templates according to the consultation process;

[0078] Step S102c: During stream synthesis, intelligent scaling and region enhancement are performed on non-main screens to ensure that key information is clearly distinguishable, while the bitrate is appropriately reduced in unimportant areas to save bandwidth.

[0079] Furthermore, the specific sub-steps of interactive synchronization in step S103 include:

[0080] Step S103a: When the chief consulting expert makes annotations on the live video frame, the terminal client captures the trajectory coordinates, operation type, and parameters of the mouse or touch screen event;

[0081] Step S103b: The terminal client immediately sends the encapsulated annotation instructions to the synchronization server through a low-latency data channel, and at the same time, previews the annotation effect on the local interface;

[0082] Step S103c: After the synchronization server verifies the sender's permissions, it adds the annotation command with the server's timestamp and then broadcasts it to all terminals participating in the consultation (including the sender itself).

[0083] Step S103d: After receiving the instruction, each receiving terminal accurately reproduces the marked graphic at the corresponding frame position in the local video stream according to the timestamp and parameters in the instruction; if the instruction arrives later than the video frame due to network jitter, the instruction frame tracking technology is used to apply the marked graphic to the correct historical frame.

[0084] Furthermore, the method also includes an interactive synchronization step for the three-dimensional image:

[0085] Step S103e: When a consulting expert rotates, scales, or adjusts the rendering parameters of the 3D reconstruction model, the terminal records the changes in the model view matrix, projection matrix, and rendering parameters before and after the operation.

[0086] Step S103f: Encapsulate the changes into three-dimensional operation instructions and send and broadcast them through the data synchronization channel;

[0087] Step S103g: After receiving the instruction, other terminals apply the same transformation to the same 3D model loaded locally, thereby presenting a consistent 3D perspective and rendering effect on all terminals and realizing collaborative operation in 3D space.

[0088] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A remote consultation and live broadcast system for multi-source medical images, characterized in that: The system includes a multi-source image acquisition and adaptation module, a streaming media processing and live broadcasting module, an intelligent consultation and collaboration module, and a data security and access management module. The multi-source image acquisition and adaptation module is used to acquire raw image data streams from different medical imaging devices in real time through at least two different interface protocols, and to standardize the raw image data streams to generate multiple image data streams. The streaming media processing and live broadcasting module is communicatively connected to the multi-source image acquisition and adaptation module. It is used to receive the multi-channel image data streams and perform parallel high-speed hardware encoding on each video data stream. The module also integrates a software-defined virtual broadcasting station. The virtual broadcasting station combines and switches at least two encoded video streams in real time according to a preset consultation scenario template or the received real-time broadcasting instructions to generate one or more composite video streams for live streaming. The intelligent consultation collaboration module is communicatively connected to the streaming media processing and live broadcasting module to provide a consultation interaction interface. The consultation interaction interface includes at least a live video display area, a patient information area, an interactive annotation tool area, and a consultation member list area. The module also establishes a low-latency data synchronization channel independent of the video stream transmission channel to synchronize interactive operation commands in real time between multiple consultation terminals. The interactive operation commands include annotation information on video frames, reconstruction parameters of three-dimensional images, perspective transformation commands, and measurement data. The data security and access control module is integrated with other modules in the system to perform two-way authentication and authorization for all users and devices accessing the system, encrypt all data streams in transmission end-to-end, and store audio and video streams, operation logs, and generated reports throughout the consultation process on the blockchain.

2. The remote consultation and live broadcast system for multi-source medical images according to claim 1, characterized in that: The streaming media processing and live broadcasting module also establishes a low-latency data synchronization channel independent of the video stream transmission channel, which is used to synchronize interactive operation instructions in real time between multiple consultation terminals. The interactive operation instructions include, but are not limited to, annotation information on video frames, reconstruction parameters of three-dimensional images, perspective transformation instructions, and measurement data.

3. The remote consultation and live broadcast system for multi-source medical images according to claim 1, characterized in that: The multi-source image acquisition and adaptation module further includes a protocol conversion submodule, a data parsing submodule, and a synchronization control submodule; The protocol conversion submodule is configured with multiple physical interfaces and protocol stacks, which are used to establish physical connections and communication sessions with heterogeneous medical imaging devices, and convert non-standard signals or proprietary protocol data output by the devices into a unified intermediate data format. The data parsing submodule is used to parse the converted intermediate data format, extract valid image frame data, device parameters, acquisition time information, and strip away control signaling unrelated to image display; The synchronization control submodule has a built-in high-precision clock, which is used to stamp each frame of image data acquired from different devices with a unified timestamp, and to synchronize the clock source based on the network time protocol or the global positioning system clock. By calculating the time difference of data packet arrival and applying a dynamic buffering mechanism, it performs audio-visual synchronization and frame-level alignment on multiple image data streams to ensure that images from different sources remain consistent on the timeline.

4. The remote consultation and live broadcast system for multi-source medical images according to claim 3, characterized in that: The high-speed hardware encoding in the streaming media processing and live broadcasting module adopts a dedicated encoder array based on FPGA or ASIC. The encoder array supports parallel processing of multiple high-resolution, high-frame-rate medical image sequences and optimizes the configuration of key parameters in the encoding process, including setting the GOP structure to a single frame I-frame or an ultra-short GOP length to reduce decoding latency, adaptively adjusting quantization parameters and bit rate according to network conditions, and preprocessing the image data before encoding to enhance the details of key diagnostic areas. The virtual control console runs as a software application on a general-purpose server or a dedicated workstation. It provides a graphical user interface for the director to operate, including a source signal preview window, a program output window, a transition effects library, and a scene template library. The virtual control console also integrates an artificial intelligence-based scene recognition unit, which can analyze the video stream content in real time, automatically identify surgical instruments, lesion areas, or specific operations, and recommend or automatically perform perspective switching operations to the director accordingly.

5. The remote consultation and live broadcast system for multi-source medical images according to claim 4, characterized in that: The low-latency data synchronization channel in the intelligent consultation collaboration module adopts a WebRTC data channel or a private reliable transmission protocol based on UDP. The interactive operation instructions are encapsulated into lightweight structured data packets. The data packet structure includes an operation type field, a timestamp field, an operation target identifier field, and a set of operation parameters. When a consultation terminal generates an interactive operation command, the command is sent to the server in real time through the data synchronization channel, and then broadcast by the server to all other consultation terminals. After receiving the command, each terminal applies the command locally to reproduce the operation effect.

6. The remote consultation and live broadcast system for multi-source medical images according to claim 5, characterized in that: The interactive operation commands specifically include annotation commands, 3D operation commands, and measurement commands; The annotation instructions include pen type, color, line width, starting coordinate sequence, pressure information, and the binding relationship between the annotation object and a specific video frame or 3D model vertex; The three-dimensional operation commands include rotation, scaling, translation angle and displacement of the three-dimensional volume rendering model, window width and window level adjustment parameters, and the position and orientation of the virtual cutting surface; The measurement instructions include the coordinates of the start and end points in length measurement, the vertex and two side vectors in angle measurement, and the range coordinates of the region of interest.

7. The remote consultation and live broadcast system for multi-source medical images according to claim 1, characterized in that: The implementation methods of the data security and access control module include: Use a public key infrastructure system based on digital certificates to authenticate the identities of consultation expert terminals and medical imaging equipment; Use the national cryptographic algorithm SM4 or AES-256-GCM to encrypt the audio and video streams and synchronization data packets during transmission; A hash function is used to generate data fingerprints for key data generated during the consultation process, including keyframes of the video stream, annotation information, and diagnostic opinions. The data fingerprints are then uploaded to a blockchain network for storage. This blockchain network is a permissioned blockchain, where only authorized nodes can participate in consensus.

8. A remote consultation and live broadcast system for multi-source medical images according to any one of claims 1 to 7, characterized in that: The system also integrates an intelligent image analysis engine, which is connected to the streaming media processing and live broadcasting module and the intelligent consultation collaboration module. The intelligent image analysis engine is loaded with a trained deep learning model for online analysis of real-time transmitted image streams. The analysis tasks include automatic lesion detection and delineation, anatomical structure recognition, and hemodynamic parameter calculation. The analysis results are displayed in real time on the live video stream in the form of an overlay, or sent to the consultation expert terminal through the low-latency data synchronization channel to provide auxiliary information for expert decision-making.

9. A method for remote consultation and live broadcasting of multi-source medical images based on the system of claim 1, characterized in that: The method includes the following steps: Step S101: Through the multi-source image acquisition and adaptation module, real-time image data streams from at least two different medical imaging devices are simultaneously accessed. Protocol parsing, format conversion, and time synchronization are performed on each data stream to output standardized, frame-aligned multi-channel image data streams. Step S102: The streaming media processing and live broadcasting module receives the standardized multi-channel image data streams, uses a hardware encoder to efficiently compress and encode them, and at the same time, the virtual broadcasting station selects one or more encoded streams to synthesize them according to the consultation requirements to generate the main live stream and at least one auxiliary live stream. Step S103: The intelligent consultation collaboration module distributes the live stream address to the authorized expert terminal, and the expert accesses the consultation through the terminal; during the consultation, any annotation or operation performed by any expert on the local terminal generates an interactive instruction, which is sent to the server through an independent data synchronization channel and broadcast to other terminals to ensure that the interfaces of all participants are synchronized; Step S104: Throughout the consultation process, the data security and access control module monitors data access and transmission in real time, performs encryption and authentication, and stores key consultation events and data in the blockchain after hashing.

10. A method for remote consultation and live broadcasting of multi-source medical images according to claim 9, characterized in that: The method also includes an interactive synchronization step for 3D images: When a consulting expert rotates, scales, or adjusts the rendering parameters of a 3D reconstruction model, the terminal records the changes in the model view matrix, projection matrix, and rendering parameters before and after the operation. The changes are encapsulated into three-dimensional operation instructions, which are then sent and broadcast through a data synchronization channel. After receiving the instruction, other terminals apply the same transformation to the same 3D model loaded locally, thereby presenting a consistent 3D perspective and rendering effect on all terminals and realizing collaborative operation in 3D space.

Images