Ultrasonic inspection remote teaching and quality control method and device based on audio and video

Abstract

The invention provides an audio and video-based ultrasonic examination remote teaching and quality control method and device. The method comprises the steps that an audio and video room server side obtains a real-time ultrasonic video stream and a doctor camera video stream collected by a consulting room side; the audio and video room server generates a multi-consulting-room real-time monitoring view in a grid layout form according to the real-time ultrasonic video stream and the doctor camera video stream; when the expert end initiates a guidance request or the consulting room end initiates a help seeking request, the audio and video room server creates an exclusive audio and video room and accesses a requester and a receiver into the exclusive audio and video room; and the audio and video room server bidirectionally transmits the audio and video streams of the requester and the receiver through the exclusive audio and video room. Experts can inspect a plurality of consulting rooms in real time through the grid layout view, discover the problem of nonstandard operation in time and intervene in guidance in time, so that the procedural quality control of ultrasonic inspection is realized, and the inspection normalization is improved from the source.

Description

Technical Field

[0001] This invention relates to the fields of telemedicine and audio-visual communication technology, specifically to a method and apparatus for remote teaching and quality control of ultrasound examinations based on audio and video. Background Technology

[0002] With the continuous development of medical technology, ultrasound examination, with its advantages of being non-invasive, real-time, and convenient, has become an indispensable core tool in clinical disease diagnosis, widely used in various diagnostic and treatment scenarios such as internal medicine, surgery, obstetrics and gynecology, and cardiology. The diagnostic accuracy of ultrasound examination highly depends on the operator's skill level, precise mastery of standard sections, and accumulated clinical experience. However, there are significant differences in the skill levels of physicians at different levels of medical institutions. Primary hospitals and young physicians often face bottlenecks in handling complex cases and implementing standardized procedures, and urgently need guidance and assistance from high-quality expert resources.

[0003] In the traditional model, experts need to be physically present in the examination room to observe the physician's techniques and image acquisition process. This makes it difficult to simultaneously monitor the real-time operation of multiple examination rooms, leading to problems such as non-standard operation and substandard image acquisition that cannot be corrected in a timely manner. This fundamentally affects the consistency of ultrasound examination quality and medical safety. At the same time, high-quality ultrasound experts are highly concentrated in large hospitals, making it difficult for primary hospitals and community health institutions to easily access expert support, further widening the gap in medical service levels between regions.

[0004] In the field of medical education and talent training, traditional ultrasound teaching models are limited by physical space and time constraints, requiring trainees to gather in person to observe expert operations. This results in a narrow teaching scope and low coverage efficiency, failing to meet the needs of large-scale, routine training. Furthermore, when primary care physicians and young doctors encounter difficult cases and need assistance, existing consultation systems often involve complex processes or are limited to asynchronous image transmission. Experts cannot intervene remotely and in real time to conduct visual communication and collaborative operations, leading to long problem-solving cycles and high communication costs. Summary of the Invention

[0005] In view of the aforementioned problems, this application is proposed to provide a method and apparatus for remote teaching and quality control of ultrasound examination based on audio and video, which overcomes or at least partially solves the aforementioned problems, comprising: A method for remote teaching and quality control of ultrasound examinations based on audio and video, the method involving an expert terminal, a consultation room terminal, and an audio / video room server terminal; the expert terminal is used to connect to experts providing remote guidance; the consultation room terminal is used to acquire local video streams; the audio / video room server terminal is used for creating, maintaining, and destroying audio / video rooms, and the audio / video rooms are used for subscribing to, publishing, and forwarding multi-party video streams; the method includes: The audio and video room server acquires real-time ultrasound video streams and doctor's camera video streams collected from the examination room. The audio-visual room server generates a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. When the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the recipient to the dedicated audio-visual room. The audio and video room server transmits audio and video streams bidirectionally between the requester and the receiver through the dedicated audio and video room, and simultaneously records audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

[0006] Furthermore, the step of the audio-visual room server acquiring the real-time ultrasound video stream and the doctor's camera video stream collected by the examination room terminal includes: The audio-visual room server receives real-time ultrasound video streams and doctor's camera video streams acquired and encoded by the clinic terminal through a streaming media transmission protocol; wherein, the real-time ultrasound video stream is obtained by the clinic terminal from the output of the local ultrasound equipment, and the doctor's camera video stream is obtained by the clinic terminal from the output of the local camera equipment; The audio and video room server decodes the received encoded video stream and performs storage and forwarding preprocessing on the decoded video stream.

[0007] Furthermore, the step of the audio-visual room server generating a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream includes: When the audio-visual room server detects that an expert has entered the monitoring page, it automatically queries all online clinic information that has been successfully pushed to the audio-visual room server, integrates the clinic information and generates an online clinic list based on the clinic information, and simultaneously obtains the target video stream that the online clinic is currently viewing; The audio and video room server performs image adaptation processing on the target video stream to ensure that only one target video stream is displayed in a single consultation room. The audio and video room server integrates the target video stream in a grid layout and pushes it to the expert terminal, which then renders and displays it in real time to form a real-time monitoring view of multiple clinic rooms.

[0008] Furthermore, when the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the recipient to the dedicated audio-visual room, including: The audio-visual room server receives a guidance request initiated by the expert after selecting a target clinic from the multi-clinic real-time monitoring view; wherein, the guidance request includes the expert identifier, the target clinic identifier, and a description of the guidance requirements; The audio / video room server creates a dedicated audio / video room based on the guidance request and generates unique room identification information; The audio and video room server sends the room identification information to the expert terminal and the target consultation room terminal respectively, and pushes a guidance call notification containing the room identification information, the expert terminal identification, and the guidance needs description to the target consultation room terminal. The audio-visual room server receives access requests sent by the expert and the target clinic based on the room identification information, verifies the identification information of both parties, and connects both parties to the exclusive audio-visual room after successful verification.

[0009] Furthermore, when the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the recipient to the dedicated audio-visual room, including: The audio-visual room server receives a help request initiated by the target expert terminal after the consultation room terminal selects the online expert; wherein, the help request includes the consultation room terminal identifier, the target expert terminal identifier, basic case information, and a description of the help problem; The audio and video room server creates a dedicated audio and video room and generates a unique room identifier based on the request for help; The audio and video room server sends the room identification information to the consultation room terminal and the target expert terminal respectively, and at the same time pushes a help call notification containing room identification information, consultation room terminal identification, basic case information and a description of the help problem to the target expert terminal. The audio-visual room server receives the access request sent by the target expert after accepting the call, as well as the access request sent by the clinic. It verifies the identification information of both parties, and after successful verification, connects both parties to the exclusive audio-visual room.

[0010] Furthermore, the audio / video room server, through the dedicated audio / video room, bidirectionally transmits the audio / video streams of the requester and the receiver, and synchronously records audio / video data, interaction timestamps, and operation logs to generate teaching and quality control records. This includes the following steps: The audio and video room server establishes a low-latency, two-way communication channel between the requester and the receiver within the dedicated audio and video room. The audio and video room server receives the audio and video streams published by the requesting party in real time, and after decoding, noise reduction and synchronization optimization of the audio and video streams, forwards them to the receiving party; The audio and video room server synchronously receives the audio and video streams published by the recipient, and after decoding, noise reduction and synchronization optimization of the audio and video streams, forwards them to the requester. The audio and video room server synchronously records the audio and video data, interaction timestamps, and operation logs within the room, generating teaching and quality control records.

[0011] Furthermore, the step of the audio-visual room server synchronously recording audio-visual data, interaction timestamps, and operation logs within the room to generate teaching and quality control records also includes: The audio and video room server will associate and integrate the recorded audio and video data, interaction timestamps, and operation logs, and package them into standardized teaching and quality control record files according to a preset format. The audio and video room server stores the teaching and quality control record files and supports retrieval and retrieval by keywords such as clinic identifier, expert identifier, time range, and case information.

[0012] A device for remote teaching and quality control of ultrasound examinations based on audio and video, the device being used to implement the steps of the method for remote teaching and quality control of ultrasound examinations based on audio and video as described in any of the preceding claims, the device involving an expert terminal, a consultation room terminal, and an audio-video room server terminal; the expert terminal is used to connect to a remotely guiding expert; the consultation room terminal is used to acquire local video streams; the audio-video room server terminal is used for creating, maintaining, and destroying audio-video rooms, and the audio-video rooms are used for subscribing to, publishing, and forwarding multi-party video streams; the device includes: The acquisition module is used by the audio and video room server to acquire the real-time ultrasound video stream and doctor's camera video stream collected by the clinic terminal; The monitoring view module is used by the audio and video room server to generate a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. The dedicated audio and video room module is used to create a dedicated audio and video room and connect the requester and the receiver to the dedicated audio and video room when the expert initiates a guidance request or the clinic initiates a help request. The bidirectional transmission module is used by the audio and video room server to bidirectionally transmit the audio and video streams of the requester and the receiver through the dedicated audio and video room, and synchronously record audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

[0013] An extraction device for remote teaching and quality control of ultrasound examination based on audio and video includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the method for remote teaching and quality control of ultrasound examination based on audio and video as described above.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the audio-visual-based remote teaching and quality control method for ultrasound examination as described above.

[0015] This application has the following advantages: In the embodiments of this application, addressing the problems of difficulty in real-time process quality control during ultrasound examinations, limitations of teaching resources in terms of time and space, low efficiency of remote collaboration, and the inability of existing systems to achieve deep integration of real-time quality control, remote consultation, and process teaching, this application provides an integrated solution through a "two-way communication model, grid monitoring, and dedicated audio-visual room interaction." Specifically, it provides a method for remote teaching and quality control of ultrasound examinations based on audio and video, involving an expert terminal, a consultation room terminal, and an audio-visual room server. The expert terminal is used to connect to remotely guiding experts; the consultation room terminal is used to acquire local video streams; and the audio-visual room server is used for the creation, maintenance, and destruction of audio-visual rooms, which are used for multi-party video... The method includes: subscription, publishing, and forwarding of streams; the audio-visual room server acquires real-time ultrasound video streams and doctor's camera video streams collected by the clinic terminal; the audio-visual room server generates a multi-clinic real-time monitoring view based on the real-time ultrasound video streams and the doctor's camera video streams in a grid layout; when the expert terminal initiates a guidance request or the clinic terminal initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the receiver to the dedicated audio-visual room; the audio-visual room server transmits the audio-visual streams of the requester and the receiver bidirectionally through the dedicated audio-visual room, and simultaneously records audio-visual data, interaction timestamps, and operation logs to generate teaching and quality control records. By employing a two-way channel design that integrates expert monitoring and in-clinic consultation, the system overcomes the limitations of existing systems that can only passively respond to requests. Experts can monitor multiple clinics in real time through a grid layout view, promptly identifying operational irregularities and providing immediate guidance. This enables process-based quality control during ultrasound examinations, fundamentally improving the standardization of examinations. Simultaneously, a dedicated audio-visual room provides an immersive, low-latency real-time interactive environment for both parties, making remote guidance feel like being on-site and efficiently resolving the issue of seeking help for difficult cases by primary care physicians. Furthermore, the system synchronously records the entire interaction process, generating teaching and quality control records. This provides real-world case materials for subsequent teaching reviews and operational training, while also enabling quality control traceability. It truly integrates the three major functions of real-time quality control, remote consultation, and process teaching, effectively overcoming time and space limitations, optimizing the allocation of high-quality medical resources, and comprehensively improving the quality of ultrasound diagnosis and treatment as well as the efficiency of talent training. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1This is a flowchart illustrating the steps of a method for remote teaching and quality control of ultrasound examination based on audio and video, provided in one embodiment of this application. Figure 2 This is a structural block diagram of a device for remote teaching and quality control of ultrasound examination based on audio and video, provided in one embodiment of this application. Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention; 1. Computer equipment; 2. External devices; 3. Processing unit; 4. Bus; 5. Network adapter; 6. I / O interface; 7. Display; 8. Memory; 9. Random access memory; 10. Cache memory; 11. Storage system; 12. Program / utility; 13. Program module. Detailed Implementation

[0018] To make the objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] The inventors, through analysis of existing technologies, discovered that in the field of ultrasound examination, experts struggle to provide real-time, visual supervision and immediate guidance to ultrasound physicians in multiple examination rooms regarding their dynamic and continuous operational techniques and image acquisition processes. This makes it impossible to fundamentally guarantee the standardization and quality of examinations. Traditional ultrasound teaching requires trainees to observe in person, which, limited by physical space and expert time, results in a narrow teaching scope and low efficiency. There is a lack of a flexible teaching mechanism that allows trainees to remotely and in real-time observe the entire process guided by experts. When primary care physicians and young physicians encounter difficult cases and need assistance, existing consultation systems are often complex or limited to asynchronous image transmission. Experts cannot remotely intervene in real-time and conduct visual communication and collaborative operations, leading to long problem-solving cycles and high communication costs.

[0020] Reference Figure 1 This application illustrates a method for remote teaching and quality control of ultrasound examinations based on audio and video, according to an embodiment of the present application. The method involves an expert terminal, a consultation room terminal, and an audio / video room server. The expert terminal is used to connect to a remotely guiding expert. The consultation room terminal is used to acquire local video streams. The audio / video room server is used to create, maintain, and destroy audio / video rooms, which are used for subscribing to, publishing, and forwarding multi-party video streams. The method includes: S110, The audio and video room server acquires the real-time ultrasound video stream and the doctor's camera video stream collected by the clinic terminal; S120. The audio and video room server generates a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. S130. When the expert initiates a guidance request or the clinic initiates a help request, the audio and video room server creates a dedicated audio and video room and connects the requester and the receiver to the dedicated audio and video room. S140. The audio and video room server transmits the audio and video streams of the requester and the receiver bidirectionally through the dedicated audio and video room, and synchronously records the audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

[0021] It should be noted that the clinic terminal is installed on a computer in the clinic. It collects local real-time ultrasound video streams and real-time video streams from the doctor's camera, and pushes these two video streams out through a streaming media server; at the same time, it can receive and interact with audio and video communication requests from the expert.

[0022] Expert Terminal: Installed on the expert's computer. The expert can simultaneously view real-time ultrasound video streams from one or more examination rooms. The expert can select any online examination room, enter the designated audio-visual room, initiate a real-time audio-visual conversation, and provide guidance on the examination; the expert can also receive examination assistance requests from the examination rooms.

[0023] Audio / Video Room Service: Responsible for the creation, maintenance, and destruction of audio / video rooms. It also handles the subscription, publishing, and forwarding of multi-party audio / video streams within the room, ensuring low latency and high smoothness. Used to enable real-time audio / video conversations between the clinic client and specialists.

[0024] In one implementation, experts monitor real-time video feeds from multiple consultation rooms across the department via an expert-facing lobby page. In the grid view of the expert lobby, if the expert notices instability in the ultrasound image acquisition section of a particular consultation room, indicating a need for correction of the technique, the expert can establish an audio-visual room call by dialing that consultation room to provide immediate guidance.

[0025] When doctors at primary care hospitals discover difficult cases during examinations, they can click the "call" button on the consultation room page to request online expert guidance. The expert accepts the request, and the two parties establish an audio-visual room call for guidance and communication.

[0026] In the embodiments of this application, addressing the problems of difficulty in real-time process quality control during ultrasound examinations, limitations of teaching resources in terms of time and space, low efficiency of remote collaboration, and the inability of existing systems to achieve deep integration of real-time quality control, remote consultation, and process teaching, this application provides an integrated solution through a "two-way communication model, grid monitoring, and dedicated audio-visual room interaction." Specifically, it provides a method for remote teaching and quality control of ultrasound examinations based on audio and video, involving an expert terminal, a consultation room terminal, and an audio-visual room server. The expert terminal is used to connect to remotely guiding experts; the consultation room terminal is used to acquire local video streams; and the audio-visual room server is used for the creation, maintenance, and destruction of audio-visual rooms, which are used for multi-party video... The method includes: subscription, publishing, and forwarding of streams; the audio-visual room server acquires real-time ultrasound video streams and doctor's camera video streams collected by the clinic terminal; the audio-visual room server generates a multi-clinic real-time monitoring view based on the real-time ultrasound video streams and the doctor's camera video streams in a grid layout; when the expert terminal initiates a guidance request or the clinic terminal initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the receiver to the dedicated audio-visual room; the audio-visual room server transmits the audio-visual streams of the requester and the receiver bidirectionally through the dedicated audio-visual room, and simultaneously records audio-visual data, interaction timestamps, and operation logs to generate teaching and quality control records. By employing a two-way channel design that integrates expert monitoring and in-clinic consultation, the system overcomes the limitations of existing systems that can only passively respond to requests. Experts can monitor multiple clinics in real time through a grid layout view, promptly identifying operational irregularities and providing immediate guidance. This enables process-based quality control during ultrasound examinations, fundamentally improving the standardization of examinations. Simultaneously, a dedicated audio-visual room provides an immersive, low-latency real-time interactive environment for both parties, making remote guidance feel like being on-site and efficiently resolving the issue of seeking help for difficult cases by primary care physicians. Furthermore, the system synchronously records the entire interaction process, generating teaching and quality control records. This provides real-world case materials for subsequent teaching reviews and operational training, while also enabling quality control traceability. It truly integrates the three major functions of real-time quality control, remote consultation, and process teaching, effectively overcoming time and space limitations, optimizing the allocation of high-quality medical resources, and comprehensively improving the quality of ultrasound diagnosis and treatment as well as the efficiency of talent training.

[0027] The following will further explain a method for remote teaching and quality control of ultrasound examination based on audio and video in this exemplary embodiment.

[0028] As described in step S110, the audio and video room server acquires the real-time ultrasound video stream and the doctor's camera video stream collected by the clinic terminal.

[0029] It should be noted that the real-time ultrasound video stream directly reflects the core operation process of the ultrasound examination, including key quality control information such as probe technique, standard section acquisition, and dynamic changes in images; while the physician's camera video stream records the physician's operation actions and communication status, assisting experts in judging the standardization of operation and the rationality of doctor-patient communication.

[0030] The audio and video room server uses a streaming media transmission protocol to receive two video streams. Before pushing the video streams, the clinic end encodes them (e.g., H.264, H.265 encoding standards) to balance transmission efficiency and image clarity. After receiving the video streams, the server first performs decoding and integrity verification to ensure that the video streams are not lost or distorted. Then, it performs storage and forwarding preprocessing to provide high-quality data support for subsequent grid layout integration, real-time push, and historical tracing. At the same time, it supports parallel reception and classification management of video streams from multiple clinics, ensuring the system's adaptability to multi-clinic scenarios.

[0031] In one embodiment of the present invention, the specific process of step S110, "the audio and video room server acquires the real-time ultrasound video stream and the doctor's camera video stream collected by the clinic terminal," can be further explained in conjunction with the following description.

[0032] As described in the following steps, the audio-visual room server receives the real-time ultrasound video stream and the doctor's camera video stream, which are acquired and encoded by the clinic terminal through a streaming media transmission protocol; wherein, the real-time ultrasound video stream is obtained by the clinic terminal from the output of the local ultrasound equipment, and the doctor's camera video stream is obtained by the clinic terminal from the output of the local camera equipment; It should be noted that the acquisition and transmission of the two video streams must meet the requirements of real-time performance and specificity: the real-time ultrasound video stream output by the local ultrasound equipment (such as an ultrasound diagnostic instrument) contains core diagnostic information such as probe operation trajectory, lesion scanning section, and dynamic changes in ultrasound images, which is the key basis for experts to judge the standardization of operation and provide case guidance; the local camera equipment (such as a high-definition camera) selectively captures the doctor's hand operation movements, probe holding posture, and doctor-patient communication scenarios to assist experts in comprehensively assessing the standardization of the examination process. The streaming media transmission protocol used at the examination room end is preferably Real-Time Transport Protocol (RTP) or Hypertext Transfer Protocol (HTTP) adaptive streaming transmission technology to ensure low-latency transmission of the video stream even in scenarios with network fluctuations; at the same time, the examination room end uses H.264 or H.265 encoding standards to compress and encode the two video streams, reducing transmission bandwidth consumption while ensuring image clarity (resolution not less than 1080P), adapting to the network environment of different medical institutions, and the audio and video room server receives the two video streams through independent stream receiving ports to avoid data interference and ensure transmission stability.

[0033] As described in the following steps, the audio and video room server decodes the received encoded video stream and performs storage and forwarding preprocessing on the decoded video stream.

[0034] It should be noted that the decoding process is strictly compatible with the encoding standard. The audio and video room server uses decoding algorithms corresponding to H.264 or H.265 to decompress and restore the encoded video stream, ensuring that the output video stream is distortion-free and stutter-free, with a frame rate maintained at 25-30fps, meeting the visual requirements of real-time monitoring and guidance. After decoding, the server first performs synchronization verification on the two video streams, using timestamp alignment technology to keep the playback progress of the ultrasound video stream and the doctor's operation video stream consistent, avoiding asynchronous issues caused by transmission delays. The storage preprocessing stage adopts a segmented storage strategy, storing the video stream data in a distributed database by time slices (e.g., every 5 minutes as a segment), while associating metadata such as clinic identification, acquisition time, and equipment information for easy subsequent retrieval and traceability. The forwarding preprocessing stage uses video stream format standardization to convert the decoded video stream into a unified format suitable for grid layout integration and multi-terminal push, while caching the most recent 30 seconds of video stream data to ensure that experts can quickly retrieve and display the data in real time when subscribing, improving the response speed of the monitoring view.

[0035] As described in step S120, the audio-visual room server generates a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream.

[0036] It's important to note that the core of this step is to achieve efficient "one-to-many" monitoring, addressing the pain point that experts cannot simultaneously cover multiple examination rooms. By integrating multiple video streams through a grid layout, it ensures that experts can intuitively observe the examination process in each examination room while avoiding the cumbersome operation and information omissions caused by switching between multiple screens. When generating the monitoring view, real-time performance, image clarity, and layout flexibility must be considered to ensure that experts can quickly identify operational standardization issues in each examination room, providing visual support for subsequent real-time guidance. Simultaneously, it supports dynamic adjustment of the grid layout and the number of monitored examination rooms according to expert needs, adapting to different quality control scenarios.

[0037] In one embodiment of the present invention, the specific process of step S120, "the audio and video room server generates a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream," can be further explained in conjunction with the following description.

[0038] As described in the following steps, when the audio-visual room server detects that an expert has entered the monitoring page, it automatically queries all online clinic information that has been successfully pushed to the audio-visual room server, integrates the clinic information and generates an online clinic list based on the clinic information, and synchronously obtains the target video stream that the online clinic is currently viewing; It should be noted that the detection is triggered by the action signal of the expert opening the monitoring page, eliminating the need for manual querying and improving operational convenience. The integrated clinic information includes key content such as clinic identification, department, online doctor, and examination items. The online clinic list is dynamically updated in real time (e.g., refreshed every 10 seconds) to ensure experts have access to the latest status. The target video stream is acquired through real-time synchronization with the clinic's playback status, accurately capturing the type of video currently focused on by the clinic (ultrasound or doctor's camera), ensuring the expert's perspective aligns with the clinic's operational perspective, facilitating rapid understanding of the core scenario.

[0039] As described in the following steps, the audio and video room server performs image adaptation processing on the target video stream to ensure that only one target video stream is displayed in a single clinic room; It's important to note that the core of the image adaptation is to unify the output specifications of all target video streams: the resolution is uniformly adjusted to 1080P, and image scaling algorithms ensure no stretching or distortion; the frame rate is synchronously calibrated to 25fps to avoid inconsistent playback speeds between video streams from different examination rooms. Simultaneously, detail enhancement processing is optimized for ultrasound video streams (such as improving contrast in lesion areas), and image clarity is optimized for physician camera video streams, ensuring that the display effect of different types of video streams adapts to the viewing needs of experts. Furthermore, the design of a single video stream per examination room avoids information redundancy and improves visual focus when monitoring multiple examination rooms.

[0040] As described in the following steps, the audio and video room server integrates the target video stream in a grid layout and pushes it to the expert terminal, which then renders and displays it in real time to form a real-time monitoring view of multiple clinics. It should be noted that the grid layout adopts an adaptive dynamic adjustment mechanism, automatically matching the optimal layout based on the number of online consultation rooms (e.g., a 2×2 grid for 2-4 consultation rooms, and a 3×2 grid for 5-6 consultation rooms), requiring no manual setup. During integration, each grid is labeled with key consultation room information tags (e.g., consultation room number, examination items), and a video stream status indicator is displayed simultaneously. The push notification uses low-latency transmission technology (latency ≤300ms), allowing experts to click on a grid to zoom in on the video stream of a single consultation room to view details; releasing the button automatically restores the original layout, balancing the needs of global monitoring of multiple consultation rooms with detailed observation of a single consultation room, improving remote quality control efficiency. The video stream push uses low-latency transmission technology (e.g., real-time push based on WebRTC), ensuring that the latency from the completion of audio and video room server integration processing to the expert's display does not exceed 300ms, meeting the needs of real-time monitoring and immediate guidance. The push method supports a combination of "full push" and "on-demand push": During initial subscription, all video streams from the target clinics are pushed in full. If an expert does not interact with a cell for more than 5 minutes, the system automatically reduces the resolution of that cell's video stream (e.g., from 1080P to 720P) to save bandwidth based on the current network status. When the expert clicks on the cell again, high-definition push is immediately restored. After receiving the video stream, the expert's end decodes and displays it in real-time using the built-in rendering engine. The monitoring view interface includes a grid screen area, clinic information labels (each cell displays the clinic name and examination items in the upper right corner), a video stream status indicator (green for normal, yellow for busy, and red for offline), and an operation toolbar (supporting layout switching, clinic switching, screen zoom, screenshots, etc.). Simultaneously, the system supports multiple experts subscribing to the same batch of clinic video streams. The audio / video room server allocates a separate push channel for each expert to avoid mutual interference and ensure a stable and smooth monitoring view for each expert.

[0041] As described in step S130, when the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the receiver to the dedicated audio-visual room.

[0042] It's important to note that this step is crucial for achieving precise "one-to-one" interaction. By creating a dedicated audio-visual room, an isolated and stable real-time communication environment is established between the expert and the clinic staff. This avoids signal interference from multi-scenario interactions while ensuring the privacy of both doctors and patients and the security of medical information. The room creation and access process must balance convenience and security, ensuring that both parties can quickly establish a connection after a request is initiated. Simultaneously, mechanisms such as identifier verification and information synchronization allow both parties to be aware of the interaction context (such as guidance needs and case information) in advance, improving communication efficiency and achieving a remote collaboration effect of "instant response and precise connection."

[0043] In one embodiment of the present invention, the specific process of step S130, "when the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the receiver to the dedicated audio-visual room," can be further explained in conjunction with the following description.

[0044] As described in the following steps, the audio-visual room server receives a guidance request initiated by the expert after selecting a target clinic from the multi-clinic real-time monitoring view; wherein, the guidance request includes the expert identifier, the target clinic identifier, and a description of the guidance requirements; It should be noted that the guidance request is triggered when an expert discovers, through the real-time monitoring view of multiple clinic rooms, that a clinic room has operational irregularities (such as incorrect probe technique or substandard slice acquisition) or requires professional support. The expert triggers the request by clicking the "Call" button for the corresponding clinic room in the monitoring view, making the process convenient and highly targeted. The expert's identifier in the request is a unique identification code (such as an expert's employee ID) used to confirm the expert's identity and permissions. The target clinic's identifier corresponds to the unique code of the clinic requiring guidance, ensuring accurate recipient identification. The guidance request description is briefly filled out by the expert (such as "probe position adjustment" or "standard slice acquisition guidance") or a preset template is selected, allowing the clinic to know the guidance direction in advance and prepare for communication. After receiving the request, the audio / video room server will first verify the expert's permissions (such as whether they have the necessary ultrasound guidance qualifications for the department) and the target clinic's status (whether it is busy or the video stream is normal). Only after successful verification will the process proceed to the next step, avoiding invalid requests consuming system resources.

[0045] As described in the following steps, the audio / video room server creates a dedicated audio / video room according to the guidance request and generates unique room identification information; It should be noted that the dedicated audio / video room is an independent virtual communication space, completely isolated from other rooms, ensuring the privacy and communication stability of the guidance process and avoiding interference from other interaction scenarios. The room identification information uses a combination of "timestamp + random characters" encoding (e.g., "202405201430_8F7D2"), which is globally unique and can quickly associate the requester, receiver, and interaction information corresponding to the room. When creating a room, the system automatically configures parameters adapted to real-time audio / video interaction, including bandwidth priority (allocating high bandwidth resources to the dedicated room), transmission protocol (using WebRTC protocol by default to ensure low latency), audio sampling rate (48kHz), and video resolution (default 1080P, supports dynamic adjustment), etc. Simultaneously, it initializes the audio / video stream receiving / forwarding channels within the room, preparing for subsequent access by both parties. Furthermore, the system associates and stores the room identification with information such as the expert's identifier, the target clinic's identifier, and the guidance requirements in the guidance request, facilitating subsequent quality control traceability and record retrieval.

[0046] As described in the following steps, the audio and video room server sends the room identification information to the expert terminal and the target consultation room terminal respectively, and pushes a guidance call notification containing the room identification information, the expert terminal identification, and the guidance needs description to the target consultation room terminal; It should be noted that room identification information is sent via an encrypted communication channel to prevent tampering or theft during transmission, ensuring room access security. The information sent to the expert only includes the room identification and a "room created successfully" message, facilitating quick identification and access by the expert. Guidance call notifications pushed to the target clinic must include complete information: the room identification for access verification, the expert identification (including the expert's name, title, and area of ​​expertise) to inform the clinic of the expert's qualifications, and a description of the guidance needs to help the clinic prepare relevant examination materials and address current operational issues in advance, improving the relevance of the guidance. Call notifications support multi-channel synchronization (such as system pop-ups, sound alerts, and SMS notifications) to ensure clinic physicians are promptly notified and avoid missing guidance requests; the notification also includes "Enter Teaching" and "Do Not Teach" options. The clinic can respond flexibly based on the progress of the examination.

[0047] As described in the following steps, the audio-visual room server receives access requests sent by the expert and the target clinic based on the room identification information, verifies the identification information of both parties, and connects both parties to the exclusive audio-visual room after successful verification.

[0048] It should be noted that the access request must include the requester's identifier (expert identifier or clinic identifier) ​​and room identifier information. The audio and video room server adopts a "dual verification" mechanism: first, it verifies whether the room identifier exists and is in an expired state (the room is valid for 10 minutes after creation by default, and will be automatically destroyed if no access is made within the time limit); second, it verifies whether the requester's identifier is consistent with the identifier associated with the room (e.g., when an expert accesses, it verifies whether its identifier matches the expert identifier when the room was created), ensuring that only authorized objects can access the room and preventing unauthorized intrusion. After successful verification, the system will allocate independent audio and video stream transmission ports to both parties, establish a point-to-point low-latency communication link, and simultaneously close the monitoring view of the corresponding clinic on the expert's end to avoid conflicts between the monitoring stream and the room interaction stream; after successful access, a notification of successful entry into the room is sent to both parties, and the audio and video acquisition and transmission functions are automatically enabled. Experts and clinic physicians can directly communicate by voice and view video images (e.g., the expert points to key points on the ultrasound image, and the clinic physician demonstrates the operation technique), achieving immersive guidance.

[0049] In one embodiment of the present invention, the audio-visual room server receives a help request initiated by the target expert terminal corresponding to the online expert selected by the consultation room terminal; wherein, the help request includes the consultation room terminal identifier, the target expert terminal identifier, basic case information, and a description of the help problem; The audio and video room server creates a dedicated audio and video room and generates a unique room identifier based on the request for help; The audio and video room server sends the room identification information to the consultation room terminal and the target expert terminal respectively, and at the same time pushes a help call notification containing room identification information, consultation room terminal identification, basic case information and a description of the help problem to the target expert terminal. The audio-visual room server receives the access request sent by the target expert after accepting the call, as well as the access request sent by the clinic. It verifies the identification information of both parties, and after successful verification, connects both parties to the exclusive audio-visual room.

[0050] It should be noted that this process is designed for scenarios where clinic staff proactively initiate requests for assistance. Its core purpose is to meet the emergency support needs of primary care physicians in complex case diagnoses and special procedures. The basic case information in the request should be concise and crucial, including the patient's gender, age, main symptoms, examination site, preliminary diagnosis, and optional screenshots of acquired ultrasound images. The target expert identifier is selected by the clinic staff from a list of online experts recommended by the system (matched by department and area of ​​expertise). The description of the request must be clear and specific (e.g., "suspected myocardial infarction, ultrasound view unclear," "fetal position unusual, standard biparietal diameter view unavailable"), helping experts quickly grasp the condition and core of the request, and prepare guidance plans in advance. The rules for creating dedicated audio / video rooms and room identifiers are consistent with those for expert-initiated guidance requests, ensuring system logic consistency. The request notification pushed to the target expert includes complete case and request information, facilitating the expert's assessment of their guidance capabilities and prompt response (supporting "Enter Teaching" and "Do Not Teach Temporarily" options). The verification mechanism for both parties to connect is the same as described above. After successful connection, audio and video interaction is automatically enabled. The consultation room can display the ultrasound operation process and images in real time. Experts can provide targeted guidance based on case information to efficiently solve the problem of seeking help. At the same time, the system will record data such as the response time of seeking help and the speed of expert access throughout the process, which will serve as the basis for expert resource scheduling and service quality evaluation.

[0051] As described in step S140, the audio and video room server transmits the audio and video streams of the requester and the receiver bidirectionally through the dedicated audio and video room, and synchronously records audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

[0052] It should be noted that this step is the core of achieving effective remote guidance and quality control traceability. Two-way transmission must ensure low latency, high definition, and synchronization of audio and video streams to ensure that the interaction between experts and the clinic is as if they were "on-site." Synchronous recording must completely retain key information from the entire interaction process, providing real case materials for subsequent teaching and review, as well as objective evidence for ultrasound examination quality review and medical dispute tracing, truly achieving closed-loop management of "effective guidance, recorded quality control, and supported traceability."

[0053] In one embodiment of the present invention, the specific process of step S140, "the audio and video room server transmits the audio and video streams of the requester and the receiver bidirectionally through the dedicated audio and video room, and synchronously records audio and video data, interaction timestamps and operation logs, and generates teaching and quality control records," can be further explained in conjunction with the following description.

[0054] As described in the following steps, the audio and video room server establishes a low-latency bidirectional communication channel between the requester and the receiver within the dedicated audio and video room; It should be noted that the communication channel adopts a hybrid architecture combining point-to-point (P2P) and server-side forwarding: when network conditions permit for both the requester and receiver, a direct P2P connection is established first to minimize transmission latency (controlled within 100-300ms); when network limitations exist (such as cross-network segment or firewall blocking), the system automatically switches to server-side forwarding mode, using high-performance forwarding nodes on the audio / video room server to ensure communication stability. When the channel is established, the system automatically configures a bandwidth adaptive mechanism, adjusting the bitrate of the audio / video streams in real time based on the network bandwidth of both parties (dynamically adapting from 500kbps to 8Mbps) to avoid stuttering or disconnections caused by bandwidth fluctuations; simultaneously, an encrypted transmission protocol (such as SRTP) is enabled for the channel, providing end-to-end encryption of audio and video data to prevent data theft or tampering during transmission, ensuring patient and medical information privacy and security. Furthermore, the channel supports dynamic expansion; if a third party (such as other experts or students) needs to be added during interaction, access nodes can be added quickly without affecting the original bidirectional transmission quality.

[0055] As described in the following steps, the audio and video room server receives the audio and video streams published by the requesting party in real time, and after decoding, noise reduction and synchronization optimization of the audio and video streams, forwards them to the receiving party; It should be noted that the audio and video streams sent by the requesting party (expert or clinic end) are encoded and compressed data. After receiving the audio and video streams, the server in the audio and video room first decodes them according to the corresponding encoding standard (H.264 / H.265 audio AAC) to restore the original audio and video data. Noise reduction processing is optimized for different data types: audio uses an adaptive noise suppression algorithm to filter environmental noise (such as clinic equipment noise, background voices) and network noise, improving speech clarity; video uses Gaussian filtering and inter-frame denoising technology to reduce image noise and optimize image quality, while preserving the detailed features of ultrasound images (such as lesion edges and blood flow signals) so as not to affect diagnostic judgment. Synchronization optimization is a key step. The server extracts the timestamps from the audio and video streams and uses linear interpolation to correct the audio-visual asynchrony caused by transmission delay, ensuring that the voice and image actions are accurately matched (audio-visual synchronization error ≤50ms); for ultrasound video streams, additional frame alignment processing is added to ensure that the ultrasound images seen by the expert are completely synchronized with the real-time operations at the clinic end, facilitating accurate guidance of probe movement, section adjustment, and other operations. After processing, the audio and video streams are re-encoded and forwarded according to the recipient's network status and device capabilities to ensure smooth playback for the recipient.

[0056] As described in the following steps, the audio and video room server synchronously receives the audio and video stream published by the recipient, and after decoding, noise reduction and synchronization optimization of the audio and video stream, forwards it to the requester; It should be noted that this step is consistent with the requester's audio and video stream processing logic, ensuring that the audio and video data received by both parties are of equal quality and that the interactive experience is balanced. The video stream sent by the receiving party (clinic end or expert end) may contain key information such as ultrasound image details and close-ups of operating techniques. During video processing, it is crucial to ensure the stability of the image resolution and frame rate (the frame rate of the ultrasound video stream should not be less than 25fps) to avoid loss of details due to processing. Audio processing prioritizes preserving the integrity and recognizability of the speech, supporting full-duplex communication to ensure that both parties can speak simultaneously without lag. During the synchronization optimization process, the server will uniformly calibrate the time base of the audio and video streams of both parties, ensuring that the playback progress of the requester and the receiver is completely consistent. For example, when the expert points to a certain location on the ultrasound image and explains, the clinic end can simultaneously see the annotation and hear the explanation, achieving precise interaction. If one party experiences a brief network fluctuation, the server will activate a caching compensation mechanism, temporarily caching 3-5 seconds of audio and video data, and smoothly playing it after the network recovers, avoiding frame skipping or audio interruption.

[0057] As described in the following steps, the audio and video room server synchronously records the audio and video data, interaction timestamps, and operation logs in the room, and generates teaching and quality control records.

[0058] It should be noted that the recording process is synchronized with bidirectional transmission, employing a real-time "record while transmitting" mechanism to ensure no data omissions or delays. The recorded audio and video data are optimized, complete streaming data, preserving the entire interactive process's visuals and audio, and reproducing key scenarios such as expert guidance details (e.g., ultrasound image annotation, operational technique correction suggestions) and the clinic's response process (e.g., adjusting probe position according to guidance, asking questions and exchanging information). Interaction timestamps are accurate to the millisecond level, including room access time, audio and video stream start time, key operation trigger times (e.g., expert initiating annotation, clinic switching ultrasound views), and room exit time, providing precise evidence for subsequent time-series analysis. The operation log comprehensively records system behavior and user operations, including both parties' identifiers, room identifiers, network status changes, bitrate adjustment records, and function usage records (e.g., screenshots, recordings, annotations), forming a complete interactive behavior trajectory. All recorded data is stored in real-time in a correlated manner, ensuring a one-to-one correspondence between audio / video, timestamps, and operation logs, providing complete data support for teaching and quality control.

[0059] In one embodiment of the present invention, the specific process of step "the audio and video room server synchronously records the audio and video data, interaction timestamps and operation logs in the room, and generates teaching and quality control records" can be further explained in conjunction with the following description.

[0060] As described in the following steps, the audio and video room server will associate and integrate the recorded audio and video data, interaction timestamps and operation logs, and package them into standardized teaching and quality control record files according to a preset format; It should be noted that the association and integration uses timestamps as the core index, accurately binding audio and video clips and operation log entries at the same time point. This ensures that during playback, the video, audio, and corresponding operation records can be viewed simultaneously. For example, when playing back to the time point of expert annotation of ultrasound images, the log entry "Expert initiated image annotation" will be displayed simultaneously. The record files are packaged in a standardized format: audio and video data are packaged in MP4 format (compatible with mainstream players), and operation logs are packaged in JSON format (facilitating computer parsing and data analysis). The entire package is packaged into a dedicated file with the ".ucr" extension (including a file header, data area, and checksum). The file header contains metadata such as clinic identifier, expert identifier, interaction time, and case information. The data area stores the integrated audio, video, and log data, and the checksum is used to verify file integrity and prevent tampering. During the packaging process, the system automatically marks key teaching links and quality control checkpoints: key teaching links include scenarios such as expert knowledge explanation, operation skill demonstration, and question answering (automatically identified through voice keyword recognition and screen action analysis); quality control checkpoints include the standardization of ultrasound section acquisition, probe operation techniques, and the standardization of doctor-patient communication (automatically marked in conjunction with preset quality control standards). The marked information is embedded in the file and can be quickly located and viewed during playback.

[0061] As described in the following steps, the audio and video room server stores the teaching and quality control record files and supports retrieval and retrieval by keywords such as clinic identifier, expert identifier, time range, and case information.

[0062] It should be noted that the storage employs a dual backup mechanism of "local + cloud": local storage is located in a distributed database on the audio / video room server to ensure fast retrieval and access; cloud storage is located on an encrypted cloud server to achieve off-site disaster recovery backup and prevent data loss due to local device failure. Hierarchical access control is implemented for files during storage. For example, department administrators can view all records within their department, specialists can only view their own interaction records, and general practitioners can only view quality control records relevant to them, ensuring data privacy. The search function supports multi-dimensional combined queries. Users can enter single keywords (such as clinic number "0305" or expert name "Zhang San") or combined keywords (such as "cardiovascular myocardial infarction cases in May 2024"). The system returns matching results within 1 second and displays core file information (such as interaction time, case summary, and teaching / quality control labels). The access function supports online playback, download and export, and sharing. During online playback, users can adjust playback speed, jump to key points, and view synchronized logs. Download and export support separate export of MP4 audio / video files and JSON log files for offline analysis or teaching purposes. Sharing and forwarding require permission verification, and only authorized users can receive and view the data to ensure data security. In addition, the system supports statistical analysis of the recorded files, automatically generating quality control reports (such as the pass rate of a clinic's operating procedures and the frequency of expert guidance) and a teaching resource library, providing data support for medical institutions to optimize the quality of ultrasound diagnosis and treatment and improve the talent training system.

[0063] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0064] Reference Figure 2 This application illustrates an embodiment of a device for remote teaching and quality control of ultrasound examinations based on audio and video. The device is used to implement the steps of the method for remote teaching and quality control of ultrasound examinations based on audio and video as described in any of the preceding claims. The device involves an expert terminal, a consultation room terminal, and an audio / video room server. The expert terminal is used to connect to a remotely guiding expert. The consultation room terminal is used to acquire local video streams. The audio / video room server is used to create, maintain, and destroy audio / video rooms, and the audio / video rooms are used for subscribing to, publishing, and forwarding multi-party video streams. The device includes: The acquisition module 210 is used by the audio and video room server to acquire the real-time ultrasound video stream and the doctor's camera video stream collected by the clinic terminal; The monitoring view module 220 is used by the audio and video room server to generate a real-time monitoring view of multiple clinic rooms in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. The dedicated audio and video room module 230 is used to create a dedicated audio and video room and connect the requester and the receiver to the dedicated audio and video room when the expert initiates a guidance request or the clinic initiates a help request. The bidirectional transmission module 240 is used by the audio and video room server to bidirectionally transmit the audio and video streams of the requester and the receiver through the dedicated audio and video room, and synchronously record audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

[0065] In one embodiment of the present invention, the acquisition module 210 includes: The video stream submodule is used by the audio and video room server to receive the real-time ultrasound video stream and the doctor's camera video stream, which are acquired and encoded by the clinic terminal through a streaming media transmission protocol; wherein, the real-time ultrasound video stream is obtained by the clinic terminal from the output of the local ultrasound equipment, and the doctor's camera video stream is obtained by the clinic terminal from the output of the local camera equipment; The decoding submodule is used by the audio and video room server to decode the received encoded video stream and perform storage and forwarding preprocessing on the decoded video stream.

[0066] In one embodiment of the present invention, the monitoring view module 220 includes: The consultation room list submodule is used to automatically query all online consultation room information that has been successfully pushed to the audio and video room server when the audio and video room server detects that the expert has entered the monitoring page, integrate the consultation room information and generate an online consultation room list based on the consultation room information, and synchronously obtain the target video stream that the online consultation room is currently viewing; The target video stream submodule is used by the audio and video room server to perform image adaptation processing on the target video stream to ensure that only one target video stream is displayed in a single clinic room. The real-time monitoring module is used by the audio and video room server to integrate the target video stream in a grid layout and push it to the expert terminal, which then renders and displays it in real time to form a real-time monitoring view of multiple clinic rooms.

[0067] In one embodiment of the present invention, the dedicated audio-visual room module 230 includes: The guidance request submodule is used by the audio-visual room server to receive a guidance request initiated by the expert terminal after selecting a target clinic from the multi-clinic real-time monitoring view; wherein, the guidance request includes the expert terminal identifier, the target clinic terminal identifier, and a description of the guidance requirements; The room identification information submodule is used by the audio and video room server to create a unique audio and video room according to the guidance request and generate unique room identification information; The guidance call submodule is used by the audio and video room server to send the room identification information to the expert terminal and the target clinic terminal respectively, and to push a guidance call notification containing room identification information, expert terminal identification and guidance requirements to the target clinic terminal; The access submodule is used by the audio and video room server to receive access requests sent by the expert terminal and the target clinic terminal based on the room identification information, verify the identification information of both parties, and connect both parties to the exclusive audio and video room after successful verification.

[0068] In one embodiment of the present invention, the dedicated audio-visual room module 230 includes: The help request submodule is used by the audio and video room server to receive a help request initiated by the target expert terminal corresponding to the online expert selected by the consultation room terminal; wherein, the help request includes the consultation room terminal identifier, the target expert terminal identifier, basic case information and a description of the help problem; The room identification information submodule is used by the audio and video room server to create a unique audio and video room and generate unique room identification information based on the help request. The help call submodule is used by the audio and video room server to send the room identification information to the consultation room terminal and the target expert terminal respectively, and at the same time push a help call notification containing room identification information, consultation room terminal identification, basic case information and a description of the help problem to the target expert terminal. The access submodule is used by the audio and video room server to receive the access request sent by the target expert after accepting the call, and the access request sent by the clinic, to verify the identification information of both parties, and to connect both parties to the exclusive audio and video room after successful verification.

[0069] In one embodiment of the present invention, the bidirectional transmission module 240 includes: The bidirectional communication channel submodule is used by the audio and video room server to establish a low-latency bidirectional communication channel between the requester and the receiver in the dedicated audio and video room. The bidirectional communication channel submodule is used by the audio and video room server to receive the audio and video streams published by the requesting party in real time, and to forward the audio and video streams to the receiving party after decoding, noise reduction and synchronization optimization. The synchronous receiving submodule is used by the audio and video room server to synchronously receive the audio and video streams published by the receiver, and after decoding, noise reduction and synchronization optimization of the audio and video streams, forward them to the requester. The synchronous recording submodule is used by the audio and video room server to synchronously record the audio and video data, interaction timestamps and operation logs in the room, and generate teaching and quality control records.

[0070] Reference Figure 3 The illustration shows a computer device for implementing a method for remote teaching and quality control of ultrasound examination based on audio and video according to the present invention, which may specifically include the following: The aforementioned computer device 1 is in the form of a general-purpose computing device. The components of the computer device 1 may include, but are not limited to: one or more processors or processing units 3, memory 8, and a bus 4 connecting different system components (including memory 8 and processing unit 3).

[0071] Bus 4 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Audio / Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0072] Computer device 1 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 1, including volatile and non-volatile media, removable and non-removable media.

[0073] Memory 8 may include computer system readable media in the form of volatile memory, such as random access memory 9 and / or cache memory 10. Computer device 1 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 11 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Although Figure 3 As not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 4 via one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 13 configured to perform the functions of the embodiments of this application.

[0074] A program / utility 12 having a set (at least one) of program modules 13 may be stored, for example, in memory. Such program modules 13 include—but are not limited to—an operating system, one or more application programs, other program modules 13, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 13 typically perform the functions and / or methods described in the embodiments of this application.

[0075] Computer device 1 can also communicate with one or more external devices 2 (e.g., keyboard, pointing device, monitor 7, camera, etc.), and with one or more devices that enable an operator to interact with computer device 1, and / or with any device that enables computer device 1 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through I / O interface 6. Furthermore, computer device 1 can also communicate with one or more networks (e.g., local area network (LAN)), wide area network (WAN), and / or public networks (e.g., the Internet) through network adapter 5. Figure 3 As shown, network adapter 5 communicates with other modules of computer device 1 via bus 4. It should be understood that, although... Figure 3 Not shown, it can be combined with computer device 1 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing unit 3, external disk drive array, RAID system, tape drive and data backup storage system 11, etc.

[0076] The processing unit 3 executes various functional applications and data processing by running programs stored in memory 8, such as implementing a method for remote teaching and quality control of ultrasound examination based on audio and video provided in the embodiments of this application.

[0077] That is, when the above-mentioned processing unit 3 executes the above-mentioned program, the audio and video room server acquires the real-time ultrasound video stream and the doctor's camera video stream collected by the clinic terminal; The audio-visual room server generates a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. When the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the recipient to the dedicated audio-visual room. The audio and video room server transmits audio and video streams bidirectionally between the requester and the receiver through the dedicated audio and video room, and simultaneously records audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

[0078] In this application embodiment, the application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for remote teaching and quality control of ultrasound examination based on audio and video, as provided in all embodiments of the application.

[0079] That is, when the program is executed by the processor, the following is achieved: the audio and video room server acquires the real-time ultrasound video stream and the doctor's camera video stream collected by the clinic terminal; The audio-visual room server generates a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. When the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the recipient to the dedicated audio-visual room. The audio and video room server transmits audio and video streams bidirectionally between the requester and the receiver through the dedicated audio and video room, and simultaneously records audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

[0080] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0081] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0082] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as a standalone software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the operator's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably.

[0083] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0084] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0085] The above provides a detailed description of the method and apparatus for remote teaching and quality control of ultrasound examination based on audio and video provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for remote teaching and quality control of ultrasound examination based on audio and video, characterized in that, The method involves an expert terminal, a consultation room terminal, and an audio / video room server terminal; the expert terminal is used to connect to experts providing remote guidance; the consultation room terminal is used to capture local video streams; The audio / video room server is used for creating, maintaining, and destroying audio / video rooms, and the audio / video rooms are used for subscribing to, publishing, and forwarding multi-party video streams; the method includes: The audio and video room server acquires real-time ultrasound video streams and doctor's camera video streams collected from the examination room. The audio-visual room server generates a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. When the expert initiates a guidance request or the clinic initiates a help request, the audio-visual room server creates a dedicated audio-visual room and connects the requester and the recipient to the dedicated audio-visual room. The audio and video room server transmits audio and video streams bidirectionally between the requester and the receiver through the dedicated audio and video room, and simultaneously records audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

2. The method according to claim 1, characterized in that, The steps for the audio-visual room server to acquire the real-time ultrasound video stream and doctor's camera video stream collected by the examination room terminal include: The audio-visual room server receives real-time ultrasound video streams and doctor's camera video streams acquired and encoded by the clinic terminal through a streaming media transmission protocol; wherein, the real-time ultrasound video stream is obtained by the clinic terminal from the output of the local ultrasound equipment, and the doctor's camera video stream is obtained by the clinic terminal from the output of the local camera equipment; The audio and video room server decodes the received encoded video stream and performs storage and forwarding preprocessing on the decoded video stream.

3. The method according to claim 1, characterized in that, The step of the audio-visual room server generating a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream includes: When the audio-visual room server detects that an expert has entered the monitoring page, it automatically queries all online clinic information that has been successfully pushed to the audio-visual room server, integrates the clinic information and generates an online clinic list based on the clinic information, and simultaneously obtains the target video stream that the online clinic is currently viewing; The audio and video room server performs image adaptation processing on the target video stream to ensure that only one target video stream is displayed in a single consultation room. The audio and video room server integrates the target video stream in a grid layout and pushes it to the expert terminal, which then renders and displays it in real time to form a real-time monitoring view of multiple clinic rooms.

4. The method according to claim 1, characterized in that, When the expert initiates a guidance request or the clinic initiates a help request, the audio / video room server creates a dedicated audio / video room and connects the requester and the recipient to the dedicated audio / video room, including the following steps: The audio-visual room server receives a guidance request initiated by the expert after selecting a target clinic from the multi-clinic real-time monitoring view; wherein, the guidance request includes the expert identifier, the target clinic identifier, and a description of the guidance requirements; The audio / video room server creates a dedicated audio / video room based on the guidance request and generates unique room identification information; The audio and video room server sends the room identification information to the expert terminal and the target consultation room terminal respectively, and pushes a guidance call notification containing the room identification information, the expert terminal identification, and the guidance needs description to the target consultation room terminal. The audio-visual room server receives access requests sent by the expert and the target clinic based on the room identification information, verifies the identification information of both parties, and connects both parties to the exclusive audio-visual room after successful verification.

5. The method according to claim 1, characterized in that, When the expert initiates a guidance request or the clinic initiates a help request, the audio / video room server creates a dedicated audio / video room and connects the requester and the recipient to the dedicated audio / video room, including the following steps: The audio-visual room server receives a help request initiated by the target expert terminal after the consultation room terminal selects the online expert; wherein, the help request includes the consultation room terminal identifier, the target expert terminal identifier, basic case information, and a description of the help problem; The audio and video room server creates a dedicated audio and video room and generates a unique room identifier based on the request for help; The audio and video room server sends the room identification information to the consultation room terminal and the target expert terminal respectively, and at the same time pushes a help call notification containing room identification information, consultation room terminal identification, basic case information and a description of the help problem to the target expert terminal. The audio-visual room server receives the access request sent by the target expert after accepting the call, as well as the access request sent by the clinic. It verifies the identification information of both parties, and after successful verification, connects both parties to the exclusive audio-visual room.

6. The method according to claim 1, characterized in that, The audio / video room server transmits audio and video streams bidirectionally between the requester and the receiver through the dedicated audio / video room, and synchronously records audio and video data, interaction timestamps, and operation logs to generate teaching and quality control records. The steps include: The audio and video room server establishes a low-latency, two-way communication channel between the requester and the receiver within the dedicated audio and video room. The audio and video room server receives the audio and video streams published by the requesting party in real time, and after decoding, noise reduction and synchronization optimization of the audio and video streams, forwards them to the receiving party; The audio and video room server synchronously receives the audio and video streams published by the recipient, and after decoding, noise reduction and synchronization optimization of the audio and video streams, forwards them to the requester. The audio and video room server synchronously records the audio and video data, interaction timestamps, and operation logs within the room, generating teaching and quality control records.

7. The method according to claim 6, characterized in that, The steps for the audio and video room server to synchronously record audio and video data, interaction timestamps, and operation logs within the room, and to generate teaching and quality control records, include: The audio and video room server will associate and integrate the recorded audio and video data, interaction timestamps, and operation logs, and package them into standardized teaching and quality control record files according to a preset format. The audio and video room server stores the teaching and quality control record files and supports retrieval and retrieval by keywords such as clinic identifier, expert identifier, time range, and case information.

8. A device for remote teaching and quality control of ultrasound examination based on audio and video, characterized in that, The device for remote teaching and quality control of ultrasound examination based on audio and video is used to implement the steps of the method for remote teaching and quality control of ultrasound examination based on audio and video as described in any one of claims 1 to 7. The device involves an expert terminal, a consultation room terminal, and an audio and video room server terminal; the expert terminal is used to connect to a remotely guiding expert; the consultation room terminal is used to acquire local video streams. The audio / video room server is used for creating, maintaining, and destroying audio / video rooms, and the audio / video rooms are used for subscribing to, publishing, and forwarding multi-party video streams; the device includes: The acquisition module is used by the audio and video room server to acquire the real-time ultrasound video stream and doctor's camera video stream collected by the clinic terminal; The monitoring view module is used by the audio and video room server to generate a multi-room real-time monitoring view in a grid layout based on the real-time ultrasound video stream and the doctor's camera video stream. The dedicated audio and video room module is used to create a dedicated audio and video room and connect the requester and the receiver to the dedicated audio and video room when the expert initiates a guidance request or the clinic initiates a help request. The bidirectional transmission module is used by the audio and video room server to bidirectionally transmit the audio and video streams of the requester and the receiver through the dedicated audio and video room, and synchronously record audio and video data, interaction timestamps and operation logs to generate teaching and quality control records.

9. An extraction device for remote teaching and quality control of ultrasound examination based on audio and video, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method for remote teaching and quality control of ultrasound examination based on audio and video as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the audio-visual-based remote teaching and quality control method for ultrasound examination as described in any one of claims 1 to 7.

Images