Multi-screen interaction method based on dynamic network adjustment and suitable for multiple teaching scenes

Abstract

The invention provides a multi-screen interaction method based on dynamic network adjustment, which is suitable for various teaching scenes and realizes screen sharing and real-time interaction between a teacher and electronic equipment of a plurality of students by using Android equipment. According to the method, the sending frequency of heartbeat information is dynamically adjusted by integrating real-time network conditions and preset teaching interaction requirements, and the adaptive code rate and multicast protocol selection are gradually optimized so as to adapt to different network conditions and teaching scene requirements and ensure high-efficiency, low-delay and high-quality display of content synchronization. The method gets rid of dependence on special hardware, the system cost is remarkably reduced, the expansibility of the system and the user interaction experience are enhanced, and a flexible and reliable interaction solution is provided for a large-scale and multi-terminal teaching scene.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a multi-screen interaction method suitable for multiple teaching scenarios based on dynamic network adjustment. Background Art

[0002] With the rapid development of digital teaching and distance learning, the demand for multi-screen interaction in teaching environments is increasing. Multi-screen interactive technology allows teachers, students and other participants to share and synchronously display content on multiple devices to enhance the interactivity and learning experience of classroom teaching. However, traditional multi-screen interactive systems often rely on dedicated hardware devices, resulting in high equipment costs, insufficient scalability, and obvious deficiencies in real-time performance, synchronization accuracy and network transmission stability, which greatly affects the effectiveness of classroom teaching. Especially in different teaching scenarios, such as ordinary classrooms, large lectures and remote teaching in remote areas, the adaptability and performance of the system vary greatly, making it difficult to meet complex and changing needs.

[0003] Therefore, there is an urgent need for a multi-screen interactive solution that can solve the problems of insufficient real-time performance, low synchronization accuracy and high cost in the prior art by optimizing data transmission and display technology, and can flexibly respond to various teaching scenarios. The multi-screen interactive method of the present application can not only reduce the dependence on hardware, but also adjust the heartbeat information sending frequency according to the real-time network conditions and the preset degree of interaction of the teaching scene, and then perform adaptive bit rate adjustment and diversified network protocol selection in turn, thereby ensuring that the system provides efficient, real-time and reliable multi-screen interaction in different network environments, and improves the applicability and stability in various teaching scenarios. Summary of the invention

[0004] The main purpose of this application is to provide a multi-screen interactive method suitable for multiple teaching scenarios based on dynamic network adjustment, to solve the limitations of the multi-screen interactive system in the prior art in terms of limited applicable scenarios, insufficient real-time performance, and low synchronization accuracy. This application adjusts the heartbeat information sending frequency, data transmission mode, adaptive bit rate and other technologies according to the real-time network conditions and the preset degree of interaction in the teaching scenario, to ensure that the system can operate efficiently in different teaching scenarios such as ordinary classrooms, large lectures, and distance learning, to meet various network environments and teaching needs, and to provide a stable, low-latency, high-quality interactive experience, thereby improving adaptability and reliability in complex and changing teaching scenarios.

[0005] A multi-screen interaction method based on dynamic network adjustment suitable for multiple teaching scenarios is provided, which can realize multi-screen interaction between teachers and multiple student electronic devices in different teaching scenarios, and specifically includes the following steps: S1. Device activation: activating the teacher's electronic device and the student's electronic device through the same activation code. Each device generates and stores a network configuration information containing a unique device identifier (MAC address) and IP address. S2, the teacher's electronic device sends a heartbeat message to each student's electronic device regularly. Each student's electronic device verifies the timestamp of the heartbeat message to determine whether the teacher's electronic device and other student electronic devices are online; S3. According to the real-time network conditions and the preset teaching scene interaction level, determine whether the heartbeat information sending frequency meets the requirements. If yes, proceed to step S4; if not, adjust the heartbeat information sending frequency until it meets the requirements; S4. When receiving the broadcast instruction from the teacher's electronic device, each student electronic device joins a multicast group identified by the multicast address and port to receive the teacher's screen content. The teacher's electronic device uses Android's MediaProjection API to start screen capture, encodes the screen content through the hardware encoder, and sends the encoded frame data to the multicast group. Each student electronic device listens to the multicast stream, decodes the video data and displays the teacher's screen content on the student's electronic device; S5. Further determine whether the adaptive bit rate meets the requirements based on the image quality and heartbeat information of the electronic device. If yes, proceed to step S6; if not, adjust the adaptive bit rate until the image quality requirements of the teaching content are met; S6. Further determine whether the adopted multicast protocol is appropriate based on the adjusted heartbeat information sending frequency and adaptive bit rate. If it is appropriate, proceed to step S7; if it is not appropriate, adjust the multicast protocol until the image quality requirements of the teaching content are met; S7, the teacher's electronic device executes step S4 again to detect whether the image quality of the teaching content meets the requirements. If yes, it goes to step S8, if not, it goes to step S5 again; S8. The teacher electronic device sends a request to specify a specific student electronic device to send its screen content via unicast. After receiving the request, the student electronic device will use Android's MediaProjection API to capture its screen content and encode the content into the format required by the teacher. After the encoding is completed, the student electronic device transmits the encoded content to the teacher electronic device. The teacher electronic device receives and decodes the content, and finally displays the student's screen content on the teacher electronic device; S9. Continuously monitor the online status of each device to ensure that all devices in the classroom environment are synchronized and meet teaching requirements.

[0006] The multi-screen interactive method of the present application can flexibly optimize the performance of the system according to different teaching scenarios (such as high-demand scenarios, standard usage scenarios, and low-bandwidth environments) by dynamically adjusting the heartbeat information frequency, adaptive bit rate, and multicast protocol selection, thereby enhancing the user's interactive experience.

[0007] The beneficial technical effects of this application are as follows.

[0008] 1. Optimized adaptability to various teaching scenarios.

[0009] The method of this application is suitable for a variety of teaching scenarios such as ordinary classrooms, large lectures, and remote teaching in remote areas, and further optimizes the switching and adaptability across scenarios. By dynamically identifying the current teaching scenario and real-time user interaction needs, the system can automatically switch the optimal settings between scenarios to achieve efficient real-time interaction and content synchronization. In addition, the system can achieve multi-device synchronous interaction without the support of additional hardware equipment, and effectively reduce delays in large-scale, multi-device scenarios, ensuring high-quality content sharing and smooth classroom experience.

[0010] 2. The frequency of sending heartbeat information is adjusted and optimized, taking into account the degree of teaching interaction and real-time network status.

[0011] The method of the present application dynamically adjusts the frequency of sending heartbeat information by comprehensively considering the preset teaching interaction level and real-time network conditions, ensuring that the system manages the device status more efficiently in different interactive scenarios. For example, in high-interaction scenarios, the system appropriately increases the frequency of sending heartbeat information based on network latency and the number of devices to ensure rapid response and smoothness of content transmission; in low-interaction or bandwidth-constrained scenarios, the system will intelligently reduce the frequency of sending heartbeat information to reduce the network burden while ensuring real-time monitoring of device status. This method combines dynamic adjustments made by interactive needs and network status to improve the system's adaptability and resource utilization efficiency, and effectively ensure the stable synchronization of classroom content.

[0012] 3. Optimize and adjust the heartbeat information frequency, adaptive bit rate and multicast protocol one by one.

[0013] The method of the present application adopts a phased optimization and adjustment method one by one, so that the heartbeat information sending frequency, adaptive bit rate, and multicast protocol can work together more efficiently. First, the system judges the device status and network stability through the heartbeat information sending frequency, and provides data basis for subsequent adaptive bit rate adjustment. Subsequently, the adaptive bit rate automatically adjusts the video clarity and transmission smoothness according to the network quality, and feeds back to the multicast protocol selection module, and selects the most appropriate protocol to ensure the stability and efficiency of large-scale device synchronization. The step-by-step adjustment of the three improves the synchronization accuracy while reducing the network pressure, ensures the continuity and high-quality display of the teaching content, and effectively avoids delays and content loss even in unstable networks. This one-by-one optimization adjustment strategy dynamically balances the transmission efficiency and display quality, bringing users a smooth interactive experience.

[0014] 4. Improve the real-time and reliability of teaching interaction.

[0015] By flexibly adjusting the heartbeat information sending frequency and combining it with the multicast protocol, it is possible to quickly monitor the device status in scenarios with high interactive demands (such as real-time Q&A and operation demonstrations) to ensure real-time synchronization of content. In scenarios with large-scale device synchronization, the system effectively prevents devices from being out of sync, ensuring efficient and reliable interaction.

[0016] 5. Improve network bandwidth utilization and device synchronization accuracy.

[0017] By combining adaptive bitrate and multicast protocol, this application reduces bandwidth usage and improves device synchronization accuracy. In an environment with sufficient bandwidth, the system can seamlessly transmit high-definition content; when bandwidth is limited, dynamically adjusting the adaptive bitrate and multicast protocol effectively reduces network pressure, ensuring that all devices receive optimized content synchronously, avoiding asynchronous or lost images.

[0018] 6. The combined effect of avoiding device disconnection and reducing transmission delays.

[0019] The system effectively reduces device disconnection and transmission delay by combining the heartbeat information sending frequency, adaptive bit rate and multicast protocol. In high-interaction scenarios, the heartbeat frequency is increased to ensure content synchronization; in low-interaction scenarios, the heartbeat frequency and bit rate are optimized, and the multicast protocol is combined to reduce disconnection and data loss, ensuring the stability of content transmission and display.

[0020] 7. Improve the scalability and cost-effectiveness of the system.

[0021] The method of the present application realizes multi-screen interaction, avoids dependence on dedicated hardware devices, and significantly reduces the system deployment cost. This solution based on Android devices has good scalability and can flexibly adapt to a variety of scenarios from small classrooms to large lectures, remote teaching, etc., and is not limited by hardware, which is convenient for future expansion and updating. It not only supports synchronous interaction of multiple devices, but also can quickly increase or reduce connected devices according to teaching needs, improving applicability and scalability.

[0022] 8. Intelligent dynamic fault handling and equipment status monitoring.

[0023] This application has an intelligent device status monitoring and fault handling mechanism. The system monitors the online status of each device in real time through heartbeat messages, and uses flexible error handling algorithms (such as dynamic threshold adjustment, event-driven algorithms, etc.) to deal with device failures or network problems. When the heartbeat times out or the network condition is poor, the system will issue a warning in time and adjust the network strategy to ensure the continuity of the teaching process and the reliable connection of the device. Even in scenarios where network conditions fluctuate greatly, the synchronization and stability of the teaching content can be maintained. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application. Figure 1 It is a flowchart of the multi-screen interaction method according to the present application. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of multiple embodiments to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0026] The embodiment of the present application provides a multi-screen interaction method suitable for multiple teaching scenarios based on dynamic network adjustment. The method runs on devices with Android 6.0 and above systems. It can optimize and adjust the heartbeat information sending frequency, adaptive bit rate, and multicast protocol more efficiently through a phased one-by-one optimization and adjustment method. Combined with efficient coding technology, it can significantly improve the efficiency of multi-screen interaction in a classroom environment, adapt to a variety of network conditions from high bandwidth to low bandwidth, and ensure smooth interaction and high-quality display in different teaching scenarios such as ordinary classrooms, large lectures or publicity activities, and support for remote education in remote areas.

[0027] This application provides a multi-screen interactive method based on dynamic network adjustment suitable for multiple teaching scenarios, such as Figure 1 As shown, the first embodiment is specifically as follows: S1, activating the teacher electronic device and the student electronic device through the same activation code. Each device generates and stores a network configuration information including a unique device identifier.

[0028] Specifically, the network configuration information includes the device's MAC address, IP address, and network configuration status (such as subnet mask, gateway, etc.). In addition, the device identifier (UUID), device type, and device name can also be combined to activate the device.

[0029] In some embodiments, teacher electronic devices generally refer to high-configuration electronic devices fixed to specific teaching locations (such as classrooms, lecture halls, etc.) and used by teachers. These devices generally include large-screen displays, consoles, sound systems, and interactive tools (such as electronic whiteboards or touch screens). Student electronic devices can be fixed group screens and portable devices distributed throughout the teaching location, designed to interconnect with teacher electronic devices. They can transmit content to each other, such as receiving teaching content from teacher devices, sending interactive feedback, and participating in various interactive activities such as voting, roll call, answering, file uploading, and remote teaching. Taking an ordinary classroom as an example, a classroom can be equipped with multiple fixed smart screens (generally 8), one of which is a teacher screen and the others are group discussion screens. At the same time, it supports students bringing their own devices (BYOD), that is, allowing students to use their own mobile phones, tablets or laptops in class. This method enhances the flexibility of learning. Students can access teaching resources, submit homework and participate in classroom discussions at any time, thereby enhancing interactivity and participation.

[0030] S2. The teacher electronic device periodically sends a heartbeat message to each student electronic device. Each student electronic device verifies the timestamp of the heartbeat message to determine whether the teacher electronic device and other student electronic devices are online.

[0031] Specifically, the heartbeat message includes the MAC address, IP address and timestamp of the source device (i.e., the teacher's electronic device). Generally, the timestamp refers to: in the heartbeat message, it is used to record the exact time when the message is generated and sent. The timestamp uses the UTC (Coordinated Universal Time) format to ensure the uniformity of time between devices. The timestamp generally includes: sending time, time format and time zone information. Among them, the sending time indicates the moment when the heartbeat message is generated; the time format uses the ISO 8601 standard to facilitate time parsing and comparison between devices; the time zone information is used to ensure the accuracy and consistency of time when the devices may be in different geographical locations.

[0032] Furthermore, if the timestamp of the received heartbeat message is invalid or the heartbeat message is not received within the predetermined time, the device status is marked as "offline", the error is recorded and a notification is sent to the teacher's electronic device; if the number of consecutive heartbeat timeouts exceeds the threshold, it is marked as "fault" and the corresponding fault handling process is started.

[0033] Generally, a variety of algorithms can be used to implement device status monitoring and fault handling, including: Dynamic threshold algorithm: Dynamically adjust the heartbeat timeout threshold according to the network status and the historical online status of the device to adapt to different usage scenarios and environments. Weighted heartbeat algorithm: Assign different weights to different types of devices, affecting the frequency and tolerance of heartbeat detection. Important devices may have more stringent heartbeat monitoring requirements. Event-driven algorithm: Using an event-driven approach, heartbeat messages are processed and monitored only when the device status changes, thereby reducing unnecessary inspections. Machine learning algorithm: Apply machine learning technology to analyze historical heartbeat data and identify abnormal patterns, thereby automatically adjusting the logic and threshold of heartbeat detection. Redundant detection algorithm: Use multiple independent heartbeat signal sources to improve the reliability of fault detection. Fault handling is only performed when multiple signal sources simultaneously show that the device is "offline."

[0034] This embodiment can select any of the above algorithms according to actual needs to achieve efficient, stable and high-quality multi-screen interaction. These algorithms can not only adapt to different network conditions and teaching scenarios, but also effectively improve user experience. Therefore, the above algorithms all fall within the scope of protection of this application, ensuring their application and implementation in the multi-screen interactive system.

[0035] S3, according to the real-time network conditions and the preset teaching scene interaction level, determine whether the heartbeat information sending frequency meets the requirements. If yes, proceed to step S4; if not, adjust the heartbeat information sending frequency until it meets the requirements.

[0036] Further, in step S3, the sending frequency range of the heartbeat message can be set to 100 milliseconds to 30 seconds. Different frequency values ​​will produce different effects and impacts. For example, more frequent heartbeat messages (such as every 1 second) ensure fast response, but increase the network burden; lower frequency heartbeats (such as every 8 seconds) are suitable for low-bandwidth environments and can effectively reduce network pressure. Real-time network conditions generally include factors such as network load, network delay, jitter and packet loss rate. Network load involves bandwidth utilization and number of connections. High load will cause data transmission speed to decrease. Network delay includes transmission delay, processing delay and queuing delay. Excessive delay may affect the performance of real-time applications. At the same time, jitter reflects the change of delay, and packet loss rate refers to the proportion of data packets lost during transmission. High packet loss rate will cause incomplete information and affect user experience. The frequency of sending heartbeat messages will be adjusted according to the above-mentioned real-time network conditions and the preset degree of interaction of the teaching scene to adapt to different operating conditions and requirements. Therefore, according to the network conditions and teaching scenes, the frequency of sending heartbeat messages should be reasonably selected to ensure that both the device status can be monitored in time and the teaching quality can be guaranteed in different scenes. In this embodiment, the frequency of sending the heartbeat message is set to once every second.

[0037] Generally speaking, the degree of interaction in a teaching scenario refers to the frequency and quality of communication between teachers and students, including real-time questions and answers, the speed of student feedback, the degree of participation in group discussions, and the frequency of interaction using electronic devices. A high degree of interaction indicates frequent communication and participation, while a low degree of interaction means less communication. This indicator helps to adjust the technical support strategy to ensure the effectiveness of teaching. Therefore, you can choose to preset this indicator to adjust the frequency of sending heartbeat messages.

[0038] Furthermore, taking ordinary classes as an example, the level of interaction is preset to be high, the number of questions and answers or interactions between teachers and students usually reaches more than 10 times, and the feedback response time is within 1 minute. In contrast, in large lectures, due to the large number of participants, the level of interaction is usually preset to be moderate, the frequency of interaction is between 5 and 10 times, and the feedback response time is between 1 and 3 minutes. Finally, in remote teaching scenarios in remote areas, due to network limitations and other factors, the level of interaction is preset to be low, the number of interactions may be small, for example, less than 5 times, and the feedback response time exceeds 3 minutes.

[0039] Furthermore, when adjusting the frequency of sending the heartbeat information in step S3, dynamic adjustment can be performed in combination with the real-time network conditions and the preset degree of interaction. In ordinary classrooms, the preset degree of interaction is high and the network conditions are good, and the frequency of sending heartbeat messages can be increased to ensure that the equipment is in good condition and the high-quality transmission of teaching content. Secondly, in large lectures, because the preset degree of interaction is moderate, if the network conditions are good, the heartbeat frequency can be moderately increased to maintain the stability of content transmission; if the network conditions are not good, the heartbeat frequency should be lowered to reduce the network burden. Finally, for distance education in remote areas, although the preset degree of interaction is low, if the network bandwidth is limited, the heartbeat frequency should also be lowered to ensure smooth transmission of teaching content under limited network conditions and to ensure teaching quality.

[0040] This adjustment strategy, which combines real-time network conditions and preset interaction levels, helps to effectively balance real-time monitoring needs and network load, thereby optimizing teaching effectiveness.

[0041] S4. When receiving the broadcast instruction from the teacher's electronic device, each student electronic device joins a multicast group identified by the multicast address and port to receive the teacher's screen content. The teacher's electronic device uses Android's MediaProjection API to start screen capture, encodes the screen content through the hardware encoder, and sends the encoded frame data to the multicast group. Each student electronic device listens to the multicast stream, decodes the video data and displays the teacher's screen content on the student's electronic device.

[0042] In some embodiments, step S4 can be further as follows: S401: the teacher's electronic device joins the multicast address A and binds the multicast port a1, and sends instructions to all student electronic devices in the teaching scene. The instruction information includes the MAC address of this device, the IP of this device, the MAC address of the target device, the IP address of the target device, the multicast address A, the multicast port a1, the timestamp (in UTC format), the behavior command and some optional parameters.

[0043] S402: After receiving the instruction information, the student electronic device creates an interface display Surface instance according to the action command to start receiving multicast, creates a hardware decoder that supports H265, and binds the interface display Surface instance to the output of the decoder. At this time, join the multicast address A to bind the multicast port a1 and start monitoring the video stream data. If the video data is monitored, start the decoding task.

[0044] S403: The teacher's electronic device applies for a screen recording instance by calling the Android system function ((MediaProjectionManager)getSystemService(Context.MEDIA_PROJECTION_SERVICE)).createScreenCaptureIntent() and requests user authorization. Create an encoder through the MediaCodec provided by the Android system, and then create an encoding input area Surface (the encoder can control the quality of the screen recording, such as frame rate, bit rate, etc.). At this time, the screen recording instance generates a virtual display, and the virtual display and the encoding input area Surface are associated. The data of the virtual display will be input into the encoding input area Surface, and the encoder will be started. At this time, the real-time screen image is obtained from the encoder's callback function onOutputBufferAvailable.

[0045] S404: Perform H265 encoding on each frame output by the encoder in real time, and ensure video quality according to encoding parameters (such as resolution, frame rate, etc.).

[0046] S405: Encapsulate the H265 encoded frame data into Nalu units, decompress and sub-packetize them according to the rules, and transmit the sub-packets through a private transmission protocol.

[0047] S406: After the student electronic device in step S402 monitors the transmitted data stream, it merges the received packetized data into frames and performs integrity check.

[0048] S407: Send the complete frame data to the decoder of the group screen in step S402 for decoding, and the teacher's screen image is displayed on the group screen.

[0049] In some embodiments, in step S406, a variety of integrity check algorithms can be used to ensure the reliability of data transmission during multi-screen interaction. Specifically, they include: cyclic redundancy check (CRC), which is used to detect any bit errors in the data packet sent from the teacher electronic device to the student electronic device to ensure the integrity of the data. Hash functions (such as SHA-256 or MD5) can generate hash values ​​for the transmitted frame data, so that the receiving student electronic device can verify the integrity of the data by comparing the calculated hash value with the sent hash value. Parity bits provide a simple and effective way to detect single-bit errors, which is particularly important in scenarios with high requirements for fast response. Implementing data packet sequence numbers allows student electronic devices to track the order in which data packets are received, thereby identifying lost or duplicated data packets. Finally, forward error correction (FEC) can send redundant data to enable student devices to reconstruct complete frames in the event of data loss.

[0050] This embodiment can select any of the above verification algorithms according to actual needs to achieve efficient, stable and high-quality multi-screen interaction. These algorithms can not only adapt to different network conditions and teaching scenarios, but also effectively improve user experience. Therefore, the above algorithms all fall within the scope of protection of this application, ensuring their application and implementation in the multi-screen interactive system.

[0051] In some embodiments, encoding and decoding may use different decoders such as H.264, H.265, VP8, VP9, ​​AV1, etc. Specifically, this embodiment uses the H.265 encoder, which can significantly reduce bandwidth requirements while maintaining high image quality by applying an optimized compression algorithm.

[0052] S5. Further determine whether the adaptive bit rate meets the requirements according to the image quality of the electronic device and the determined heartbeat information sending frequency. If it meets the requirements, proceed to step S6; if it does not meet the requirements, adjust the adaptive bit rate until the image quality requirements of the teaching content are met.

[0053] Further, in step S5, when judging whether the adaptive bit rate meets the requirements, specifically: first, evaluate the highest resolution supported by the device, for example, the teacher's electronic device may support 4K, while the student's electronic device may support 1080p. Monitor the clarity and smoothness of the playback through a real-time feedback mechanism, and conduct a comprehensive evaluation in combination with the received picture quality data (such as user feedback, the number of playback freezes, and video quality indicators); second, monitor the online status and network delay through regularly sent heartbeat messages. If the delay of the heartbeat message is less than 200 milliseconds and the packet loss rate is low, it indicates that the network condition is good. At this time, the adaptive bit rate can be appropriately increased, such as selecting 1080p (16Mbps) to improve the picture quality; conversely, if the heartbeat delay exceeds 500 milliseconds or the packet loss rate increases, the bit rate needs to be reduced, such as adjusting to 720p (8Mbps) to ensure smooth playback. Finally, the received picture quality indicators and the heartbeat information are comprehensively considered, and the corresponding threshold is set to dynamically and real-time adjust the adaptive bit rate, thereby optimizing the user experience of multi-screen interaction.

[0054] In some embodiments, adaptive bit rate (ABR) streaming media transmission is set to target different resolutions and bit rates, usually ranging from 120p to 4K and 1Mbps to 40Mbps respectively. The quality of teaching content is dynamically adjusted according to network conditions to ensure the best viewing experience. The above settings can be performed through the teacher's electronic device, capturing the teacher's screen content in real time, and automatically adjusting the resolution and bit rate of the video stream according to the current network bandwidth and terminal device conditions. In this embodiment, the adaptive bit rate can be: the resolution is 4K and the bit rate is 20Mbps.

[0055] S6. Further determine whether the adopted multicast protocol is appropriate according to the adjusted heartbeat information sending frequency and adaptive bit rate. If it is appropriate, proceed to step S7; if it is not appropriate, adjust the multicast protocol until the image quality requirements of the teaching content are met.

[0056] Furthermore, when judging whether the multicast protocol meets the requirements, specifically: First, the frequency of sending heartbeat information can reflect the stability and load of the network. If the frequency of sending heartbeat messages is high, it means that the network status is good, and it is suitable to choose an efficient multicast protocol, such as Internet Group Management Protocol (IGMP), Simple Multicast Protocol (SMP) or User Datagram Protocol (UDP), to ensure timely transmission of data streams and high-quality experience; if the frequency of sending heartbeat messages is low, it means that the network status is poor. At this time, you should choose a highly adaptable multicast protocol, such as Protocol Independent Multicast (PIM), Distance Vector Multicast Routing Protocol (DVMRP) or Real-time Transport Protocol (RTP), to ensure that data can still be effectively transmitted under unstable network conditions.

[0057] Secondly, the adaptive bitrate directly affects the quality of the video stream. For example, if the current adaptive bitrate is high (such as 4K (20Mbps) or 1080p (16Mbps)), it is necessary to select a multicast protocol that supports higher bandwidth, such as Internet Group Management Protocol (IGMP) or Multiprotocol Label Switching (MPLS) to ensure the quality of teaching content and transmission efficiency. For lower adaptive bitrates (such as 720p (8Mbps)), it is reasonable to select simpler multicast protocols such as Protocol Independent Multicast (PIM) or Distance Vector Multicast Routing Protocol (DVMRP), because these protocols can work effectively in medium or low bandwidth environments.

[0058] Finally, by comprehensively evaluating the sending frequency of heartbeat information and the adaptive bit rate, the corresponding selection criteria are set to ensure that the adopted multicast protocol can effectively support the needs of multi-screen interaction in different teaching scenarios, improve user experience and teaching effect. In this way, the multicast protocol can be dynamically adjusted to adapt to different network environments and device characteristics, thereby ensuring the efficiency of data transmission and the reliability of teaching content quality.

[0059] In some embodiments, the multicast protocol may be, but is not limited to: Internet Group Management Protocol (IGMP), Protocol Independent Multicast (PIM), Distance Vector Multicast Routing Protocol (DVMRP), Multi-Protocol Label Switching (MPLS), Real-time Transport Protocol (RTP) and Real-time Transport Control Protocol (RTCP), Simple Multicast Protocol (SMP) and Source Specific Multicast (SSM). According to different teaching scenarios and usage areas (for example: ordinary classrooms, high-definition lectures or large-scale events, such as remote multinational or remote areas, rural areas, etc.), combined with the sending frequency of heartbeat information and adaptive rate code, the above multicast protocol is reasonably selected to meet the requirements of efficient transmission, high-quality image quality and multi-screen interaction, and enhance the teaching experience. Specifically, in this embodiment, the multicast protocol adopts the Internet Group Management Protocol (IGMP).

[0060] In some embodiments, the teacher electronic device is responsible for creating a multicast stream and broadcasting the screen content to the student electronic devices according to a selected multicast protocol.

[0061] S7, the teacher's electronic device executes step S4 again to detect whether the image quality of the teaching content meets the requirements. If yes, it goes to step S8, if not, it goes to step S5 again.

[0062] Furthermore, the teacher's electronic device executes step S4 again and detects whether the image quality received by the student's electronic device meets the requirements. Specifically, if the image quality meets the requirements, it proceeds to step S8, and the teacher can request a specific student device to share its screen content. If the image quality does not meet the requirements, the system will return to step S5, readjust the adaptive bit rate, and then proceed to step S6 to further adjust the multicast protocol. After multiple adjustments, the system ensures that all devices can synchronously receive high-quality teaching content, improving the classroom interaction effect.

[0063] S8. The teacher electronic device sends a request to specify a specific student electronic device to send its screen content via unicast. After receiving the request, the student electronic device will use Android's MediaProjection API to capture its screen content and encode the content into the format required by the teacher. After the encoding is completed, the student electronic device transmits the encoded content to the teacher electronic device. The teacher electronic device receives and decodes the content and finally displays the student's screen content on the teacher electronic device.

[0064] In some embodiments, multiple request methods are linked to different teaching scenarios, and teachers can obtain students' screen content in different ways, including: in ordinary classrooms, teachers can use instant messaging tools (such as WeChat groups, QQ, etc.) to send requests, and students can quickly respond and share their screens after receiving them; in large-scale lecture scenarios, teachers can use special mobile applications (such as the official APP of the teaching platform), which provides a "Request Screen Sharing" button, and students only need to click to send requests to teachers. In addition, the application can also integrate real-time feedback functions. After clicking the request, students can choose the application or screen area to share, simplifying the operation process and improving the efficiency of classroom interaction. At the same time, the application displays the request status in real time to ensure unimpeded communication between teachers and students; in remote teaching in remote areas, teachers can use voice recognition technology to request students to share screen content through voice commands. Student devices can recognize teachers' voice requests in real time and feedback operation results, which can reduce dependence on complex operations and improve the convenience of interaction. In summary, these different request mechanisms can be flexibly applied according to specific teaching scenarios to improve the interactivity and effectiveness of teaching.

[0065] In some embodiments, after obtaining the students' screen content, the teacher can perform multi-content comparison and collaborative feedback to enhance classroom interaction and learning effects. For example, in an ordinary classroom, the teacher can display the screen content of multiple students at the same time, compare their problem-solving ideas in real time, help students learn from each other and identify differences in understanding. In large lectures, teachers can collect student feedback through online voting tools, and aggregate the screen content of different groups for discussion and analysis to promote in-depth thinking collisions. In remote teaching in remote areas, students can answer verbally through a microphone, and teachers use voice recognition technology to convert these voices into text to understand students' views or questions. The teacher can then combine these contents with the screen content shared by the students to provide more targeted feedback. For example, if a student shows the problem-solving process on the screen, the teacher can directly point out the errors in his ideas or provide further explanations based on the student's voice feedback.

[0066] In addition, teachers can annotate students' screen content in real time and provide specific suggestions, while students can also provide feedback to teachers, thus achieving two-way interaction. Finally, teachers can also save screen content at different time periods for historical analysis, compare students' progress, develop personalized learning plans, and improve teaching effectiveness. Through these mechanisms, teachers can optimize teaching strategies and promote more efficient learning for students.

[0067] S9. Continuously monitor the online status of each device to ensure that all devices in the classroom environment are synchronized and meet teaching requirements.

[0068] In some embodiments, during real-time monitoring, the performance of each device is comprehensively evaluated by regularly exchanging heartbeat information and monitoring multi-dimensional indicators such as adaptive bitrate. Heartbeat information reflects the online status of the device and the quality of the network connection, while the adaptive bitrate shows the current quality of the video stream and bandwidth usage. For example, a teacher can monitor the heartbeat frequency of a student's device to determine whether the network is stable; at the same time, observe the adaptive bitrate. If a student's bitrate is low, the teacher can proactively ask the student if there are network problems or technical difficulties. Through these multi-dimensional monitoring indicators, teachers can identify and resolve potential technical problems in a timely manner to ensure smooth classroom interaction.

[0069] Furthermore, classroom management can be further optimized by collecting the power, CPU usage, and memory usage of the device. For example, teachers can check the power status of each student's device and promptly determine whether they need to remind students to charge to avoid device shutdown or freezing due to insufficient power. If the CPU usage of a device is too high, the system can automatically suggest that students close unnecessary applications to free up resources and ensure smooth video playback and classroom interaction. Through these comprehensive measures, teachers can better maintain the classroom environment and improve teaching effectiveness.

[0070] Furthermore, an intelligent early warning mechanism can be set up to automatically send a notification to the teacher when a device's heartbeat message is lost more than three times, indicating that the device may have a network problem. For example, if a student's device frequently loses heartbeats, the teacher can proactively ask the student to help them solve the network connection problem and avoid class interruptions.

[0071] Furthermore, the system can automatically adjust teaching strategies based on real-time monitoring data. For example, if a student's network delay reaches 500 milliseconds, the system can temporarily disable the student's screen sharing function to ensure that other students' experience is not affected. This dynamic adjustment can maintain the overall stability of the class.

[0072] Furthermore, heartbeat messages and device status data can be collected regularly to generate usage reports. For example, teachers can view the performance reports of each device after class to understand which devices perform well and which devices have problems, thereby optimizing future device configuration and teaching arrangements. Such data analysis can help teachers better adapt to students' needs.

[0073] Furthermore, instant feedback tools can be introduced to allow students to quickly report their experiences or problems in class. For example, at the end of each class, students can submit their feelings, such as "the picture is frozen" or "the sound is not clear", through the feedback button in the app. Teachers can view these feedbacks instantly and make targeted adjustments.

[0074] Furthermore, heartbeat messages and device status data can be stored in the cloud for teachers to analyze and replay teaching afterwards. For example, teachers can view classroom monitoring data from the past week and identify which periods of time the device performed poorly, so as to improve teaching methods and technical support in a targeted manner.

[0075] The second embodiment of the above method is specifically as follows: S1, activating the teacher electronic device and the student electronic device by the same activation code. Each device generates and stores a network configuration information including a unique device identifier.

[0076] Specifically, the network configuration information includes the device's MAC address, IP address, and network configuration status (such as subnet mask, gateway, etc.). In addition, the device identifier (UUID), device type, and device name can also be combined to activate the device.

[0077] In some embodiments, teacher electronic devices generally refer to high-configuration electronic devices fixed to specific teaching locations (such as classrooms, lecture halls, etc.) and used by teachers. These devices generally include large-screen displays, consoles, sound systems, and interactive tools (such as electronic whiteboards or touch screens). Student electronic devices can be fixed group screens and portable devices distributed throughout the teaching location, designed to interconnect with teacher electronic devices. They can transmit content to each other, such as receiving teaching content from teacher devices, sending interactive feedback, and participating in various interactive activities such as voting, roll call, answering, file uploading, and remote teaching. Taking an ordinary classroom as an example, a classroom can be equipped with multiple fixed smart screens (generally 8), one of which is a teacher screen and the others are group discussion screens. At the same time, it supports students bringing their own devices (BYOD), that is, allowing students to use their own mobile phones, tablets or laptops in class. This method enhances the flexibility of learning. Students can access teaching resources, submit homework and participate in classroom discussions at any time, thereby enhancing interactivity and participation.

[0078] S2. The teacher electronic device periodically sends a heartbeat message to each student electronic device. Each student electronic device verifies the timestamp of the heartbeat message to determine whether the teacher electronic device and other student electronic devices are online.

[0079] Specifically, the heartbeat message includes the MAC address, IP address and timestamp of the source device (i.e., the teacher's electronic device). Generally, the timestamp refers to: in the heartbeat message, it is used to record the exact time when the message is generated and sent. The timestamp uses the UTC (Coordinated Universal Time) format to ensure the uniformity of time between devices. The timestamp generally includes: sending time, time format and time zone information. Among them, the sending time indicates the moment when the heartbeat message is generated; the time format uses the ISO 8601 standard to facilitate time parsing and comparison between devices; the time zone information is used to ensure the accuracy and consistency of time when the devices may be in different geographical locations.

[0080] Furthermore, if the timestamp of the received heartbeat message is invalid or the heartbeat message is not received within the predetermined time, the device status is marked as "offline", the error is recorded and a notification is sent to the teacher's electronic device; if the number of consecutive heartbeat timeouts exceeds the threshold, it is marked as "fault" and the corresponding fault handling process is started.

[0081] Generally, a variety of algorithms can be used to implement device status monitoring and fault handling, including: Dynamic threshold algorithm: Dynamically adjust the heartbeat timeout threshold according to the network status and the historical online status of the device to adapt to different usage scenarios and environments. Weighted heartbeat algorithm: Assign different weights to different types of devices, affecting the frequency and tolerance of heartbeat detection. Important devices may have more stringent heartbeat monitoring requirements. Event-driven algorithm: Using an event-driven approach, heartbeat messages are processed and monitored only when the device status changes, thereby reducing unnecessary inspections. Machine learning algorithm: Apply machine learning technology to analyze historical heartbeat data and identify abnormal patterns, thereby automatically adjusting the logic and threshold of heartbeat detection. Redundant detection algorithm: Use multiple independent heartbeat signal sources to improve the reliability of fault detection. Fault handling is only performed when multiple signal sources simultaneously show that the device is "offline."

[0082] This embodiment can select any of the above algorithms according to actual needs to achieve efficient, stable and high-quality multi-screen interaction. These algorithms can not only adapt to different network conditions and teaching scenarios, but also effectively improve user experience. Therefore, the above algorithms all fall within the scope of protection of this application, ensuring their application and implementation in the multi-screen interactive system.

[0083] S3, according to the real-time network conditions and the preset teaching scene interaction level, determine whether the heartbeat information sending frequency meets the requirements. If yes, proceed to step S4; if not, adjust the heartbeat information sending frequency until it meets the requirements.

[0084] Further, in step S3, the sending frequency range of the heartbeat message can be set to 100 milliseconds to 30 seconds. Different frequency values ​​will produce different effects and impacts. For example, more frequent heartbeat messages (such as every 1 second) ensure fast response, but increase the network burden; lower frequency heartbeats (such as every 8 seconds) are suitable for low-bandwidth environments and can effectively reduce network pressure. Real-time network conditions generally include factors such as network load, network delay, jitter and packet loss rate. Network load involves bandwidth utilization and number of connections. High load will cause data transmission speed to decrease. Network delay includes transmission delay, processing delay and queuing delay. Excessive delay may affect the performance of real-time applications. At the same time, jitter reflects the change of delay, and packet loss rate refers to the proportion of data packets lost during transmission. High packet loss rate will cause incomplete information and affect user experience. The frequency of sending heartbeat messages will be adjusted according to the above-mentioned real-time network conditions and the preset degree of interaction of the teaching scene to adapt to different operating conditions and requirements. Therefore, according to the network conditions and teaching scenes, the frequency of sending heartbeat messages should be reasonably selected to ensure that both the device status can be monitored in time and the teaching quality can be guaranteed in different scenes. In this embodiment, the frequency of sending the heartbeat message is set to once every 3 seconds.

[0085] Generally speaking, the degree of interaction in a teaching scenario refers to the frequency and quality of communication between teachers and students, including real-time questions and answers, the speed of student feedback, the degree of participation in group discussions, and the frequency of interaction using electronic devices. A high degree of interaction indicates frequent communication and participation, while a low degree of interaction means less communication. This indicator helps to adjust the technical support strategy to ensure the effectiveness of teaching. Therefore, you can choose to preset this indicator to adjust the frequency of sending heartbeat messages.

[0086] Furthermore, taking ordinary classes as an example, the level of interaction is preset to be high, the number of questions and answers or interactions between teachers and students usually reaches more than 10 times, and the feedback response time is within 1 minute. In contrast, in large lectures, due to the large number of participants, the level of interaction is usually preset to be moderate, the frequency of interaction is between 5 and 10 times, and the feedback response time is between 1 and 3 minutes. Finally, in remote teaching scenarios in remote areas, due to network limitations and other factors, the level of interaction is preset to be low, the number of interactions may be small, for example, less than 5 times, and the feedback response time exceeds 3 minutes.

[0087] Furthermore, when adjusting the frequency of sending the heartbeat information in step S3, dynamic adjustment can be performed in combination with the real-time network conditions and the preset degree of interaction. In ordinary classrooms, the preset degree of interaction is high and the network conditions are good, and the frequency of sending heartbeat messages can be increased to ensure that the equipment is in good condition and the high-quality transmission of teaching content. Secondly, in large lectures, because the preset degree of interaction is moderate, if the network conditions are good, the heartbeat frequency can be moderately increased to maintain the stability of content transmission; if the network conditions are not good, the heartbeat frequency should be lowered to reduce the network burden. Finally, for distance education in remote areas, although the preset degree of interaction is low, if the network bandwidth is limited, the heartbeat frequency should also be lowered to ensure smooth transmission of teaching content under limited network conditions and to ensure teaching quality.

[0088] This adjustment strategy, which combines real-time network conditions and preset interaction levels, helps to effectively balance real-time monitoring needs and network load, thereby optimizing teaching effectiveness.

[0089] S4. When receiving the broadcast instruction from the teacher's electronic device, each student electronic device joins a multicast group identified by the multicast address and port to receive the teacher's screen content. The teacher's electronic device uses Android's MediaProjection API to start screen capture, encodes the screen content through the hardware encoder, and sends the encoded frame data to the multicast group. Each student electronic device listens to the multicast stream, decodes the video data and displays the teacher's screen content on the student's electronic device.

[0090] In some embodiments, step S4 can be further as follows: S401: the teacher's electronic device joins the multicast address A and binds the multicast port a1, and sends instructions to all student electronic devices in the teaching scene. The instruction information includes the MAC address of this device, the IP of this device, the MAC address of the target device, the IP address of the target device, the multicast address A, the multicast port a1, the timestamp (in UTC format), the behavior command and some optional parameters.

[0091] S402: After receiving the instruction information, the student electronic device creates an interface display Surface instance according to the action command to start receiving multicast, creates a hardware decoder that supports H265, and binds the interface display Surface instance to the output of the decoder. At this time, join the multicast address A to bind the multicast port a1 and start monitoring the video stream data. If the video data is monitored, start the decoding task.

[0092] S403: The teacher's electronic device applies for a screen recording instance by calling the Android system function ((MediaProjectionManager)getSystemService(Context.MEDIA_PROJECTION_SERVICE)).createScreenCaptureIntent() and requests user authorization. Create an encoder through the MediaCodec provided by the Android system, and then create an encoding input area Surface (the encoder can control the quality of the screen recording, such as frame rate, bit rate, etc.). At this time, the screen recording instance generates a virtual display, and the virtual display and the encoding input area Surface are associated. The data of the virtual display will be input into the encoding input area Surface, and the encoder will be started. At this time, the real-time screen image is obtained from the encoder's callback function onOutputBufferAvailable.

[0093] S404: Perform H265 encoding on each frame output by the encoder in real time, and ensure video quality according to encoding parameters (such as resolution, frame rate, etc.).

[0094] S405: Encapsulate the H265 encoded frame data into Nalu units, decompress and sub-packetize them according to the rules, and transmit the sub-packets through a private transmission protocol.

[0095] S406: After the student electronic device in step S402 monitors the transmitted data stream, it merges the received packetized data into frames and performs integrity check.

[0096] S407: Send the complete frame data to the decoder of the group screen in step S402 for decoding, and the teacher's screen image is displayed on the group screen.

[0097] In some embodiments, in step S406, a variety of integrity check algorithms can be used to ensure the reliability of data transmission during multi-screen interaction. Specifically, they include: cyclic redundancy check (CRC), which is used to detect any bit errors in the data packet sent from the teacher electronic device to the student electronic device to ensure the integrity of the data. Hash functions (such as SHA-256 or MD5) can generate hash values ​​for the transmitted frame data, so that the receiving student electronic device can verify the integrity of the data by comparing the calculated hash value with the sent hash value. Parity bits provide a simple and effective way to detect single-bit errors, which is particularly important in scenarios with high requirements for fast response. Implementing data packet sequence numbers allows student electronic devices to track the order in which data packets are received, thereby identifying lost or duplicated data packets. Finally, forward error correction (FEC) can send redundant data to enable student devices to reconstruct complete frames in the event of data loss.

[0098] This embodiment can select any of the above verification algorithms according to actual needs to achieve efficient, stable and high-quality multi-screen interaction. These algorithms can not only adapt to different network conditions and teaching scenarios, but also effectively improve user experience. Therefore, the above algorithms all fall within the scope of protection of this application, ensuring their application and implementation in the multi-screen interactive system.

[0099] In some embodiments, encoding and decoding may use different decoders such as H.264, H.265, VP8, VP9, ​​AV1, etc. Specifically, this embodiment uses the H.265 encoder, which can significantly reduce bandwidth requirements while maintaining high image quality by applying an optimized compression algorithm.

[0100] S5. Further determine whether the adaptive bit rate meets the requirements according to the image quality of the electronic device and the determined heartbeat information sending frequency. If it meets the requirements, proceed to step S6; if it does not meet the requirements, adjust the adaptive bit rate until the image quality requirements of the teaching content are met.

[0101] Further, in step S5, when judging whether the adaptive bit rate meets the requirements, specifically: first, evaluate the highest resolution supported by the device, for example, the teacher's electronic device may support 4K, while the student's electronic device may support 1080p. Monitor the clarity and smoothness of the playback through a real-time feedback mechanism, and conduct a comprehensive evaluation in combination with the received picture quality data (such as user feedback, the number of playback freezes, and video quality indicators); second, monitor the online status and network delay through regularly sent heartbeat messages. If the delay of the heartbeat message is less than 200 milliseconds and the packet loss rate is low, it indicates that the network condition is good. At this time, the adaptive bit rate can be appropriately increased, such as selecting 1080p (16Mbps) to improve the picture quality; conversely, if the heartbeat delay exceeds 500 milliseconds or the packet loss rate increases, the bit rate needs to be reduced, such as adjusting to 720p (8Mbps) to ensure smooth playback. Finally, the received picture quality indicators and the heartbeat information are comprehensively considered, and the corresponding threshold is set to dynamically and real-time adjust the adaptive bit rate, thereby optimizing the user experience of multi-screen interaction.

[0102] In some embodiments, adaptive bit rate (ABR) streaming media transmission is set to target different resolutions and bit rates, usually ranging from 120p to 4K and 1Mbps to 40Mbps respectively. The quality of teaching content is dynamically adjusted according to network conditions to ensure the best viewing experience. The above settings can be performed through the teacher's electronic device, capturing the teacher's screen content in real time, and automatically adjusting the resolution and bit rate of the video stream according to the current network bandwidth and terminal device conditions. In this embodiment, the adaptive bit rate can be: the resolution is 1080p and the bit rate is 16Mbps.

[0103] S6. Further determine whether the adopted multicast protocol is appropriate according to the adjusted heartbeat information sending frequency and adaptive bit rate. If it is appropriate, proceed to step S7; if it is not appropriate, adjust the multicast protocol until the image quality requirements of the teaching content are met.

[0104] Furthermore, when judging whether the multicast protocol meets the requirements, specifically: First, the frequency of sending heartbeat information can reflect the stability and load of the network. If the frequency of sending heartbeat messages is high, it means that the network status is good, and it is suitable to choose an efficient multicast protocol, such as Internet Group Management Protocol (IGMP), Simple Multicast Protocol (SMP) or User Datagram Protocol (UDP), to ensure timely transmission of data streams and high-quality experience; if the frequency of sending heartbeat messages is low, it means that the network status is poor. At this time, you should choose a highly adaptable multicast protocol, such as Protocol Independent Multicast (PIM), Distance Vector Multicast Routing Protocol (DVMRP) or Real-time Transport Protocol (RTP), to ensure that data can still be effectively transmitted under unstable network conditions.

[0105] Secondly, the adaptive bitrate directly affects the quality of the video stream. For example, if the current adaptive bitrate is high (such as 4K (20Mbps) or 1080p (16Mbps)), it is necessary to select a multicast protocol that supports higher bandwidth, such as Internet Group Management Protocol (IGMP) or Multiprotocol Label Switching (MPLS) to ensure the quality of teaching content and transmission efficiency. For lower adaptive bitrates (such as 720p (8Mpbs)), it is reasonable to select simpler multicast protocols such as Protocol Independent Multicast (PIM) or Distance Vector Multicast Routing Protocol (DVMRP), because these protocols can work effectively in medium or low bandwidth environments.

[0106] Finally, by comprehensively evaluating the sending frequency of heartbeat information and the adaptive bit rate, the corresponding selection criteria are set to ensure that the adopted multicast protocol can effectively support the needs of multi-screen interaction in different teaching scenarios, improve user experience and teaching effect. In this way, the multicast protocol can be dynamically adjusted to adapt to different network environments and device characteristics, thereby ensuring the efficiency of data transmission and the reliability of teaching content quality.

[0107] In some embodiments, the multicast protocol may be, but is not limited to, Internet Group Management Protocol (IGMP), Protocol Independent Multicast (PIM), Distance Vector Multicast Routing Protocol (DVMRP), Multi-Protocol Label Switching (MPLS), Real-time Transport Protocol (RTP) and Real-time Transport Control Protocol (RTCP), Simple Multicast Protocol (SMP) and Source Specific Multicast (SSM). According to different teaching scenarios and usage areas (for example, ordinary classrooms, high-definition lectures or large-scale events, such as remote multinational or remote areas, rural areas, etc.), combined with the sending frequency of heartbeat information and adaptive rate code, the above multicast protocol is reasonably selected to meet the requirements of efficient transmission, high-quality multi-screen interaction, and enhance the teaching experience. Specifically, in this embodiment, the multicast protocol adopts Protocol Independent Multicast (PIM).

[0108] In some embodiments, the teacher electronic device is responsible for creating a multicast stream and broadcasting the screen content to the student electronic devices according to a selected multicast protocol.

[0109] S7, the teacher's electronic device executes step S4 again to detect whether the image quality of the teaching content meets the requirements. If yes, it goes to step S8, if not, it goes to step S5 again.

[0110] Furthermore, the teacher's electronic device executes step S4 again and detects whether the image quality received by the student's electronic device meets the requirements. Specifically, if the image quality meets the requirements, it proceeds to step S8, and the teacher can request a specific student device to share its screen content. If the image quality does not meet the requirements, the system will return to step S5, readjust the adaptive bit rate, and then proceed to step S6 to further adjust the multicast protocol. After multiple adjustments, the system ensures that all devices can synchronously receive high-quality teaching content, improving the classroom interaction effect.

[0111] S8. The teacher electronic device sends a request to specify a specific student electronic device to send its screen content via unicast. After receiving the request, the student electronic device will use Android's MediaProjection API to capture its screen content and encode the content into the format required by the teacher. After the encoding is completed, the student electronic device transmits the encoded content to the teacher electronic device. The teacher electronic device receives and decodes the content and finally displays the student's screen content on the teacher electronic device.

[0112] In some embodiments, multiple request methods are linked to different teaching scenarios, and teachers can obtain students' screen content in different ways, including: in ordinary classrooms, teachers can use instant messaging tools (such as WeChat groups, QQ, etc.) to send requests, and students can quickly respond and share their screens after receiving them; in large-scale lecture scenarios, teachers can use special mobile applications (such as the official APP of the teaching platform), which provides a "Request Screen Sharing" button, and students only need to click to send requests to teachers. In addition, the application can also integrate real-time feedback functions. After clicking the request, students can choose the application or screen area to share, simplifying the operation process and improving the efficiency of classroom interaction. At the same time, the application displays the request status in real time to ensure unimpeded communication between teachers and students; in remote teaching in remote areas, teachers can use voice recognition technology to request students to share screen content through voice commands. Student devices can recognize teachers' voice requests in real time and feedback operation results, which can reduce dependence on complex operations and improve the convenience of interaction. In summary, these different request mechanisms can be flexibly applied according to specific teaching scenarios to improve the interactivity and effectiveness of teaching.

[0113] In some embodiments, after obtaining the students' screen content, the teacher can perform multi-content comparison and collaborative feedback to enhance classroom interaction and learning effects. For example, in an ordinary classroom, the teacher can display the screen content of multiple students at the same time, compare their problem-solving ideas in real time, help students learn from each other and identify differences in understanding. In large lectures, teachers can collect student feedback through online voting tools, and aggregate the screen content of different groups for discussion and analysis to promote in-depth thinking collisions. In remote teaching in remote areas, students can answer verbally through a microphone, and teachers use voice recognition technology to convert these voices into text to understand students' views or questions. The teacher can then combine these contents with the screen content shared by the students to provide more targeted feedback. For example, if a student shows the problem-solving process on the screen, the teacher can directly point out the errors in his ideas or provide further explanations based on the student's voice feedback.

[0114] In addition, teachers can annotate students' screen content in real time and provide specific suggestions, while students can also provide feedback to teachers, thus achieving two-way interaction. Finally, teachers can also save screen content at different time periods for historical analysis, compare students' progress, develop personalized learning plans, and improve teaching effectiveness. Through these mechanisms, teachers can optimize teaching strategies and promote more efficient learning for students.

[0115] S9. Continuously monitor the online status of each device to ensure that all devices in the classroom environment are synchronized and meet teaching requirements.

[0116] In some embodiments, during real-time monitoring, the performance of each device is comprehensively evaluated by regularly exchanging heartbeat information and monitoring multi-dimensional indicators such as adaptive bitrate. Heartbeat information reflects the online status of the device and the quality of the network connection, while the adaptive bitrate shows the current quality of the video stream and bandwidth usage. For example, a teacher can monitor the heartbeat frequency of a student's device to determine whether the network is stable; at the same time, observe the adaptive bitrate. If a student's bitrate is low, the teacher can proactively ask the student if there are network problems or technical difficulties. Through these multi-dimensional monitoring indicators, teachers can identify and resolve potential technical problems in a timely manner to ensure smooth classroom interaction.

[0117] Furthermore, classroom management can be further optimized by collecting the power, CPU usage, and memory usage of the device. For example, teachers can check the power status of each student's device and promptly determine whether they need to remind students to charge to avoid device shutdown or freezing due to insufficient power. If the CPU usage of a device is too high, the system can automatically suggest that students close unnecessary applications to free up resources and ensure smooth video playback and classroom interaction. Through these comprehensive measures, teachers can better maintain the classroom environment and improve teaching effectiveness.

[0118] Furthermore, an intelligent early warning mechanism can be set up to automatically send a notification to the teacher when a device's heartbeat message is lost more than three times, indicating that the device may have a network problem. For example, if a student's device frequently loses heartbeats, the teacher can proactively ask the student to help them solve the network connection problem and avoid class interruptions.

[0119] Furthermore, the system can automatically adjust teaching strategies based on real-time monitoring data. For example, if a student's network delay reaches 500 milliseconds, the system can temporarily disable the student's screen sharing function to ensure that other students' experience is not affected. This dynamic adjustment can maintain the overall stability of the class.

[0120] Furthermore, heartbeat messages and device status data can be collected regularly to generate usage reports. For example, teachers can view the performance reports of each device after class to understand which devices perform well and which devices have problems, thereby optimizing future device configuration and teaching arrangements. Such data analysis can help teachers better adapt to students' needs.

[0121] Furthermore, instant feedback tools can be introduced to allow students to quickly report their experiences or problems in class. For example, at the end of each class, students can submit their feelings, such as "the picture is frozen" or "the sound is not clear", through the feedback button in the app. Teachers can view these feedbacks instantly and make targeted adjustments.

[0122] Furthermore, heartbeat messages and device status data can be stored in the cloud for teachers to analyze and replay teaching afterwards. For example, teachers can view classroom monitoring data from the past week and identify which periods of time the device performed poorly, so as to improve teaching methods and technical support in a targeted manner.

[0123] The third embodiment of the above method is as follows: S1, activating the teacher electronic device and the student electronic device by the same activation code. Each device generates and stores a network configuration information including a unique device identifier.

[0124] Specifically, the network configuration information includes the device's MAC address, IP address, and network configuration status (such as subnet mask, gateway, etc.). In addition, the device identifier (UUID), device type, and device name can also be combined to activate the device.

[0125] In some embodiments, teacher electronic devices generally refer to high-configuration electronic devices fixed to specific teaching locations (such as classrooms, lecture halls, etc.) and used by teachers. These devices generally include large-screen displays, consoles, sound systems, and interactive tools (such as electronic whiteboards or touch screens). Student electronic devices can be fixed group screens and portable devices distributed throughout the teaching location, designed to interconnect with teacher electronic devices. They can transmit content to each other, such as receiving teaching content from teacher devices, sending interactive feedback, and participating in various interactive activities such as voting, roll call, answering, file uploading, and remote teaching. Taking an ordinary classroom as an example, a classroom can be equipped with multiple fixed smart screens (generally 8), one of which is a teacher screen and the others are group discussion screens. At the same time, it supports students bringing their own devices (BYOD), that is, allowing students to use their own mobile phones, tablets or laptops in class. This method enhances the flexibility of learning. Students can access teaching resources, submit homework and participate in classroom discussions at any time, thereby enhancing interactivity and participation.

[0126] S2. The teacher electronic device periodically sends a heartbeat message to each student electronic device. Each student electronic device verifies the timestamp of the heartbeat message to determine whether the teacher electronic device and other student electronic devices are online.

[0127] Specifically, the heartbeat message includes the MAC address, IP address and timestamp of the source device (i.e., the teacher's electronic device). Generally, the timestamp refers to: in the heartbeat message, it is used to record the exact time when the message is generated and sent. The timestamp uses the UTC (Coordinated Universal Time) format to ensure the uniformity of time between devices. The timestamp generally includes: sending time, time format and time zone information. Among them, the sending time indicates the moment when the heartbeat message is generated; the time format uses the ISO 8601 standard to facilitate time parsing and comparison between devices; the time zone information is used to ensure the accuracy and consistency of time when the devices may be in different geographical locations.

[0128] Furthermore, if the timestamp of the received heartbeat message is invalid or the heartbeat message is not received within the predetermined time, the device status is marked as "offline", the error is recorded and a notification is sent to the teacher's electronic device; if the number of consecutive heartbeat timeouts exceeds the threshold, it is marked as "fault" and the corresponding fault handling process is started.

[0129] Generally, a variety of algorithms can be used to implement device status monitoring and fault handling, including: Dynamic threshold algorithm: Dynamically adjust the heartbeat timeout threshold according to the network status and the historical online status of the device to adapt to different usage scenarios and environments. Weighted heartbeat algorithm: Assign different weights to different types of devices, affecting the frequency and tolerance of heartbeat detection. Important devices may have more stringent heartbeat monitoring requirements. Event-driven algorithm: Using an event-driven approach, heartbeat messages are processed and monitored only when the device status changes, thereby reducing unnecessary inspections. Machine learning algorithm: Apply machine learning technology to analyze historical heartbeat data and identify abnormal patterns, thereby automatically adjusting the logic and threshold of heartbeat detection. Redundant detection algorithm: Use multiple independent heartbeat signal sources to improve the reliability of fault detection. Fault handling is only performed when multiple signal sources simultaneously show that the device is "offline."

[0130] This embodiment can select any of the above algorithms according to actual needs to achieve efficient, stable and high-quality multi-screen interaction. These algorithms can not only adapt to different network conditions and teaching scenarios, but also effectively improve user experience. Therefore, the above algorithms all fall within the scope of protection of this application, ensuring their application and implementation in the multi-screen interactive system.

[0131] S3, according to the real-time network conditions and the preset teaching scene interaction level, determine whether the heartbeat information sending frequency meets the requirements. If yes, proceed to step S4; if not, adjust the heartbeat information sending frequency until it meets the requirements.

[0132] Further, in step S3, the sending frequency range of the heartbeat message can be set to 100 milliseconds to 30 seconds. Different frequency values ​​will produce different effects and impacts. For example, more frequent heartbeat messages (such as every 1 second) ensure fast response, but increase the network burden; lower frequency heartbeats (such as every 8 seconds) are suitable for low-bandwidth environments and can effectively reduce network pressure. Real-time network conditions generally include factors such as network load, network delay, jitter and packet loss rate. Network load involves bandwidth utilization and number of connections. High load will cause data transmission speed to decrease. Network delay includes transmission delay, processing delay and queuing delay. Excessive delay may affect the performance of real-time applications. At the same time, jitter reflects the change of delay, and packet loss rate refers to the proportion of data packets lost during transmission. High packet loss rate will cause incomplete information and affect user experience. The frequency of sending heartbeat messages will be adjusted according to the above-mentioned real-time network conditions and the preset degree of interaction of the teaching scene to adapt to different operating conditions and requirements. Therefore, according to the network conditions and teaching scenes, the frequency of sending heartbeat messages should be reasonably selected to ensure that both the device status can be monitored in time and the teaching quality can be guaranteed in different scenes. In this embodiment, the frequency of sending the heartbeat message is set to once every 6 seconds.

[0133] Generally speaking, the degree of interaction in a teaching scenario refers to the frequency and quality of communication between teachers and students, including real-time questions and answers, the speed of student feedback, the degree of participation in group discussions, and the frequency of interaction using electronic devices. A high degree of interaction indicates frequent communication and participation, while a low degree of interaction means less communication. This indicator helps to adjust the technical support strategy to ensure the effectiveness of teaching. Therefore, you can choose to preset this indicator to adjust the frequency of sending heartbeat messages.

[0134] Furthermore, taking ordinary classes as an example, the level of interaction is preset to be high, the number of questions and answers or interactions between teachers and students usually reaches more than 10 times, and the feedback response time is within 1 minute. In contrast, in large lectures, due to the large number of participants, the level of interaction is usually preset to be moderate, the frequency of interaction is between 5 and 10 times, and the feedback response time is between 1 and 3 minutes. Finally, in remote teaching scenarios in remote areas, due to network limitations and other factors, the level of interaction is preset to be low, the number of interactions may be small, for example, less than 5 times, and the feedback response time exceeds 3 minutes.

[0135] Furthermore, when adjusting the frequency of sending the heartbeat information in step S3, dynamic adjustment can be performed in combination with the real-time network conditions and the preset degree of interaction. In ordinary classrooms, the preset degree of interaction is high and the network conditions are good, and the frequency of sending heartbeat messages can be increased to ensure that the equipment is in good condition and the high-quality transmission of teaching content. Secondly, in large lectures, because the preset degree of interaction is moderate, if the network conditions are good, the heartbeat frequency can be moderately increased to maintain the stability of content transmission; if the network conditions are not good, the heartbeat frequency should be lowered to reduce the network burden. Finally, for distance education in remote areas, although the preset degree of interaction is low, if the network bandwidth is limited, the heartbeat frequency should also be lowered to ensure smooth transmission of teaching content under limited network conditions and to ensure teaching quality.

[0136] This adjustment strategy, which combines real-time network conditions and preset interaction levels, helps to effectively balance real-time monitoring needs and network load, thereby optimizing teaching effectiveness.

[0137] S4. When receiving the broadcast instruction from the teacher's electronic device, each student electronic device joins a multicast group identified by the multicast address and port to receive the teacher's screen content. The teacher's electronic device uses Android's MediaProjection API to start screen capture, encodes the screen content through the hardware encoder, and sends the encoded frame data to the multicast group. Each student electronic device listens to the multicast stream, decodes the video data and displays the teacher's screen content on the student's electronic device.

[0138] In some embodiments, step S4 can be further as follows: S401: the teacher's electronic device joins the multicast address A and binds the multicast port a1, and sends instructions to all student electronic devices in the teaching scene. The instruction information includes the MAC address of this device, the IP of this device, the MAC address of the target device, the IP address of the target device, the multicast address A, the multicast port a1, the timestamp (in UTC format), the behavior command and some optional parameters.

[0139] S402: After receiving the instruction information, the student electronic device creates an interface display Surface instance according to the action command to start receiving multicast, creates a hardware decoder that supports H265, and binds the interface display Surface instance to the output of the decoder. At this time, join the multicast address A to bind the multicast port a1 and start monitoring the video stream data. If the video data is monitored, start the decoding task.

[0140] S403: The teacher's electronic device applies for a screen recording instance by calling the Android system function ((MediaProjectionManager)getSystemService(Context.MEDIA_PROJECTION_SERVICE)).createScreenCaptureIntent() and requests user authorization. Create an encoder through the MediaCodec provided by the Android system, and then create an encoding input area Surface (the encoder can control the quality of the screen recording, such as frame rate, bit rate, etc.). At this time, the screen recording instance generates a virtual display, and the virtual display and the encoding input area Surface are associated. The data of the virtual display will be input into the encoding input area Surface, and the encoder will be started. At this time, the real-time screen image is obtained from the encoder's callback function onOutputBufferAvailable.

[0141] S404: Perform H265 encoding on each frame output by the encoder in real time, and ensure video quality according to encoding parameters (such as resolution, frame rate, etc.).

[0142] S405: Encapsulate the H265 encoded frame data into Nalu units, decompress and sub-packetize them according to the rules, and transmit the sub-packets through a private transmission protocol.

[0143] S406: After the student electronic device in step S402 monitors the transmitted data stream, it merges the received packetized data into frames and performs integrity check.

[0144] S407: Send the complete frame data to the decoder of the group screen in step S402 for decoding, and the teacher's screen image is displayed on the group screen.

[0145] In some embodiments, in step S406, a variety of integrity check algorithms can be used to ensure the reliability of data transmission during multi-screen interaction. Specifically, they include: cyclic redundancy check (CRC), which is used to detect any bit errors in the data packet sent from the teacher electronic device to the student electronic device to ensure the integrity of the data. Hash functions (such as SHA-256 or MD5) can generate hash values ​​for the transmitted frame data, so that the receiving student electronic device can verify the integrity of the data by comparing the calculated hash value with the sent hash value. Parity bits provide a simple and effective way to detect single-bit errors, which is particularly important in scenarios with high requirements for fast response. Implementing data packet sequence numbers allows student electronic devices to track the order in which data packets are received, thereby identifying lost or duplicated data packets. Finally, forward error correction (FEC) can send redundant data to enable student devices to reconstruct complete frames in the event of data loss.

[0146] This embodiment can select any of the above verification algorithms according to actual needs to achieve efficient, stable and high-quality multi-screen interaction. These algorithms can not only adapt to different network conditions and teaching scenarios, but also effectively improve user experience. Therefore, the above algorithms all fall within the scope of protection of this application, ensuring their application and implementation in the multi-screen interactive system.

[0147] In some embodiments, encoding and decoding may use different decoders such as H.264, H.265, VP8, VP9, ​​AV1, etc. Specifically, this embodiment uses the H.265 encoder, which can significantly reduce bandwidth requirements while maintaining high image quality by applying an optimized compression algorithm.

[0148] S5. Further determine whether the adaptive bit rate meets the requirements according to the image quality of the electronic device and the determined heartbeat information sending frequency. If it meets the requirements, proceed to step S6; if it does not meet the requirements, adjust the adaptive bit rate until the image quality requirements of the teaching content are met.

[0149] Further, in step S5, when judging whether the adaptive bit rate meets the requirements, specifically: first, evaluate the highest resolution supported by the device, for example, the teacher's electronic device may support 4K, while the student's electronic device may support 1080p. Monitor the clarity and smoothness of the playback through a real-time feedback mechanism, and conduct a comprehensive evaluation in combination with the received picture quality data (such as user feedback, the number of playback freezes, and video quality indicators); second, monitor the online status and network delay through regularly sent heartbeat messages. If the delay of the heartbeat message is less than 200 milliseconds and the packet loss rate is low, it indicates that the network condition is good. At this time, the adaptive bit rate can be appropriately increased, such as selecting 1080p (16Mbps) to improve the picture quality; conversely, if the heartbeat delay exceeds 500 milliseconds or the packet loss rate increases, the bit rate needs to be reduced, such as adjusting to 720p (8Mbps) to ensure smooth playback. Finally, the received picture quality indicators and the heartbeat information are comprehensively considered, and the corresponding threshold is set to dynamically and real-time adjust the adaptive bit rate, thereby optimizing the user experience of multi-screen interaction.

[0150] In some embodiments, adaptive bit rate (ABR) streaming media transmission is set to target different resolutions and bit rates, usually ranging from 120p to 4K and 1Mbps to 40Mbps respectively. The quality of teaching content is dynamically adjusted according to network conditions to ensure the best viewing experience. The above settings can be performed through the teacher's electronic device, capturing the teacher's screen content in real time, and automatically adjusting the resolution and bit rate of the video stream according to the current network bandwidth and terminal device conditions. In this embodiment, the adaptive bit rate can be: the resolution is 720p and the bit rate is 8 Mbps.

[0151] S6. Further determine whether the adopted multicast protocol is appropriate according to the adjusted heartbeat information sending frequency and adaptive bit rate. If it is appropriate, proceed to step S7; if it is not appropriate, adjust the multicast protocol until the image quality requirements of the teaching content are met.

[0152] Furthermore, when judging whether the multicast protocol meets the requirements, specifically: First, the frequency of sending heartbeat information can reflect the stability and load of the network. If the frequency of sending heartbeat messages is high, it means that the network status is good, and it is suitable to choose an efficient multicast protocol, such as Internet Group Management Protocol (IGMP), Simple Multicast Protocol (SMP) or User Datagram Protocol (UDP), to ensure timely transmission of data streams and high-quality experience; if the frequency of sending heartbeat messages is low, it means that the network status is poor. At this time, you should choose a highly adaptable multicast protocol, such as Protocol Independent Multicast (PIM), Distance Vector Multicast Routing Protocol (DVMRP) or Real-time Transport Protocol (RTP), to ensure that data can still be effectively transmitted under unstable network conditions.

[0153] Secondly, the adaptive bitrate directly affects the quality of the video stream. For example, if the current adaptive bitrate is high (such as 4K (20Mbps) or 1080p (16Mbps)), it is necessary to select a multicast protocol that supports higher bandwidth, such as Internet Group Management Protocol (IGMP) or Multiprotocol Label Switching (MPLS) to ensure the quality of teaching content and transmission efficiency. For lower adaptive bitrates (such as 720p (8Mpbs)), it is reasonable to select simpler multicast protocols such as Protocol Independent Multicast (PIM) or Distance Vector Multicast Routing Protocol (DVMRP), because these protocols can work effectively in medium or low bandwidth environments.

[0154] Finally, by comprehensively evaluating the sending frequency of heartbeat information and the adaptive bit rate, the corresponding selection criteria are set to ensure that the adopted multicast protocol can effectively support the needs of multi-screen interaction in different teaching scenarios, improve user experience and teaching effect. In this way, the multicast protocol can be dynamically adjusted to adapt to different network environments and device characteristics, thereby ensuring the efficiency of data transmission and the reliability of teaching content quality.

[0155] In some embodiments, the multicast protocol may be, but is not limited to: Internet Group Management Protocol (IGMP), Protocol Independent Multicast (PIM), Distance Vector Multicast Routing Protocol (DVMRP), Multi-Protocol Label Switching (MPLS), Real-time Transport Protocol (RTP) and Real-time Transport Control Protocol (RTCP), Simple Multicast Protocol (SMP) and Source Specific Multicast (SSM). According to different teaching scenarios and usage areas (for example: ordinary classrooms, high-definition lectures or large-scale events, such as remote multinational or remote areas, rural areas, etc.), combined with the sending frequency of heartbeat information and adaptive rate code, the above multicast protocol is reasonably selected to meet the requirements of efficient transmission, high-quality multi-screen interaction, and enhance the teaching experience. Specifically, in this embodiment, the multicast protocol adopts the Distance Vector Multicast Routing Protocol (DVMRP).

[0156] In some embodiments, the teacher electronic device is responsible for creating a multicast stream and broadcasting the screen content to the student electronic devices according to a selected multicast protocol.

[0157] S7, the teacher's electronic device executes step S4 again to detect whether the image quality of the teaching content meets the requirements. If yes, it goes to step S8, if not, it goes to step S5 again.

[0158] Furthermore, the teacher's electronic device executes step S4 again and detects whether the image quality received by the student's electronic device meets the requirements. Specifically, if the image quality meets the requirements, it proceeds to step S8, and the teacher can request a specific student device to share its screen content. If the image quality does not meet the requirements, the system will return to step S5, readjust the adaptive bit rate, and then proceed to step S6 to further adjust the multicast protocol. After multiple adjustments, the system ensures that all devices can synchronously receive high-quality teaching content, improving the classroom interaction effect.

[0159] S8. The teacher electronic device sends a request to specify a specific student electronic device to send its screen content via unicast. After receiving the request, the student electronic device will use Android's MediaProjection API to capture its screen content and encode the content into the format required by the teacher. After the encoding is completed, the student electronic device transmits the encoded content to the teacher electronic device. The teacher electronic device receives and decodes the content and finally displays the student's screen content on the teacher electronic device.

[0160] In some embodiments, multiple request methods are linked to different teaching scenarios, and teachers can obtain students' screen content in different ways, including: in ordinary classrooms, teachers can use instant messaging tools (such as WeChat groups, QQ, etc.) to send requests, and students can quickly respond and share their screens after receiving them; in large-scale lecture scenarios, teachers can use special mobile applications (such as the official APP of the teaching platform), which provides a "Request Screen Sharing" button, and students only need to click to send requests to teachers. In addition, the application can also integrate real-time feedback functions. After clicking the request, students can choose the application or screen area to share, simplifying the operation process and improving the efficiency of classroom interaction. At the same time, the application displays the request status in real time to ensure unimpeded communication between teachers and students; in remote teaching in remote areas, teachers can use voice recognition technology to request students to share screen content through voice commands. Student devices can recognize teachers' voice requests in real time and feedback operation results, which can reduce dependence on complex operations and improve the convenience of interaction. In summary, these different request mechanisms can be flexibly applied according to specific teaching scenarios to improve the interactivity and effectiveness of teaching.

[0161] In some embodiments, after obtaining the students' screen content, the teacher can perform multi-content comparison and collaborative feedback to enhance classroom interaction and learning effects. For example, in an ordinary classroom, the teacher can display the screen content of multiple students at the same time, compare their problem-solving ideas in real time, help students learn from each other and identify differences in understanding. In large lectures, teachers can collect student feedback through online voting tools, and aggregate the screen content of different groups for discussion and analysis to promote in-depth thinking collisions. In remote teaching in remote areas, students can answer verbally through a microphone, and teachers use voice recognition technology to convert these voices into text to understand students' views or questions. The teacher can then combine these contents with the screen content shared by the students to provide more targeted feedback. For example, if a student shows the problem-solving process on the screen, the teacher can directly point out the errors in his ideas or provide further explanations based on the student's voice feedback.

[0162] In addition, teachers can annotate students' screen content in real time and provide specific suggestions, while students can also provide feedback to teachers, thus achieving two-way interaction. Finally, teachers can also save screen content at different time periods for historical analysis, compare students' progress, develop personalized learning plans, and improve teaching effectiveness. Through these mechanisms, teachers can optimize teaching strategies and promote more efficient learning for students.

[0163] S9. Continuously monitor the online status of each device to ensure that all devices in the classroom environment are synchronized and meet teaching requirements.

[0164] In some embodiments, during real-time monitoring, the performance of each device is comprehensively evaluated by regularly exchanging heartbeat information and monitoring multi-dimensional indicators such as adaptive bitrate. Heartbeat information reflects the online status of the device and the quality of the network connection, while the adaptive bitrate shows the current quality of the video stream and bandwidth usage. For example, a teacher can monitor the heartbeat frequency of a student's device to determine whether the network is stable; at the same time, observe the adaptive bitrate. If a student's bitrate is low, the teacher can proactively ask the student if there are network problems or technical difficulties. Through these multi-dimensional monitoring indicators, teachers can identify and resolve potential technical problems in a timely manner to ensure smooth classroom interaction.

[0165] Furthermore, classroom management can be further optimized by collecting the power, CPU usage, and memory usage of the device. For example, teachers can check the power status of each student's device and promptly determine whether they need to remind students to charge to avoid device shutdown or freezing due to insufficient power. If the CPU usage of a device is too high, the system can automatically suggest that students close unnecessary applications to free up resources and ensure smooth video playback and classroom interaction. Through these comprehensive measures, teachers can better maintain the classroom environment and improve teaching effectiveness.

[0166] Furthermore, an intelligent early warning mechanism can be set up to automatically send a notification to the teacher when a device's heartbeat message is lost more than three times, indicating that the device may have a network problem. For example, if a student's device frequently loses heartbeats, the teacher can proactively ask the student to help them solve the network connection problem and avoid class interruptions.

[0167] Furthermore, the system can automatically adjust teaching strategies based on real-time monitoring data. For example, if a student's network delay reaches 500 milliseconds, the system can temporarily disable the student's screen sharing function to ensure that other students' experience is not affected. This dynamic adjustment can maintain the overall stability of the class.

[0168] Furthermore, heartbeat messages and device status data can be collected regularly to generate usage reports. For example, teachers can view the performance reports of each device after class to understand which devices perform well and which devices have problems, thereby optimizing future device configuration and teaching arrangements. Such data analysis can help teachers better adapt to students' needs.

[0169] Furthermore, instant feedback tools can be introduced to allow students to quickly report their experiences or problems in class. For example, at the end of each class, students can submit their feelings, such as "the picture is frozen" or "the sound is not clear", through the feedback button in the app. Teachers can view these feedbacks instantly and make targeted adjustments.

[0170] Furthermore, heartbeat messages and device status data can be stored in the cloud for teachers to analyze and replay teaching afterwards. For example, teachers can view classroom monitoring data from the past week and identify which periods of time the device performed poorly, so as to improve teaching methods and technical support in a targeted manner.

[0171] In summary, the optimization of the adaptive bit rate, multicast protocol and heartbeat information sending frequency in the three embodiments is adapted to the application of different teaching scenarios, as shown in Table 1:

[0172] It can be seen that in this method, by optimizing the heartbeat information sending frequency, adaptive bit rate and multicast protocol, the effect of multi-screen interaction in different teaching scenarios is significantly improved. Specifically: the effective combination of heartbeat information sending frequency, adaptive bit rate and multicast protocol provides the best technical support for different scenarios. In high-demand scenarios (such as large lectures or activities), the configuration of Example 1 (4K (20Mbps), IGMP, heartbeat information sending frequency every 1 second) ensures efficient synchronization and high-definition display of multiple devices, while monitoring the device status in real time to minimize the risk of network delay and device disconnection. In the standard classroom scenario, Example 2 (1080 (16Mbps), PIM, heartbeat information sending frequency every 3 seconds) ensures stable and efficient transmission of teaching content through moderate parameters to meet classroom interaction needs. In a low-bandwidth environment, Example 3 (720p (8Mbps), DVMRP, heartbeat information sending frequency every 6 seconds) effectively reduces network transmission pressure by reducing the heartbeat frequency and adaptive bit rate, ensuring stable interactive fluency and ensuring the smooth progress of teaching.

[0173] It can be seen that the flexible combination of the above three enables the system to accurately optimize teaching scenarios in different environments and regions, thereby improving classroom interactivity and the efficiency of teaching content transmission, and ensuring the best user experience under various network environments and equipment conditions.

[0174] It should be understood that the various forms of steps shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel or sequentially, as long as the desired results of the technical solution of this application can be achieved, and this document is not limited here.

[0175] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A multi-screen interactive method based on dynamic network adjustment and applicable to multiple teaching scenarios, characterized in that: The following steps are involved: S1. Activate the teacher's electronic device and the student's electronic device using the same activation code; S2, regularly sending a heartbeat message from the teacher electronic device to each of the student electronic devices to verify whether each device is online; S3, judging whether the sending frequency of the heartbeat information meets the requirements, if yes, proceeding to step S4; If it does not meet the requirements, adjust the frequency of sending heartbeat information and record the adjustment; Wherein, adjusting the frequency of sending the heartbeat information includes: dynamically adjusting the frequency of sending the heartbeat information according to the real-time network conditions and the preset degree of interaction of the teaching scene; S4, the teacher's electronic device sends the teaching content to each of the student's electronic devices through a multicast protocol; S5, judging whether the adaptive bit rate meets the requirements, if so, proceeding to step S6, if not, adjusting the adaptive bit rate and recording the adjustment; S6, judging whether the multicast protocol meets the requirements, if so, proceeding to step S7; if not, adjusting the multicast protocol and recording the adjustment; S7, the teacher electronic device executes step S4 again to detect whether the image quality displayed by the student electronic device meets the requirements, if yes, then proceed to step S8, if not, then proceed to step S5 again; S8, the teacher electronic device sends a request to the designated student electronic device, and after receiving the request, the student electronic device transmits its screen content to the teacher electronic device; S9. Continuously monitor the online status of each of the electronic devices to ensure that all devices in the classroom environment remain online and meet teaching requirements.

2. The method according to claim 1, characterized in that The preset teaching scenario interaction level indicates the frequency and quality of communication between teachers and students, and generally includes: real-time questions and answers, speed of student feedback, degree of participation in group discussions, and frequency of interaction using electronic devices.

3. The method according to claim 2, characterized in that The adjusting the adaptive bit rate comprises: The adaptive bit rate is dynamically adjusted according to the image quality of the electronic device and the adjusted heartbeat information sending frequency.

4. The method according to claim 3, characterized in that The adjusting the multicast protocol comprises: According to the adjusted heartbeat information sending frequency and the adaptive rate code, the multicast protocol is adjusted to ensure that the quality of the teaching content is further optimized.

5. The method according to claim 2, characterized in that: In step S2, the verification of whether each device is online includes: If the timestamp of the received heartbeat message is invalid or the heartbeat message is not received within the predetermined time, the status of the device is marked as "offline", the error is recorded and a notification is sent to the teacher's electronic device; if the number of consecutive heartbeat timeouts exceeds the threshold, it is marked as "fault" and the corresponding fault handling process is started.

6. The method according to claim 1, characterized in that The teaching scenarios include: ordinary classrooms, large lectures or remote teaching in remote areas.

7. The method according to claim 1, characterized in that The adaptive bitrate has a resolution range of 120p to 4K, and a bitrate range of 1 Mbps to 40 Mbps.

8. The method according to claim 1, characterized in that The heartbeat information sending frequency ranges from 100 milliseconds to 30 seconds.

9. The method according to claim 1, characterized in that: Step S4 further includes: the teacher's electronic device applies for screen recording by calling an Android system function and creates an encoder.

10. The method according to claim 9, characterized in that The encoder is H.265.

Images