Smart classroom system

Abstract

The embodiment of the invention belongs to a smart classroom technology, and provides a smart classroom system. The intelligent classroom system comprises at least two intelligent interaction panels; the intelligent interaction tablet is configured to perform device discovery interaction with at least one target device through broadcast; the target equipment is terminal equipment and / or other intelligent interaction tablets in the intelligent classroom system; establishing a TCP communication connection with the target device; and after the TCP communication connection is established, carrying out screen transmission and / or intelligent education interaction with the target equipment. The application scene of the smart classroom system is expanded, the compatibility of the smart classroom system is improved, and the use requirements of users are better met.

Description

Technical Field

[0001] The present application embodiment relates to smart classroom technology, and more specifically, to a smart classroom system. Background Art

[0002] A smart classroom is a typical materialization of a smart learning environment and a high-end form of multimedia and network classrooms. It is a new type of classroom built with the help of Internet of Things technology, cloud computing technology, and intelligent technology. This new type of classroom includes tangible physical space and intangible digital space. It uses various types of intelligent equipment to assist in the presentation of teaching content, facilitate the acquisition of learning resources, and promote classroom interaction.

[0003] At present, smart classrooms are usually deployed with smart classroom systems, which usually include multiple devices, and the devices interact with each other to achieve educational and teaching purposes. However, existing smart classroom systems have poor device compatibility and cannot meet user needs. Summary of the invention

[0004] The exemplary implementation of the present application provides a smart classroom system, which can improve the compatibility of smart classroom system equipment and better meet user needs.

[0005] In a first aspect, the present application provides a smart classroom system, the smart classroom system comprising: at least two smart interactive tablets; the smart interactive tablets are configured as:

[0006] Perform device discovery interaction with at least one target device through broadcasting; the target device is a terminal device and / or other smart interactive tablets in the smart classroom system;

[0007] Establishing a TCP communication connection with the target device;

[0008] After the TCP communication connection is established, screen transmission and / or smart education interaction is performed with the target device.

[0009] Optionally, the smart interactive tablet is a discoverable device, and the smart interactive tablet is configured as follows:

[0010] Receiving a device discovery request broadcasted by the target device;

[0011] Based on the device discovery request, sending a device discovery reply to the target device, wherein the device discovery reply carries the identifier of the smart interactive tablet, the IP address and the port number of the smart interactive tablet;

[0012] A TCP communication connection is established with the target device according to the IP address and port number.

[0013] Optionally, the smart interactive tablet is a discovery device, and the smart interactive tablet is configured as follows:

[0014] Broadcast device discovery request;

[0015] Receive a device discovery response returned by the target device, and obtain the IP address and port number of the target device;

[0016] A TCP communication connection is established with the target device according to the IP address and port number of the target device.

[0017] Optionally, the smart interactive tablet is configured as:

[0018] Establishing a connectionless multicast channel for a multicast group and a multicast listening channel;

[0019] Sending multiple data packets of at least one frame of screen transmission data to one or more target devices in the multicast group through the multicast channel, and caching the sent data packets;

[0020] When a packet loss compensation command of a target device in the multicast group is monitored through the multicast monitoring channel, a target data packet indicated by the packet loss compensation command is extracted from a cache, and the target data packet is resent to the target device through the multicast channel.

[0021] Optionally, the smart interactive tablet is configured as:

[0022] If the number of frames corresponding to the data packets in the cache is greater than the preset number of frames, at least one frame of the data packets of the screen transmission data is deleted in the order of the frame numbers from small to large.

[0023] Optionally, the smart interactive tablet is configured as:

[0024] Joining a multicast group established by the target device based on a connectionless multicast channel;

[0025] Receiving multiple data packets of at least one frame of screen transmission data sent by the target device through the multicast channel of the multicast group, and buffering the received data packets;

[0026] Determine whether packet loss occurs based on the currently received data packet and the last received data packet in the cache;

[0027] If the target data packet is lost, a packet loss compensation command is sent to the target device through the multicast monitoring channel of the multicast group; the packet loss compensation command is used to indicate that the target data packet is lost;

[0028] The target data packet retransmitted by the target device is received through the multicast channel.

[0029] Optionally, the smart interactive tablet is configured as:

[0030] If the number of frames corresponding to the data packets in the cache is greater than the preset number of frames, then taking out at least one frame of the data packet of the screen transmission data in the order of the frame numbers from small to large for decoding and playback;

[0031] and / or,

[0032] If the data packets of the first N frames of screen transmission data in the cache have been received, the data packets of the first N frames of screen transmission data are taken out in sequence for decoding and playback, where N is greater than or equal to 1.

[0033] Optionally, the header of the data packet includes the following fields:

[0034] The first field is used to identify the frame number corresponding to the data packet;

[0035] The second field is used to identify the sequence number of the data packet;

[0036] The third field is used to identify whether the data packet is the first data packet of the frame;

[0037] The fourth field is used to identify whether the data packet is the last data packet of the frame;

[0038] The fifth field is used to identify the length of the frame to which the data packet belongs.

[0039] Optionally, the smart interactive tablet is configured as:

[0040] Perform at least one of the following smart education interactions with the target device:

[0041] Wireless annotation, file transfer, reverse control, reverse screen transmission.

[0042] Optionally, the smart interactive tablet is configured as:

[0043] Based on the screen transmission protocol framework, screen transmission is performed with the target device; the screen transmission protocol framework includes a mirror screen transmission sub-protocol framework and a broadcast screen transmission sub-protocol framework, the broadcast screen transmission sub-protocol framework is used to transmit screen transmission data in one-to-multiple scenarios based on a connectionless protocol, and the mirror screen transmission sub-protocol framework is used to transmit screen transmission data in multiple-to-one scenarios based on the TCP protocol.

[0044] In a second aspect, the present application provides a smart interactive tablet, which is applied to the smart classroom system described in any one of the first aspects, and the smart interactive tablet is configured as:

[0045] Perform device discovery interaction with at least one target device through broadcasting; the target device is a terminal device and / or other smart interactive tablets in the smart classroom system;

[0046] Establishing a TCP communication connection with the target device;

[0047] After the TCP communication connection is established, screen transmission and / or smart education interaction is performed with the target device.

[0048] In a third aspect, the present application provides a smart interactive tablet, the tablet comprising:

[0049] A device discovery module, used to perform device discovery interaction with at least one target device through broadcasting; the target device is a terminal device and / or other smart interactive tablets in the smart classroom system;

[0050] A main service module, used to establish a TCP communication connection with the target device;

[0051] The sub-business module is used to perform screen transmission and / or smart education interaction with the target device after establishing the TCP communication connection.

[0052] In a fourth aspect, the present application provides a computer-readable storage medium having computer-executable instructions stored thereon. When the computer-executable instructions are executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0053] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect or the second aspect.

[0054] The smart classroom system provided by the present application includes at least two smart interactive tablets, wherein the smart interactive tablet can perform device discovery interaction with at least one target device through broadcasting, and after discovering the target device, establish a TCP communication connection with the target device to perform screen transmission and / or smart education interaction with the target device based on the TCP communication connection. In the smart classroom system provided by the present application, the smart interactive tablet discovers the target device through broadcast discovery and establishes a TCP communication channel with it for interaction. This method of device discovery through broadcasting only requires the smart interactive tablet and the target device to be able to send and receive broadcast information, and there are no other requirements for the system and hardware of the corresponding device. Therefore, in this way, the dependence on devices or systems in different interaction scenarios in the smart classroom system can be stripped, so that more types of devices can be connected to the smart classroom system for interaction, expanding the application scenarios and better meeting the user's usage needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0056] Figure 1 This is a schematic diagram of the equipment structure in a smart classroom system;

[0057] Figure 2 It is a Miracast protocol architecture;

[0058] Figure 3 A schematic diagram of the hardware configuration of a smart interactive tablet provided in this application;

[0059] Figure 4 A schematic diagram of the internal interaction process of a smart classroom system provided for this application;

[0060] Figure 5 A schematic diagram of the internal interaction process of the second smart classroom system provided for this application;

[0061] Figure 6 A schematic diagram of a screen transmission protocol framework applied to a smart classroom system provided in this application;

[0062] Figure 7 A flowchart of a screen transmission method for a smart classroom system provided in this application;

[0063] Figure 8 A flowchart of the second smart classroom system screen transmission method provided in this application;

[0064] Fig. 9 A schematic diagram of the internal interaction of a smart classroom system provided for this application;

[0065] Fig.10 A software design diagram for a smart classroom system provided in this application;

[0066] Fig.11 A schematic diagram of the structure of a smart interactive tablet provided in this application. DETAILED DESCRIPTION

[0067] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0068] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0069] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0070] The following is an explanation of some of the terms involved in this application:

[0071] Smart interactive tablet: an integrated device that uses touch technology to control the content displayed on the screen and realize human-computer interaction operations. It integrates multiple technologies such as human-computer interaction, flat-panel display, multimedia information processing and network transmission. It is the preferred solution for office, teaching, and interactive graphic presentations in the information age. It is widely used in education and teaching, corporate meetings and other fields.

[0072] Miracast: A wireless display standard based on Wi-Fi Direct. Devices that support this standard (such as Android phones and Linux computers) can share video images wirelessly. For example, a mobile phone can use Miracast to play videos or photos directly on a TV or other device without any connection cables.

[0073] Airplay: A standard screen transmission protocol that allows images, audio, video, etc. to be transmitted to display devices that support Airplay via Wi-Fi.

[0074] Hotspot networking: a technology that sets a device as a wireless hotspot, and other devices connect to the wireless hotspot to achieve interconnection and intercommunication, thus completing the networking.

[0075] Socket: An abstraction of an endpoint for two-way communication between application processes on different hosts in a network. A socket is one end of process communication on a network, providing a mechanism for application layer processes to exchange data using network protocols.

[0076] User Datagram Protocol (UDP): It is a connectionless transport layer protocol in the Open System Interconnect (OSI) reference model. The connection here refers to the communication resources established by the communicating parties to serve each other, such as sockets. Since UDP is a transport layer protocol that does not establish a connection, it is mainly used to provide simple and unreliable information transmission services for transactions. It does not transmit data packets in a grouping order. The packet transmission order is checked and sorted by the application layer.

[0077] Transmission Control Protocol (TCP): It is a connection-oriented, reliable, byte stream-based transport layer communication protocol in the OSI reference model.

[0078] Screen mirroring: refers to a technology that mirrors the content on the display screen of a mobile phone, computer or other smart device to other devices (such as TVs, projectors, etc.).

[0079] Three-way handshake means that when establishing a TCP connection, the client and the server will send a total of three segments to establish a TCP communication connection between the two. When establishing a normal TCP network connection between the client and the server, the client first sends a Synchronize Sequence Numbers (SYN) message, and the server uses SYN+Acknowledge character (ACK) to respond to indicate that the message has been received; finally, the client responds with an ACK message. In this way, a reliable TCP connection can be established between the client and the server, and data can be transmitted between the client and the server.

[0080] Real-time Transport Protocol (RTP): is a network transmission protocol that specifies the standard data packet format for transmitting audio and video over the Internet. RTP is used together with RTP Control Protocol (RTCP), and it is built on top of UDP in the data transmission protocol framework.

[0081] Real Time Streaming Protocol (RTSP): It is an application layer protocol in the TCP / IP protocol system. This protocol defines how applications can effectively transmit multimedia data over IP networks. RTSP is located above RTP and RTCP in the architecture of the transmission protocol and uses TCP or UDP to complete data transmission.

[0082] In teaching scenarios, there are usually two roles, students and teachers. Students usually form groups. Figure 1 This is a schematic diagram of the equipment structure in a smart classroom system. Figure 1 As shown, the smart classroom system may include one or more smart interactive tablets (also known as large screens). Among them, the smart interactive tablet controlled and used by the teacher is also called the teacher screen, and the student group is usually equipped with a smart interactive tablet, also known as the group screen. The devices inside the smart classroom system usually interact based on a local area network. The smart interactive tablets in the smart classroom system can be connected to the local area network through a wired connection or a wireless connection to communicate with each other. The above-mentioned wired connection can be achieved, for example, by connecting the smart interactive tablet to a local area network composed of a switch and a router. The wireless connection can be achieved, for example, by connecting the smart interactive tablet to a wireless router, or by connecting the smart interactive tablet to the local area network through hotspot networking.

[0083] In addition, terminal devices (such as mobile phones, tablets, computers, etc.) can also be connected to the smart classroom system. The switches and routers that enable the smart interactive tablet in the smart classroom system to access the local area network, as well as the terminal devices, can belong to the smart classroom system or be independent of the classroom system. Figure 1 The terminal device can access the local area network where the smart classroom system is located through a wireless connection, for example, by accessing the wireless router of the local area network where the smart classroom system is located, or by accessing the wireless hotspot created by any smart interactive tablet in the smart classroom system.

[0084] There are usually complex interaction requirements within the smart classroom system and between devices connected to the local area network where the smart classroom system is located. For example, the teacher's terminal device usually needs to project the courseware to the teacher screen and / or the group screen for students to view the courseware, or the classroom screen usually needs to project the courseware to the group screen and / or the student's terminal device for students to see the courseware more clearly, which requires screen transmission and interaction between the terminal device and the smart interactive tablet.

[0085] In addition, smart classroom systems usually have many smart education interaction requirements, such as whiteboard collaboration, file transfer, remote control, wireless annotation, reverse screen transmission, etc.

[0086] File transfer refers to the transfer of files between smart interactive tablets, or between smart classroom systems and terminal devices.

[0087] The remote control refers to the smart interactive tablet or terminal device itself, which can remotely control other smart interactive tablets and / or terminal devices like a remote control, thereby realizing remote control of other devices.

[0088] Wireless annotation means that during the screen sharing interaction process, the end receiving the screen sharing data can annotate the screen during the screen sharing process.

[0089] Whiteboard collaboration refers to the ability for smart interactive tablets to interact with each other in a collaborative manner. For example, different smart interactive tablets can share the same display screen and operate on the display screen at the same time.

[0090] Reverse screen transmission means that the end initiating the screen transmission can also receive the screen transmission data sent by the other end, thereby realizing the screen transmission of the other end's screen to the device initiating the screen transmission.

[0091] At present, the interaction between devices in the smart classroom system often relies on standard interaction protocols. For example, screen projection between devices is usually based on standard protocols such as Miracast and Airplay. However, the above standard interaction protocols often have limitations. For example, Airplay is limited to devices that support the IOS system, while other devices cannot interact with each other based on Airplay. Figure 2 It is a Miracast protocol architecture, such as Figure 2 As shown in the figure, the Miracast screen projection protocol is a combination of a series of protocols such as device discovery, RTSP, and RTP. Device discovery relies on the underlying P2P device discovery function of Wi-Fi, and this device discovery function is strongly related to hardware and drivers. Only devices with specific hardware or systems can support this device discovery function, such as devices with specific systems such as Android and Linux. This results in many limitations in the interaction between devices in the smart classroom system. For devices that do not support the above protocols, normal interaction functions in the smart classroom scenario cannot be realized.

[0092] At present, device discovery in a local area network by broadcasting is a simple and highly usable device discovery method. In this way, only the devices connected to the local area network need to be able to receive and send messages based on the local area network, that is, the mutual discovery of devices in the local area network can be achieved without any other special requirements for the systems and hardware deployed by the devices themselves. The inventors consider that if this device discovery method can be applied to the smart classroom system, the device compatibility of the current smart classroom system can be improved, and the performance of the smart classroom system can be improved.

[0093] In view of this, the present application provides a smart classroom system, which can realize mutual discovery between devices within the system through broadcasting, instead of adopting the standard communication protocol in the prior art, and does not need to rely on Wi-Fi P2P and other device discovery functions that need to rely on specific hardware and systems. Therefore, the parts related to the device system and hardware in the internal interaction process of the smart classroom system are stripped away, and the device dependence of the interaction within the smart classroom system is weakened. In other words, all types of devices can be connected to the smart classroom system for interaction, thereby improving the system compatibility of the smart classroom system and improving the user experience.

[0094] Figure 3 This is a hardware configuration diagram of a smart interactive tablet provided in this application. Figure 3 As shown, in some embodiments, the smart interactive tablet 200 may include a controller 250, a display screen 275, a communicator 220, an input / output interface 255 (eg Figure 3 ), and at least one of the memory 260.

[0095] like Figure 3 As shown, the controller 250 includes at least one of a random access memory 251 (Random Access Memory, RAM), a read-only memory 252 (Read-Only Memory, ROM), a central processing unit 254 (Central Processing Unit, CPU), and a communication bus 256 (Bus). The communication bus connects various components.

[0096] In some embodiments, RAM 251 is used to store temporary data for the operating system or other running programs.

[0097] In some embodiments, ROM 252 is used to store various system startup instructions. In some embodiments, ROM 252 is used to store a basic input and output system, called a basic input and output system (BIOS), which is used to complete the power-on self-test of the system, the initialization of each functional module in the system, the basic input / output driver of the system, and the booting operating system.

[0098] In some embodiments, when receiving the power-on signal, the power of the smart interactive tablet 200 starts to start, and the CPU runs the system startup instruction in the ROM 252, and copies the temporary data of the operating system stored in the memory to the RAM 251, so as to start or run the operating system. When the operating system is started, the CPU copies the temporary data of various applications in the memory to the RAM 251. Then, various applications are started or run.

[0099] In some embodiments, the CPU processor 254 is used to execute operating system and application instructions stored in the memory, and to execute various applications, data and content according to various interactive instructions received from external input, so as to finally display and play various audio and video contents.

[0100] In some exemplary embodiments, the CPU processor 254 may include multiple processors. The multiple processors may include a main processor and one or more sub-processors. The main processor is used to perform some operations of the display device 200 in the pre-power-on mode and / or the operation of displaying the picture in the normal mode. The one or more sub-processors are used for an operation in the standby mode and the like.

[0101] The display screen 275 can be used for displaying images and / or handwriting. In some embodiments, the display screen 275 can include a display screen component for presenting images. In some embodiments, depending on the type of the display screen 275, a drive component for driving the display is also included. In some embodiments, the display screen 275 is a projection display screen, and can also include a projection device and a projection screen.

[0102] In some embodiments, the display panel of the display screen 275 can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 can include 1 or Q display screens 275, where Q is a positive integer greater than 1.

[0103] In some embodiments, the communicator 220 is a component for communicating with an external device or an external server according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip, and other network communication protocol chips or a near field communication protocol chip, and an infrared receiver. In some embodiments, the display device 200 may establish control signal and data signal transmission and reception between the communicator 220 and the external device or content providing device.

[0104] In some embodiments, Figure 3 As shown, the input / output interface 255 is configured to perform data transmission between the controller 250 and other external devices or other controllers 250, such as receiving video signal data and audio signal data, or command instruction data from external devices.

[0105] In some embodiments, the memory 260 may include and store various software modules for driving the display device 200, such as at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

[0106] This application does not limit the networking method of the local area network where the smart interactive tablet and terminal devices are located in the smart classroom, and it can be any networking method of the local area network in the smart classroom system in the prior art.

[0107] The technical solution of the present application is described in detail below in conjunction with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0108] As mentioned above, the smart teacher system includes a smart interactive tablet, and may also include a terminal device, or access the terminal device in a teaching scenario. The following describes how the smart interactive tablet in the smart teacher system specifically interacts with the target device. The target device mentioned here may be a terminal device and / or other smart interactive tablets in the smart classroom system. It should be understood that the following embodiments are only embodiments based on the smart interactive tablet as an example. For other devices in the local area network where the smart classroom system is connected, the implementation method of the following embodiments can be referred to, which will not be repeated here.

[0109] Figure 4 The present application provides a schematic diagram of the internal interaction process of a smart classroom system, wherein the smart classroom system includes: at least two smart interactive tablets. The method of this embodiment is applied to any of the smart interactive tablets, such as Figure 4 As shown, the method may include the following steps:

[0110] S101: Perform device discovery interaction with at least one target device through broadcasting.

[0111] In this step, the smart interactive tablet performs device discovery interaction with at least one target device through broadcasting, that is, the smart interactive tablet can simultaneously broadcast and discover at least one target device. Subsequently, the smart interactive tablet can establish a communication connection with at least one target device at the same time based on the parameters required for establishing communication obtained during the device discovery interaction process to interact. The parameters required for establishing communication can be, for example, each other's IP addresses, port numbers, etc. The port number can be a dynamically generated port number or a fixed port number.

[0112] During the device discovery interaction process, the smart interactive tablet may actively broadcast the device discovery request through the local area network, or the target device may actively broadcast the device discovery request to perform device discovery interaction.

[0113] This application does not limit the type of communication protocol based on which the above-mentioned device discovery through broadcasting is based. It can be any protocol that can realize broadcast device discovery, such as the UDP protocol. The specific process of device discovery through broadcasting between the smart interactive tablet and the target device is not limited. For example, the parameters required to establish communication can be obtained through one interaction, or it can be achieved through multiple interactions. Taking one interaction and the smart interactive tablet actively initiating an interaction request as an example, the smart interactive tablet

[0114] In this step, the smart interactive tablet performs device discovery interaction with at least one target device through broadcasting. Since this broadcasting device discovery method only requires devices connected to the same local area network to be able to send and receive information to achieve broadcast interaction, and has no special requirements for hardware and systems, this method can make the device interaction in the smart classroom scenario have better compatibility, avoiding the problem that some terminal devices or smart interactive tablets cannot achieve mutual discovery and subsequent communication due to device or system dependence.

[0115] S102: Establish a TCP communication connection with the target device.

[0116] In this step, the smart interactive tablet establishes a TCP communication connection with the target device to meet the subsequent data transmission requirements between the smart interactive tablet and the target device. Since the TCP communication connection has reliable transmission performance, the subsequent interaction performance between the smart interactive tablet and the target device can be ensured.

[0117] In this step, the way of establishing a TCP communication connection between the smart interactive tablet and the target device is related to the networking method of the smart classroom system. For example, if the smart interactive tablet and the target device are networked through a hotspot, or the local area network of the smart classroom system is realized by jointly accessing a router, the smart interactive tablet can, for example, establish a TCP communication connection with the target device through a hotspot or router networking.

[0118] S103: After establishing the TCP communication connection, screen transmission and / or smart education interaction are performed with the target device.

[0119] The screen transmission mentioned here can be the screen transmission from the smart interactive tablet to the target device, or the screen transmission from the target device to the smart interactive tablet. Depending on the number of target devices connected to the smart interactive tablet and the actual screen transmission requirements, there may be multiple forms of screen transmission. For example, if the smart interactive tablet establishes a TCP communication connection with multiple target devices at the same time, the smart interactive tablet can adopt a one-to-many screen transmission method to the multiple target devices, or only transmit the screen to one of the target devices; the multiple target devices can also adopt a many-to-one transmission method to transmit the screen to the smart interactive tablet. Among them, one-to-many transmission means that one device transmits the screen to multiple devices at the same time; many-to-one transmission means that multiple devices transmit the screen to the same device at the same time. The screen transmission method can be, for example, mirroring.

[0120] The above-mentioned smart classroom interaction is the interaction between devices in the smart classroom system or the local area network connected to the smart classroom system through data transmission between each other in order to meet the needs of education and teaching. For example, it may include any one or more of wireless annotation, file transfer, reverse control, reverse screen transmission, remote control, whiteboard collaboration, etc. It should be understood that the above-mentioned smart classroom interaction is only an example. In addition to the above-mentioned smart classroom interaction, this application does not limit whether there are other smart classroom interactions between the smart interactive tablet and the target device.

[0121] After establishing a TCP communication connection, the smart interactive tablet can perform screen transmission and / or smart education interaction with the target device based on TCP; it can also interact based on other protocols, such as the UDP protocol; some functions can be carried out based on the TCP protocol and some based on other protocols, which is not limited in this application.

[0122] Optionally, the process of establishing the above-mentioned TCP communication connection can also be referred to as the main business communication. After the main business communication is established between the smart interactive tablet and the target device, screen transmission and / or smart classroom interaction can be carried out between the smart interactive tablet and the target device, which can also be referred to as sub-business communication. After the main business communication is established, it is equivalent to that the communication channel between the smart interactive tablet and the target device has been opened. Subsequently, based on the communication channel, the smart interactive tablet can carry out any sub-business communication. This application does not limit its type and implementation method.

[0123] In this embodiment, the smart interactive tablet can perform device discovery interaction with at least one target device through broadcasting, and after discovering the target device, establish a TCP communication connection with the target device to perform screen transmission and / or smart education interaction with the target device based on the TCP communication connection. In the smart classroom system provided by the present application, the smart interactive tablet discovers the target device through broadcast discovery and establishes a TCP communication channel with it for interaction. This method of device discovery through broadcasting only requires the smart interactive tablet and the target device to be able to send and receive broadcast information, and there are no other requirements for the system and hardware of the corresponding device. Therefore, in this way, the dependence on devices or systems in different interaction scenarios in the smart classroom system can be stripped, so that more types of devices can be connected to the smart classroom system for interaction, expanding the application scenarios and better meeting the user's usage needs.

[0124] It should be understood that in the device discovery process, any smart interactive tablet in the smart classroom system can act as a discovery device and actively initiate device discovery interaction to the target device through the local area network, or it can act as a discovered device and be discovered by other devices in the local area network where the smart classroom system is located. The following takes the example of the smart interactive tablet in the smart classroom system acting as a discovery device and actively initiating device discovery interaction to the target device to explain in detail the device discovery and communication channel establishment method. Figure 5 The second smart classroom system internal interaction flow diagram provided for this application is as follows: Figure 5 As shown, the method may include the following steps:

[0125] S201, intelligent interactive tablet broadcast device discovery request.

[0126] This application does not limit the conditions for the smart interactive tablet to trigger the broadcast device discovery request, for example, it can be triggered by obtaining a broadcast discovery indication input by the user. The broadcast discovery indication can be obtained by the smart interactive tablet through the user interaction interface deployed on the smart interactive tablet, for example, the user clicks on the relevant control of the user interface to trigger the smart interactive tablet broadcast device discovery request; or, it can also be triggered by the user clicking a button on the smart interactive tablet.

[0127] Alternatively, the smart interactive tablet can also trigger a device discovery request by scanning a code. For example, after obtaining the code scanning instruction input by the user, the smart interactive tablet can scan the QR code used by the target device for device discovery to obtain the authentication information of the target device, and then broadcast the device discovery request carrying the authentication information of the smart target device through the local area network, thereby realizing the device discovery interaction with the target device. Alternatively, the smart interactive tablet can be triggered by obtaining the personal identification number (PIN) of the target device input by the user, and subsequently, the smart interactive tablet broadcasts the device discovery request carrying the PIN of the target device through the local area network to realize device discovery interaction with the target device.

[0128] The specific triggering method used is related to actual needs. For example, if the smart interactive tablet needs to discover each other with a specific device connected to the local area network of the smart classroom system, it can be triggered by scanning a code or by obtaining the PIN of the target device entered by the user; if the smart interactive tablet needs to discover each other with all devices connected to the local area network, the device discovery request issued by the smart interactive tablet may not carry authentication information, but only broadcast a general device discovery request through the local area network.

[0129] Based on the above content, this application does not limit the specific content of the device discovery request, for example, it can be a string with fixed content, and the target device can identify the device discovery request based on the fixed content. For example, when the smart interactive tablet intends to discover all devices in the local area network connected to the smart classroom system, the device discovery request can be as follows:

[0130] <Discover id=“XXX”>

[0131] Among them, Discover represents that the current request is a device discovery request, and XXX represents the identifier of the current device discovery request, for example, it may be the serial number of the device discovery request.

[0132] Correspondingly, the target device receives the device discovery request broadcast by the smart interactive tablet.

[0133] S202: The target device sends a device discovery response to the smart interactive tablet based on the device discovery request.

[0134] The device discovery reply carries the target device identifier, the target device IP address and the port number.

[0135] The identification of the target device may include, for example, any one or more of a name of the target device and a universally unique identifier (Universally Unique Identifier, UUID).

[0136] In this step, the target device generates the port number of the smart interactive tablet and assigns it to the corresponding smart interactive tablet, that is, in this case, the port number of the smart interactive tablet is not fixed, but dynamically generated. If the smart interactive tablet uses a fixed port number, since different smart interactive tablets do not know each other's port numbers, there may be a problem that the port numbers of different smart interactive tablets are the same, that is, a port number conflict problem occurs. However, if the port number of the smart interactive tablet is dynamically generated by the target device and sent to the smart interactive tablet through the device discovery reply, since the target device will generate a currently unused port number when generating the port number, port number conflicts can be avoided in this way, thereby improving the communication quality. Specifically, the present application does not limit the method by which the target device generates the port number, and it can be any method of dynamically generating the port number in the prior art, and specific reference can be made to the prior art.

[0137] Exemplarily, after obtaining the device discovery request, the target device dynamically generates the port number of the smart interactive tablet, and then encapsulates the port number of the smart interactive tablet and the identifier of the target device into the device discovery reply, and encapsulates the IP address of the target device at the network layer, and then sends the encapsulated device discovery reply to the smart interactive tablet.

[0138] Exemplarily, the device discovery response may be as follows:

[0139] <Alive id=“100”>

[0140] <daemonport> 52545< / daemonport>

[0141] <devicename> 1< / devicename>

[0142] <uuid> 506A87FD-4D2A-63E0-3F92-F521845C5049< / uuid>

[0143]

[0144] Among them, Alive id represents the serial number of the current device discovery reply, which is consistent with the serial number of the device discovery request, 52545 represents the port number of the smart interactive tablet, call 1 represents the name of the target device, and 506A87FD-4D2A-63E0-3F92-F521845C5049 is the UUID of the target device. It should be understood that the IP address of the target device can be encapsulated in the network layer and is not reflected in the reply content.

[0145] Correspondingly, the smart interactive tablet receives the device discovery response returned by the target device, decapsulates the device discovery response, and obtains the IP address and port number of the target device.

[0146] In addition to the above content, this application does not limit the packaging method of the device discovery response, and whether it includes other content for limitation, for example, it can also include other device information of the target device, which is determined according to actual usage requirements.

[0147] S203: The smart interactive tablet establishes a TCP communication connection with the target device according to the IP address and port number of the target device.

[0148] For example, the smart interactive tablet can establish a TCP communication connection with the target device according to the IP address and port number of the target device through a "three-way handshake". Exemplarily, the smart interactive tablet first sends a SYN packet to the IP address and the corresponding port number according to the IP address and port number of the target device; then, the smart interactive tablet receives the SYN+ACK packet fed back by the target device; then, the smart interactive tablet sends an ACK packet to the corresponding target device according to the above IP address and port number, thereby realizing a TCP communication connection between the smart interactive tablet and the target device.

[0149] As described above, the smart interactive tablet can be used as a discovery device or as a discovered device. The above embodiment is described using the smart interactive tablet as one end that actively initiates device discovery interaction. When the smart interactive tablet is used as a discovered device and the target device is used as one end that actively initiates device discovery, the roles of the two in the above embodiment can be swapped and will not be repeated here.

[0150] In this embodiment, the target device receives the device discovery request broadcast by the smart interactive tablet; and based on the device discovery request, sends a device discovery reply to the smart interactive tablet, the device discovery reply carries the identification of the smart interactive tablet, the IP address and port number of the smart interactive tablet; subsequently, the smart interactive tablet establishes a TCP communication connection with the port number of the target device, so that the screen transmission and / or smart classroom interaction between the smart interactive tablet and the target device can be realized based on the TCP connection. In the above implementation, firstly, since the smart interactive tablet and the target device realize mutual discovery through broadcast discovery, there are no hardware and system requirements for the device that performs device discovery, that is, it can realize mutual device discovery between any device that can be connected to the local area network where the smart classroom system is located, which can expand the application scenario of the smart classroom system and improve the compatibility of the smart classroom system. In addition, in the above implementation, the port number is generated by the target device and then allocated to the corresponding smart interactive tablet, that is, the port number is dynamically generated. In this way, the problem of port number conflict caused by fixed port numbers can be avoided, and the system application performance, as well as the stability and security of output transmission can be improved.

[0151] As described above, after the TCP communication connection between the smart interactive tablet and the target device is established, the smart interactive tablet can perform screen transmission and / or smart education interaction with the target device. The following describes how the smart interactive tablet performs screen transmission with the target device.

[0152] A possible implementation is Figure 6 A schematic diagram of a screen transmission protocol framework for a smart classroom system provided in this application is shown in FIG. Figure 6 As shown, the smart interactive tablet can transmit the screen to the target device based on the screen transmission protocol framework. The screen transmission protocol framework includes a mirror screen transmission sub-protocol framework and a broadcast screen transmission sub-protocol framework. The broadcast screen transmission sub-protocol framework is used to transmit screen transmission data in a one-to-many scenario based on a connectionless protocol, and the mirror screen transmission sub-protocol framework is used to transmit screen transmission data in a multiple-to-one scenario based on the TCP protocol. The above-mentioned connectionless protocol is a protocol that does not require the establishment of a connection relationship between the smart interactive tablet and the target device, for example, it can be a UDP protocol.

[0153] The mirror screen transmission sub-protocol framework applied to multiple transmissions to one scenario:

[0154] Reference Figure 2 In the prior art, the transmission of Miracast media streams uses RTP+UDP. Since UDP is an unreliable data transmission protocol, in the case of network jitter, problems such as frame loss and screen distortion will inevitably occur in the Miracast-based screen transmission scenario.

[0155] Optionally, in this application, for the mirror screen transmission scenario in one-to-one or multiple-to-one scenarios, the TCP protocol is used in the data transmission layer. Since TCP is a reliable data transmission protocol based on connection, in this scenario, this application will have better screen transmission performance than the prior art, and can reduce the possibility of frame loss and screen distortion in network jitter scenarios.

[0156] In the prior art, in a streaming media transmission scenario based on TCP, RTSP is usually encapsulated on the TCP upper layer to realize streaming media data transmission. However, RTSP is a universal streaming media transmission protocol, which covers a variety of transmission rules, but in a smart classroom system, only some of the transmission rules are usually needed to realize the streaming media transmission requirements in the smart classroom scenario.

[0157] Therefore, optionally, the present application simplifies the RTSP streaming media control process in the upper layer of the TCP protocol, and encapsulates a control protocol adapted to the screen transmission scenario on the basis of the TCP protocol, such as a data transmission protocol related to mouse reverse control and a protocol related to touch control (the present application does not limit the specific implementation method of the data transmission protocol, which can be an existing one in the prior art or developed by itself, Figure 6 The data transmission protocol is a schematic diagram of the media data transmission protocol developed by ourselves as an example), which can simplify the transmission signaling of screen transmission in the smart classroom scenario, so that both parties of the screen transmission interaction can respond to the signaling quickly and improve the screen transmission performance.

[0158] Optional, see Figure 6 In this application, in the mirror screen transmission scenario in the one-to-one or multiple-to-one scenario, streaming media transmission can use TCP for video transmission and UDP for audio transmission, and the specific format is not limited. This application does not limit the method of audio and video synchronization, for example, the timestamp method can be used to achieve audio and video synchronization.

[0159] Optionally, all control signaling in this application can be transmitted using TCP to ensure the transmission reliability of the control signaling. The control signaling mentioned here can be, for example, Figure 6 Reverse control signaling, mouse data acquisition, etc.

[0160] The sub-protocol framework for broadcast screen transmission in the one-to-many transmission scenario:

[0161] The above-mentioned mirror screen transmission is implemented based on the connection-oriented TCP protocol. The TCP protocol has the characteristics of reliable data transmission, so there will be no problem of packet loss causing screen distortion under normal network environment, which is suitable for multiple transmissions and one screen transmission scenario. However, in actual applications, there are scenarios where the screen image of one device needs to be transmitted to multiple other devices. For example, the teacher may need to broadcast the teacher's screen to all terminal devices so that students can see the content of the teacher's screen more clearly and conveniently. If TCP is used, it is also necessary to establish multiple persistent and stable connections. Even when there are dozens of screens transmitted from one device, the hardware performance requirements for the one screen transmission data sending device are getting higher and higher.

[0162] Connectionless protocols (such as UDP) do not require a connection between the smart interactive tablet and the target device, so the hardware performance requirements of the screen transmission sender are relatively low. Figure 6 For the screen transmission scenario of one-to-many transmission, this application can use the UDP-based RTP protocol for broadcast screen transmission.

[0163] Optional, see Figure 6 , this application can ensure the quality of streaming media data transmission in the one-to-many scenario through the packet loss compensation strategy developed by itself. Optional, continue to refer to Figure 5 ,This application can also introduce a multicast protocol to control the transmission range of the broadcast.

[0164] It should be understood that the one-to-multiple screen transmission scenario mentioned here can be that the smart interactive tablet transmits the screen to multiple target devices at the same time, or the terminal device transmits the screen to multiple target devices at the same time, that is, the smart interactive tablet can serve as the receiving end or the sending end of the frame screen transmission data.

[0165] The following takes the example of screen transmission from a smart interactive tablet to one or more target devices to illustrate how the smart interactive tablet can achieve one-to-many screen transmission in a smart classroom scenario through specific embodiments.

[0166] Figure 7 A flowchart of a screen transmission method for a smart classroom system provided in this application is shown in FIG. Figure 7 As shown, the method may include the following steps:

[0167] S301. The smart interactive tablet establishes a connectionless multicast channel of a multicast group and a multicast monitoring channel.

[0168] The above-mentioned connectionless multicast channel, that is, the establishment of a multicast channel based on a transmission protocol that does not require the establishment of a connection, can not occupy too much communication resources of the smart interactive tablet in this way, reducing the requirements for the device performance of the smart interactive tablet. The present application does not limit the implementation method of the connectionless multicast channel for the smart interactive tablet to establish a multicast group. For example, it can be a multicast channel for establishing a multicast group based on UDP, as well as a multicast listening channel. Among them, the multicast channel is used for the smart interactive tablet to send data to the target device in the multicast group, and the multicast listening channel is used to monitor whether there is a target device in the multicast group that loses data packets.

[0169] This application does not limit the specific number of target devices included in the multicast group, which can be set according to the actual usage scenario.

[0170] Accordingly, the target device joins the multicast group of the target device.

[0171] S302. The smart interactive tablet sends multiple data packets of at least one frame of screen transmission data to one or more target devices in the multicast group through a multicast channel, and caches the sent data packets.

[0172] The at least one frame of screen transmission data can be, for example, data obtained by encoding after the smart interactive tablet takes a screenshot of its own display screen. This application does not limit the specific encoding method, which is set according to actual usage requirements. The smart interactive tablet obtains the screen image currently being displayed on its own screen and sends it to the smart interactive tablet as a frame of screen transmission data for playback to achieve the screen transmission process.

[0173] Since a frame of screen transmission data is often large, and the size of the data packet that can be sent at one time by the existing data transmission protocol is usually limited, after obtaining at least one frame of screen transmission data, the smart interactive tablet splits the at least one frame of screen transmission data into multiple data packets according to the data transmission requirements of the protocol on which the multicast channel is established, so as to facilitate subsequent transmission to one or more target devices in the multicast group through the multicast channel.

[0174] This application does not limit the specific size of each data packet, for example, it can be 1024k, which is set according to actual needs. The above data packet may include, for example, a header and the frame screen transmission data after packetization. This application does not limit the structure of the header of the data packet, the number of fields included, the type of fields, and the arrangement of the fields. In a possible implementation, the header of the data packet may include, for example, one or more of the following fields:

[0175] The first field is used to identify the frame number corresponding to the data packet;

[0176] The second field is used to identify the sequence number of the data packet;

[0177] The third field is used to identify whether the data packet is the first data packet of the frame;

[0178] The fourth field is used to identify whether the data packet is the last data packet of the frame;

[0179] The fifth field is used to identify the length of the frame to which the data packet belongs.

[0180] The present application does not limit the order of the fields of the first field to the fifth field, for example, they may be the first field, the second field, the third field, the fourth field, and the fifth field in sequence. Exemplarily, the header of a data packet may include, for example: "1, 1, 1, 0, 10", indicating that the current data packet belongs to the first frame, the sequence number of the data packet is 1, and the data packet is the first data packet of the first frame data, not the first data packet of the first frame data, and the first frame includes a total of 10 data packets. It should be understood that the content included in the above data packet header is only an example. The present application does not limit whether the header of the data packet includes other fields in addition to the above fields, and it is set according to actual needs.

[0181] For example, after collecting a frame of screen data from its own screen, the smart interactive tablet can divide the frame of screen data into multiple data packets according to a preset rule, and add a corresponding packet header to each data packet. Subsequently, the smart interactive tablet can send multiple data packets of at least one frame of screen data to one or more target devices in the multicast group through a multicast channel.

[0182] In this step, the smart interactive tablet stores multiple data packets of at least one frame of screen transmission data in the cache of the smart interactive tablet, so that when the smart interactive tablet sends data to the target device, if packet loss occurs, the smart interactive tablet can obtain the lost data packets from the cached data packets and resend the lost data packets.

[0183] The size of the cache of the smart interactive tablet is related to the data transmission delay between the smart interactive tablet and the target device. For example, if the data transmission delay between the smart interactive tablet and the target device is large, the cache of the smart interactive tablet can be expanded to ensure that the smart interactive tablet can store the lost data packets after knowing the lost data packets based on the interaction process with the target device, thereby reissuing the data packets. On this basis, this application does not limit the specific size of the cache of the smart interactive tablet, and those skilled in the art can set it according to actual needs, for example, up to 100 frames of data can be stored.

[0184] Optionally, if the number of frames corresponding to the data packet in the cache is greater than the preset number of frames, at least one frame of the data packet of the screen transmission data is deleted in the order of the frame numbers from small to large. The present application does not limit the specific value of the above-mentioned preset number of frames. For example, it can be consistent with the maximum number of frames that can be stored in the cache of the smart interactive tablet. In this way, it can be ensured that the number of data packets stored in the cache is not too large, avoiding data redundancy caused by too many stored data packets, which affects the performance of the smart interactive tablet.

[0185] Correspondingly, the target device receives multiple data packets of at least one frame of screen transmission data sent by the smart interactive tablet through the multicast channel of the multicast group, and caches the received data packets. The cache mentioned here is the cache of the target device. This application does not limit the specific value of the cache of the target device, and those skilled in the art can set it according to actual needs.

[0186] In this step, the target device is provided with a cache to cache the received data packets, and then the cached data packets can be used to determine whether packet loss occurs during the data transmission process.

[0187] S303: The target device determines whether packet loss occurs based on the currently received data packet and the last received data packet in the cache.

[0188] In a possible implementation, the target device can determine whether packet loss occurs based on the header of the last received data packet and the header of the currently received data packet. Exemplarily, the header of the data packet may include the sequence number of each data packet, and the target device determines whether the sequence number of the currently received data packet is greater than the sequence number of the last received data packet plus 1. If so, it indicates that at least a data packet with a sequence number equal to the sequence number of the last received data packet plus 1 is missing between the last data packet and the currently received data packet, indicating that the target data packet is lost. The target data packet mentioned here is the data packet between the currently received data packet and the last received data packet.

[0189] If the target data packet is lost, in order to ensure the quality of screen transmission, it is necessary to obtain the lost data packet, and then execute step S304.

[0190] S304: The target device sends a packet loss compensation command to the target device through the multicast monitoring channel of the multicast group.

[0191] The packet loss compensation command is used to indicate that the target data packet is lost. The present application does not limit the actual content of the packet loss compensation command, for example, it may include the sequence number of the target data packet, which can be set according to actual needs.

[0192] In this step, the target device sends a packet loss compensation command to the smart interactive tablet through the multicast monitoring channel of the multicast group, thereby obtaining the missing data packets.

[0193] Optional, see Figure 5 ,When the target device sends the packet loss compensation command, it can be sent based on the TCP protocol to ensure that the packet loss compensation command can be reliably sent to the smart interactive tablet, thereby achieving the timely resending of the target number of packets.

[0194] S305. When a packet loss compensation command of a target device in the multicast group is monitored through the multicast monitoring channel, the smart interactive tablet extracts the target data packet indicated by the packet loss compensation command from the cache, and resends the target data packet to the target device through the multicast channel.

[0195] In this step, the smart interactive tablet monitors the packet loss compensation command of the target device in the multicast group through the multicast monitoring channel. Since the smart interactive tablet has a cache for storing data packets corresponding to the frame screen transmission data, the smart interactive tablet extracts the target data packet indicated by the packet loss compensation command from the cache and resends the target data packet through the multicast channel.

[0196] Correspondingly, the target device receives the target data packet retransmitted by the smart interactive tablet through the multicast channel.

[0197] Since the smart interactive tablet retransmits the target data packet through the multicast channel, other devices in the multicast group will also receive the target data packet. Optionally, after receiving the data packet, the target device in the multicast group can determine whether the sequence number of the currently received data packet is the same as the packet header of the historically received data packet. If so, it indicates that the target device has received a duplicate data packet, and the currently received data packet is deleted. In this way, it can be avoided that the target device that does not need to retransmit the data packet repeatedly plays the same data packet, which affects the screen transmission quality.

[0198] As described above, the smart interactive tablet can be used as one end for sending frame screen data, or as one end for receiving frame screen data. The above embodiment is an embodiment in which the smart interactive tablet is used as one end for sending frame screen data. When the smart interactive tablet is used as one end for receiving frame screen data, it is only necessary to exchange the roles of the smart interactive tablet and the target device in the above embodiment.

[0199] In this embodiment, the smart interactive tablet establishes a connectionless multicast channel for the multicast group, and accordingly, the target device joins the multicast group. Subsequently, the smart interactive tablet can send screen transmission data to the target devices in the group through the multicast group, so that the smart interactive tablet can transmit the screen to multiple target devices at the same time. Since there is no need to establish a connection between the smart interactive tablet and the target device, this method can effectively save bandwidth and reduce the dependence on the performance of the smart interactive tablet device, thereby expanding the application scenarios of the smart classroom system of this application, improving the screen transmission performance in the one-to-many scenario, and is a powerful supplement to the education and teaching scenario.

[0200] In addition, although the present application establishes a multicast channel for a multicast group based on a protocol that does not establish a connection, it realizes monitoring and retransmission of packets lost during data packet transmission by establishing a multicast monitoring channel. This implementation method can further avoid the loss of data packets in the multicast screen transmission scenario provided by the present application, improve the screen transmission quality, and ensure the performance of multicast screen transmission in the smart classroom system.

[0201] Taking the smart interactive tablet as an example, in order to realize that the smart interactive tablet can transmit the screen to the target device, the target device will play the received frame screen data in sequence after receiving the frame screen data. Figure 8 The flowchart of the second smart classroom system screen transmission method provided in this application is as follows: Figure 8 As shown, in this embodiment, when the target device plays the screen data sequentially, the following method can be used for playback:

[0202] S401, determining whether the data packets of the first N frames of screen transmission data in the cache have been received.

[0203] The above N is greater than or equal to 1. The specific value of N is related to actual needs and is not limited in this application. For example, the target device can only judge the first-ranked frame screen data, or it can judge whether the data packets of multiple frames of screen data are complete at one time.

[0204] In this step, the target device determines whether the data packets of the first N frames of screen transmission data in the cache have been received, and then determines whether the data packets of the first N frames of screen transmission data meet the conditions for playback.

[0205] If yes, execute step S402.

[0206] Optionally, if not, execute step S403.

[0207] S402, sequentially retrieve the data packets of the first N frames of screen transmission data for decoding and playback.

[0208] The above-mentioned method of decoding and playing the data packet is consistent with the method of encoding the screen transmission data. On this basis, the specific implementation method is not limited.

[0209] S403: Determine whether the number of frames corresponding to the data packet in the cache is greater than a preset number of frames.

[0210] This application does not limit the specific value of the above-mentioned preset frame number, and technicians in this field can set it according to actual usage requirements.

[0211] If so, it indicates that a large amount of cache data has been stored in the cache, that is, the current target device has been waiting for a long time to resend the data packet because the data packet of the previous N frames of screen transmission data has not been received. If the waiting continues, the screen transmission process will be stuck and the frame screen transmission data will not be played continuously. Then execute step S404.

[0212] If not, continue to wait for the data packets of the first N frames of screen transmission data to be received, or the number of frames corresponding to the data packets in the cache is greater than the preset number of frames.

[0213] S404: According to the order of frame numbers from small to large, take out a data packet of at least one frame of screen transmission data for decoding and playback.

[0214] As mentioned above, since it has taken a long time to complete the reception of the data packets of the first N frames of screen transmission data, continued waiting will cause the screen transmission to be stuck. Therefore, in this step, the target device takes out at least one frame of screen transmission data packet from the cache in the order of frame number from small to large, and decodes and plays it to avoid the occurrence of the above-mentioned screen transmission stuck problem.

[0215] In this embodiment, the target device first determines whether the data packets of the first N frames of screen transmission data in the cache have been received. If so, the data packets of the first N frames of screen transmission data are taken out in sequence for decoding and playback. If not, it determines whether the number of frames corresponding to the data packets in the cache is greater than the preset number of frames. If it is greater than the preset number of frames, at least one frame of screen transmission data packet is taken out in ascending order of frame numbers for decoding and playback.

[0216] Through the above method, on the one hand, the target device can wait for the data packets of the first N frames of screen transmission data to be received by setting the cache, that is, it can wait for the target data packets to be re-sent from the smart interactive tablet, thereby avoiding the problem of packet loss and screen distortion to a certain extent; on the other hand, when the target device cannot obtain the lost target data packet for a long time, the target device can judge the length of the current waiting time by judging the number of frame screen transmission data in the cache area, and then when the waiting time is long, even if the data packet is not received, the frame screen transmission data ranked first in the current cache can be taken out for playback. In this case, although the screen distortion problem can be caused, the more serious problem of screen transmission jamming is avoided, thus achieving a balance between avoiding screen distortion and playback fluency. In other words, even if the data packet corresponding to the current frame is incomplete, since the number of frames corresponding to the data packet in the cache is greater than the preset number of frames, that is, the target device has been waiting for a long time, at this time, the target device will still play the current frame, and will not cause excessive reduction in video playback fluency due to the intention to avoid screen distortion. The above method can improve the screen transmission performance as a whole.

[0217] As mentioned above, the smart classroom scenario mainly includes two main lines, namely the interaction between smart interactive tablets and the interaction between smart interactive tablets and terminal devices; it includes a total of three interaction scenarios, namely device discovery, main business communication, and sub-business communication. The following is a detailed description of the smart classroom system provided by this application through specific examples. Fig. 9 A schematic diagram of the internal interaction of a smart classroom system provided for this application, such as Fig. 9 As shown, the smart classroom system includes a teacher screen, a group screen, and terminal devices.

[0218] The teacher screen can interact with the terminal device, and can also interact with the group screen. The group screen can also interact with the terminal device, as explained below:

[0219] Device discovery process:

[0220] When the teacher screen interacts with the terminal device, the terminal device acts as a discovery device and broadcasts a device discovery request to the devices in the smart classroom system; the teacher screen acts as a discovered device and performs broadcast monitoring to listen to the device discovery requests broadcast by the terminal devices connected to the LAN of the smart classroom system.

[0221] When the teacher screen and the group screen interact, the group screen acts as a discovery device and broadcasts a device discovery request to the devices in the smart classroom system; the teacher screen acts as a discovered device and conducts broadcast listening to monitor the device discovery requests broadcast by the group screen in the smart classroom system.

[0222] When the group screen and the terminal device interact, the terminal device acts as a discovery device and broadcasts a device discovery request to the devices in the smart classroom system; the group screen acts as a discovered device and conducts broadcast listening to monitor the device discovery requests broadcast by the terminal device in the smart classroom system.

[0223] Main service communication:

[0224] When the teacher screen and the terminal device interact, the teacher screen acts as a server and receives the TCP communication connection establishment request sent by the terminal device. The terminal device acts as a client and sends a TCP communication connection establishment request to the user.

[0225] When the teacher screen and the group screen interact, the teacher screen acts as a server and receives the TCP communication connection establishment request sent by the group screen. The group screen acts as a client and sends a TCP communication connection establishment request to the user.

[0226] When the group screen and the terminal device interact, the group screen acts as a server and receives the TCP communication connection establishment request sent by the terminal device. The terminal device acts as a client and sends a TCP communication connection establishment request to the user.

[0227] Sub-services:

[0228] When the teacher screen and the terminal device interact, after the TCP communication connection is established between the teacher screen and the terminal device, various sub-services such as courseware demonstration, remote control, screen sharing, reverse transmission, wireless annotation, and file transfer can be carried out between the teacher screen and the terminal device. The implementation methods of the above various sub-services based on the TCP communication channel can refer to the existing technology and will not be elaborated here.

[0229] Taking the screen sharing from the terminal device to the teacher screen as an example, after the TCP communication channel is established,

[0230] The terminal device can, for example, send to the teacher screen:

[0231] <seviceMode id=”102”(current service identifier)category=”request”(request reply)>

[0232] <direction>(Interaction Direction)

[0233] opsReceive (Peer Receive)

[0234] < / direction>

[0235] <serverport>(Terminal device port number) 55645

[0237] < / serverport>

[0238] <remotecontrol / > / / (Add when reverse control is supported)

[0239]

[0240] The teacher screen can reply:

[0241] <repServiceMode id=”102” (Current service identifier, consistent with the terminal device identifier)

[0242] category=”response” (Reply)

[0243] status=”OK” (Agree to establish screen sharing interaction) code=”200” / >

[0244] After the interaction between the terminal device and the teacher screen as described above, the terminal device can send frame screen sharing data to the teacher screen to achieve screen sharing to the teacher screen. It should be understood that the above message interaction is only an example, and in actual applications, the present application does not limit the content included in the specific screen sharing interaction message.

[0245] When the teacher screen and the group screen interact, after the teacher screen and the terminal device establish a TCP communication connection, various sub-services such as file transfer, whiteboard collaboration, broadcast screen sharing, and screen sharing data reception can be carried out between the teacher screen and the group screen. The specific implementation method can refer to the prior art.

[0246] When the group screen and the terminal device interact, after the group screen and the terminal device establish a TCP communication connection, various sub-services such as courseware demonstration, remote control, screen sharing, reverse transmission, wireless annotation, and file transfer can be carried out between the group screen and the terminal device.

[0247] Optionally, intelligent interactive flat panels in the local area network accessing the smart classroom system, and the terminal device can implement the above device discovery, main services, and sub-service interactions through an application software (Application, APP) for example. Fig.10 A schematic diagram of the application software architecture for a smart classroom system provided by the present application is shown as Fig.10 shown. The application software design of the smart classroom system can be divided into four layers, namely the system layer, the protocol layer, the interface layer, and the application layer.

[0248] (1) The protocol layer includes all the protocols involved in device discovery, main services, and sub-service operations, such as device discovery protocol, screen sharing protocol, signaling and session control protocol, data transmission protocol, reverse control protocol, file transfer protocol, airplay protocol, etc. The present application does not limit the development language of the protocol. For example, it can be developed using C / C++ to generate corresponding dynamic link libraries (Dynamic Link Library, DLL).

[0249] Through the design and deployment of many protocols at the protocol layer, when the user needs to use related functions through the APP, the APP can call the corresponding protocol to support the implementation of the corresponding function. For example, if the smart interactive tablet needs to transmit the screen, the screen transmission protocol can be called through the APP to support the implementation of the screen transmission function.

[0250] (2) The interface layer encapsulates the interface of the protocol layer according to the business scenario and simplifies the calling method. For example, C interface encapsulation can be performed.

[0251] The design of the above interface layer can realize the encapsulation of the interface of the protocol layer, so that when the devices included in the local area network call the protocol through the APP, they can simply and quickly call the corresponding protocol, and then quickly realize the rapid call of related functions such as device discovery, main business, and sub-business.

[0252] (3) The application layer is developed based on the actual system and development language. For example, for screen capture, QT interface and WinDriectX interface can be used for screen capture and rendering under Windows. The rendering method is designed according to the actual project requirements and supports multi-window and single-window multi-split screen layouts.

[0253] Through the above design, the APP provided in this application can, on the one hand, provide the functions involved in the above-mentioned device discovery, main business, and sub-business; on the other hand, it can adapt to the own system of the device connected to the local area network of the smart classroom system, improve software availability, and expand application scenarios.

[0254] (4) The system layer mainly calls third-party interfaces and system APIs.

[0255] This layer is mainly used to call the device's own system functions and provide system-level support for APP function implementation.

[0256] The present application also provides a smart interactive tablet, which is applied to the smart classroom system described in any of the above embodiments, and the smart interactive tablet is configured as follows:

[0257] Perform device discovery interaction with at least one target device through broadcasting; the target device is a terminal device and / or other smart interactive tablets in the smart classroom system;

[0258] Establishing a TCP communication connection with the target device;

[0259] After the TCP communication connection is established, screen transmission and / or smart education interaction is performed with the target device. Fig.11 A schematic diagram of the structure of a smart interactive tablet provided in this application, such as Fig.11As shown, the smart interactive tablet includes: a device discovery module 11, a main business module 12, and a sub-business module 13.

[0260] The device discovery module 11 is used to perform device discovery interaction with at least one target device through broadcasting; the target device is a terminal device and / or other smart interactive tablets in the smart classroom system.

[0261] The main service module 12 is used to establish a TCP communication connection with the target device.

[0262] The sub-business module 13 is used to perform screen transmission and / or smart education interaction with the target device after establishing the TCP communication connection.

[0263] Optionally, the smart interactive tablet is a discovered device, and the device discovery module 11 is specifically used to receive the device discovery request broadcast by the target device; based on the device discovery request, send a device discovery reply to the target device, and the device discovery reply carries the identifier of the smart interactive tablet, the IP address and port number of the smart interactive tablet. The main business module 12 is specifically used to establish a TCP communication connection with the target device according to the IP address and port number.

[0264] Optionally, the smart interactive tablet is a discovery device, and the device discovery module 11 specifically broadcasts a device discovery request; receives a device discovery reply returned by the target device, and obtains the IP address and port number of the target device. The main business module 12 is specifically used to establish a TCP communication connection with the target device according to the IP address and port number of the target device.

[0265] Optionally, the sub-business module 13 is specifically used to establish a connectionless multicast channel for a multicast group, and a multicast listening channel; send multiple data packets of at least one frame of screen transmission data to one or more target devices in the multicast group through the multicast channel, and cache the sent data packets; when a packet loss compensation command of a target device in the multicast group is monitored through the multicast listening channel, the target data packet indicated by the packet loss compensation command is extracted from the cache, and the target data packet is resent to the target device through the multicast channel.

[0266] Optionally, the sub-business module 13 is further used to delete at least one frame of screen transmission data packet in ascending order of frame number if the number of frames corresponding to the data packet in the cache is greater than a preset number of frames.

[0267] Optionally, the sub-business module 13 is also used to join the multicast group established by the target device based on a connectionless multicast channel; receive multiple data packets of at least one frame of screen transmission data sent by the target device through the multicast channel of the multicast group, and cache the received data packets; determine whether packet loss occurs based on the currently received data packet and the last cached received data packet; if the target data packet is lost, send a packet loss compensation command to the target device through the multicast listening channel of the multicast group; the packet loss compensation command is used to indicate that the target data packet is lost; and receive the target data packet retransmitted by the target device through the multicast channel.

[0268] Optionally, the sub-business module 13 is also used to take out the data packets of at least one frame of screen transmission data in ascending order of frame numbers for decoding and playback if the number of frames corresponding to the data packets in the cache is greater than a preset number of frames; and / or, if the data packets of the first N frames of screen transmission data in the cache have been received, take out the data packets of the first N frames of screen transmission data in sequence for decoding and playback, where N is greater than or equal to 1.

[0269] Optionally, the header of the data packet includes the following fields: a first field, used to identify the frame number corresponding to the data packet; a second field, used to identify the sequence number of the data packet; a third field, used to identify whether the data packet is the first data packet of the frame; a fourth field, used to identify whether the data packet is the last data packet of the frame; and a fifth field, used to identify the length of the frame to which the data packet belongs.

[0270] Optionally, the sub-business module 13 is specifically used to perform at least one of the following smart education interactions with the target device: wireless annotation, file transfer, reverse control, and reverse screen transmission.

[0271] Optionally, the main business module 12 is specifically used to transmit the screen to the target device based on the screen transmission protocol framework; the screen transmission protocol framework includes a mirror screen transmission sub-protocol framework and a broadcast screen transmission sub-protocol framework, the broadcast screen transmission sub-protocol framework is used to transmit screen transmission data in one-to-multiple scenarios based on a connectionless protocol, and the mirror screen transmission sub-protocol framework is used to transmit screen transmission data in multiple-to-one scenarios based on the TCP protocol. The smart interactive tablet provided in the present application is used to execute the embodiment of the smart interactive tablet interaction method in the aforementioned smart classroom system, and its implementation principle and technical effect are similar, which will not be repeated here. It should be noted that the above Fig.11 The division of the modules shown is only a schematic diagram, and the present application does not limit the division of the modules and the naming of the modules.

[0272] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0273] The present application also provides a program product, which includes an execution instruction, which is stored in a readable storage medium. At least one control module of the smart interactive tablet can read the execution instruction from the readable storage medium, and at least one control module executes the execution instruction so that the display device implements the methods provided in the above various embodiments.

[0274] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0275] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A smart classroom system, characterized in that: The smart classroom system includes: at least two smart interactive tablets; the smart interactive tablets are configured as follows: Perform device discovery interaction with at least one target device through broadcasting; the target device is a terminal device and / or other smart interactive tablets in the smart classroom system; Establishing a TCP communication connection with the target device; After the TCP communication connection is established, screen transmission and / or smart education interaction is performed with the target device.

2. The system according to claim 1, characterized in that The smart interactive tablet is a discoverable device, and the smart interactive tablet is configured as follows: Receiving a device discovery request broadcasted by the target device; Based on the device discovery request, sending a device discovery reply to the target device, wherein the device discovery reply carries the identifier of the smart interactive tablet, the IP address and the port number of the smart interactive tablet; A TCP communication connection is established with the target device according to the IP address and port number.

3. The system according to claim 1, characterized in that The smart interactive tablet is a discovery device, and the smart interactive tablet is configured as follows: Broadcast device discovery request; Receive a device discovery response returned by the target device, and obtain the IP address and port number of the target device; A TCP communication connection is established with the target device according to the IP address and port number of the target device.

4. The system according to claim 1, characterized in that The smart interactive tablet is configured as follows: Establishing a connectionless multicast channel for a multicast group and a multicast listening channel; Sending multiple data packets of at least one frame of screen transmission data to one or more target devices in the multicast group through the multicast channel, and caching the sent data packets; When a packet loss compensation command of a target device in the multicast group is monitored through the multicast monitoring channel, a target data packet indicated by the packet loss compensation command is extracted from a cache, and the target data packet is resent to the target device through the multicast channel.

5. The system according to claim 4, characterized in that The smart interactive tablet is configured as follows: If the number of frames corresponding to the data packets in the cache is greater than the preset number of frames, at least one frame of the data packets of the screen transmission data is deleted in the order of the frame numbers from small to large.

6. The system according to claim 1, characterized in that The smart interactive tablet is configured as follows: Joining a multicast group established by the target device based on a connectionless multicast channel; Receiving multiple data packets of at least one frame of screen transmission data sent by the target device through the multicast channel of the multicast group, and buffering the received data packets; Determine whether packet loss occurs based on the currently received data packet and the last received data packet in the cache; If the target data packet is lost, a packet loss compensation command is sent to the target device through the multicast monitoring channel of the multicast group; the packet loss compensation command is used to indicate that the target data packet is lost; The target data packet retransmitted by the target device is received through the multicast channel.

7. The system according to claim 6, characterized in that The smart interactive tablet is configured as follows: If the number of frames corresponding to the data packets in the cache is greater than the preset number of frames, then taking out at least one frame of the data packet of the screen transmission data in the order of the frame numbers from small to large for decoding and playback; and / or, If the data packets of the first N frames of screen transmission data in the cache have been received, the data packets of the first N frames of screen transmission data are taken out in sequence for decoding and playback, where N is greater than or equal to 1.

8. The system according to any one of claims 4 to 7, characterized in that: The header of the data packet includes the following fields: The first field is used to identify the frame number corresponding to the data packet; The second field is used to identify the sequence number of the data packet; The third field is used to identify whether the data packet is the first data packet of the frame; The fourth field is used to identify whether the data packet is the last data packet of the frame; The fifth field is used to identify the length of the frame to which the data packet belongs.

9. The system according to any one of claims 1 to 7, characterized in that: The smart interactive tablet is configured as follows: Perform at least one of the following smart education interactions with the target device: Wireless annotation, file transfer, reverse control, reverse screen transmission.

10. The system according to claim 9, characterized in that The smart interactive tablet is configured as follows: Based on the screen transmission protocol framework, screen transmission is performed with the target device; the screen transmission protocol framework includes a mirror screen transmission sub-protocol framework and a broadcast screen transmission sub-protocol framework, the broadcast screen transmission sub-protocol framework is used to transmit screen transmission data in one-to-multiple scenarios based on a connectionless protocol, and the mirror screen transmission sub-protocol framework is used to transmit screen transmission data in multiple-to-one scenarios based on the TCP protocol.

Images