Education Internet of Things intelligent collaboration method and system based on multi-modal protocol self-adaption

Abstract

The invention relates to the field of adaptive control systems, and particularly discloses an education Internet of Things intelligent collaboration method and system based on multi-modal protocol adaptation, and the method comprises the steps: obtaining the communication characteristics of education Internet of Things equipment corresponding to a multi-modal protocol, constructing a multi-modal protocol fingerprint database according to the communication characteristics, the multi-mode protocol fingerprint database comprises protocol fingerprints corresponding to the education Internet of Things equipment; acquiring a multi-mode collaborative network corresponding to the current teaching scene; based on the multi-modal protocol fingerprint database and the multi-modal collaborative network, generating a self-adaptive collaborative control instruction for the education Internet of Things equipment in the current teaching scene; and issuing the adaptive cooperative control instruction to the education Internet of Things equipment in the current teaching scene for execution. According to the invention, the equipment cooperation efficiency and stability of the education Internet of Things under multi-scene switching can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of adaptive control systems, and in particular to an intelligent collaboration method and system for an educational Internet of Things based on multimodal protocol adaptation. Background Art

[0002] The current educational IoT system includes devices with various modal protocols, such as personal terminals, smart sockets, smart street lights, power monitoring meters, traffic asset processing terminals, smart switches, smart whiteboards, experimental sensors, AR terminals, etc. In a teaching scenario, devices with multiple modal protocols are often required to collaborate. However, when deploying educational devices with multi-modal protocols, the problem of protocol heterogeneity is faced: the use of proprietary communication protocols by devices from different manufacturers makes interoperability difficult, and existing solutions rely on manually predefined rules and cannot dynamically perceive changes in device communication characteristics; at the same time, the device collaboration strategy lacks adaptive capabilities when switching teaching scenarios, and command conflicts or response delays frequently occur, seriously interfering with the teaching process. Summary of the Invention

[0003] The present invention provides an intelligent collaboration method and system for the educational Internet of Things based on multimodal protocol adaptation, aiming to adaptively control educational Internet of Things devices with multimodal protocols in teaching scenarios, so as to improve the stability and efficiency of the collaborative control of the educational Internet of Things with multimodal protocols in different teaching scenarios.

[0004] To achieve the above objectives, the present invention proposes an intelligent collaboration method for educational Internet of Things based on multimodal protocol adaptation, which includes: Acquire communication characteristics of an educational Internet of Things device corresponding to a multimodal protocol, and construct a multimodal protocol fingerprint library based on the communication characteristics, wherein the multimodal protocol fingerprint library includes protocol fingerprints corresponding to the educational Internet of Things device; Obtain a multimodal collaborative network corresponding to the current teaching scenario. Each teaching scenario corresponds to a multimodal collaborative network. The multimodal collaborative network includes multimodal device nodes and connection edges between device nodes. Each device node corresponds to a control parameter of an educational Internet of Things device. The connection edges are determined based on the collaborative response probability between the corresponding educational Internet of Things devices. Based on the multimodal protocol fingerprint library and the multimodal collaborative network, generating adaptive collaborative control instructions for the educational Internet of Things devices in the current teaching scenario; The adaptive collaborative control instructions are sent to the educational Internet of Things devices in the current teaching scene for execution.

[0005] Optionally, obtaining communication characteristics of educational IoT devices corresponding to the multimodal protocol and constructing a multimodal protocol fingerprint library based on the communication characteristics includes: Obtain communication data packets of educational IoT devices corresponding to multimodal protocols during different teaching periods; Parsing the communication data packet to obtain the protocol type, data frame structure, transmission delay, response frequency and message verification rules as communication features; Perform unsupervised clustering on communication features to generate protocol feature clusters; The cluster center feature vector corresponding to each of the protocol feature clusters is extracted as the protocol fingerprint and stored in the fingerprint library. Each modal protocol corresponds to one protocol fingerprint.

[0006] Optionally, after obtaining the communication characteristics of the educational Internet of Things device corresponding to the multimodal protocol and constructing a multimodal protocol fingerprint library according to the communication characteristics, the method further includes: Real-time monitoring of the communication packet loss rate and response delay time of the educational IoT device; When the communication packet loss rate is greater than a packet loss rate threshold, or the response delay time is greater than a delay time threshold, extracting communication features of the abnormal communication data packet; The communication feature is subjected to Euclidean distance calculation with the protocol fingerprint in the multimodal protocol fingerprint library. If the calculated minimum Euclidean distance is greater than a distance threshold, the protocol fingerprint corresponding to the minimum Euclidean distance in the multimodal protocol fingerprint library is updated.

[0007] Optionally, before the step of obtaining the multimodal collaborative network corresponding to the current teaching scenario, the method further includes: For each teaching scenario, identifying the activity corresponding to each of the educational IoT devices in the teaching scenario; Selecting candidate educational Internet of Things devices whose activity is greater than an activity threshold from each of the educational Internet of Things devices in the teaching scenario, and mapping the candidate educational Internet of Things devices into device nodes, where each device node corresponds to one candidate educational Internet of Things device mapping; For each candidate educational IoT device, obtain a historical linkage log corresponding to the candidate educational IoT device, wherein the historical linkage target log includes historical communication data when the candidate educational IoT device collaborates with candidate educational IoT devices of other modalities in the teaching scenario; Determining, based on the historical communication data, a probability of coordinated response between each of the candidate educational IoT devices; Through the collaborative response probability between each of the candidate educational Internet of Things devices, the device nodes corresponding to each of the candidate educational Internet of Things devices are connected to obtain a multimodal collaborative network corresponding to each of the teaching scenarios.

[0008] Optionally, the identifying the activity corresponding to each of the educational IoT devices in the teaching scenario includes: Determine the working time of each of the educational Internet of Things devices in the teaching scenario, and the number of times each of the educational Internet of Things devices is triggered by teacher-student interactions; Determining the activity of each of the educational IoT devices in the teaching scenario based on the working hours and the number of times triggered by teacher-student interactions; Optionally, determining the collaborative response probability between the candidate educational IoT devices based on the historical communication data includes: Determining a historical success rate between each of the candidate educational IoT devices based on the historical communication data; Determining a protocol compatibility factor based on the protocol fingerprint corresponding to each of the candidate educational IoT devices; The collaborative response probability between each of the candidate educational Internet of Things devices is determined based on the historical success rate, the protocol compatibility factor, and the bandwidth factor.

[0009] Optionally, generating adaptive collaborative control instructions for educational IoT devices in the current teaching scenario based on the multimodal protocol fingerprint library and the multimodal collaborative network includes: Encoding the multimodal protocol fingerprint library into a node feature matrix, and using the multimodal collaborative network as an adjacency matrix of the node feature matrix, wherein each node in the node feature matrix represents a protocol fingerprint, and the connecting edges between the nodes represent the similarity between the protocol fingerprints; Inputting the node feature matrix and the adjacency matrix into a pre-trained graph convolutional network, and outputting control parameters with the goal of minimizing collaborative delay; The control parameters are converted into a device executable instruction set to obtain the adaptive collaborative control instructions corresponding to the educational Internet of Things devices in the current teaching scenario.

[0010] Optionally, before the step of inputting the node feature matrix and the adjacency matrix into a pre-trained graph convolutional network and outputting control parameters with the goal of minimizing collaborative delay, the method further includes: Constructing a training data set and a graph convolutional network to be trained, wherein the training data set includes: a sample node feature matrix obtained by encoding the multimodal protocol fingerprint library, historical multimodal collaborative networks corresponding to different teaching scenarios, and real response delays corresponding to the historical multimodal collaborative networks, wherein each sample node in the sample node feature matrix represents a protocol fingerprint, and the connecting edges between the sample nodes represent the similarity between the protocol fingerprints; Inputting the sample node feature matrix and the historical multimodal collaborative network into the graph convolutional network to be trained to perform response delay prediction to obtain a predicted response delay; The error loss between the predicted response delay and the actual response delay is calculated by the loss function, and the minimization of the error loss is taken as the optimization goal. The network parameters of the graph convolutional network to be trained are adjusted by the back propagation algorithm, and the adjustment process of the network parameters is iterated until the training stop condition is met, and the training is stopped to obtain a pre-trained graph convolutional network.

[0011] Optionally, calculating the error loss between the predicted response delay and the actual response delay using a loss function includes: Calculating a weighted mean square error between the predicted response delay and the actual response delay; determining a multimodal collaboration loss factor based on the historical multimodal collaboration network; Determining a feature constraint factor based on the sample node feature matrix; An error loss between the predicted response delay and the actual response delay is calculated based on the weighted mean square error, the multimodal synergy loss factor, and the feature constraint factor.

[0012] In the second aspect, an embodiment of the present invention also provides an intelligent collaborative system of the educational Internet of Things based on multimodal protocol adaptation. The intelligent collaborative system of the educational Internet of Things based on multimodal protocol adaptation includes a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the computer program is executed by the processor, the steps of the intelligent collaborative method of the educational Internet of Things based on multimodal protocol adaptation as described in any one of the embodiments of the present invention are implemented.

[0013] The present invention is based on the technical solution of the intelligent collaborative method of the educational Internet of Things based on multimodal protocol adaptation, obtains the communication characteristics of the educational Internet of Things devices corresponding to the multimodal protocol, constructs a multimodal protocol fingerprint library according to the communication characteristics, and the multimodal protocol fingerprint library includes the protocol fingerprints corresponding to the educational Internet of Things devices; obtains the multimodal collaborative network corresponding to the current teaching scene, each teaching scene corresponds to a multimodal collaborative network, the multimodal collaborative network includes multimodal device nodes and connection edges between device nodes, each device node corresponds to the control parameters of an educational Internet of Things device, and the connection edges are determined according to the collaborative response probability between the corresponding educational Internet of Things devices; based on the multimodal protocol fingerprint library and the multimodal collaborative network, generates adaptive collaborative control instructions for the educational Internet of Things devices in the current teaching scene; and sends the adaptive collaborative control instructions to the educational Internet of Things devices in the current teaching scene for execution. The present invention obtains multimodal protocol features to build a protocol fingerprint library to solve the problem of heterogeneous compatibility of device protocols; by establishing a multimodal collaborative network driven by teaching scenarios, the linkage relationship between educational Internet of Things devices under multimodal protocols is quantified; based on the fingerprint library and the collaborative network, adaptive collaborative control instructions are generated to achieve intelligent scheduling of cross-modal devices; finally, closed-loop control is formed by issuing and executing instructions, which significantly improves the device collaboration efficiency and stability of the educational Internet of Things under multi-scenario switching. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of an intelligent collaborative method for the Internet of Things of Education based on multimodal protocol adaptation provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a specific process of step S1 in an embodiment of the present invention; Figure 3 A schematic diagram of a multimodal protocol fingerprint library update process provided by an embodiment of the present invention; Figure 4 A schematic diagram of a multimodal collaborative network construction process provided by an embodiment of the present invention; Figure 5 This is a schematic diagram of a specific process of step S5 in an embodiment of the present invention; Figure 6 This is a schematic diagram of a specific process of step S8 in an embodiment of the present invention; Figure 7 This is a schematic diagram of a specific process of step S3 in an embodiment of the present invention; Figure 8 A schematic diagram of the training process of a pre-trained graph convolutional network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0016] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0017] It should also be noted that when an element is referred to as being "fixed on" or "disposed on" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element.

[0018] In addition, the descriptions of "first", "second", etc. in the present invention are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0019] See Figure 1 , Figure 1 : This is a flow chart of an intelligent collaborative method for an educational Internet of Things based on multimodal protocol adaptation provided by an embodiment of the present invention. The intelligent collaborative method for an educational Internet of Things based on multimodal protocol adaptation includes the following steps: Step S1: Obtain the communication characteristics of the educational Internet of Things device corresponding to the multimodal protocol, and build a multimodal protocol fingerprint library based on the communication characteristics.

[0020] Among them, the multimodal protocol fingerprint library includes the protocol fingerprints corresponding to educational IoT devices.

[0021] Step S2: Obtain the multimodal collaborative network corresponding to the current teaching scenario.

[0022] Among them, each teaching scenario corresponds to a multimodal collaborative network, which includes multimodal device nodes and connection edges between device nodes. Each device node corresponds to the control parameters of an educational Internet of Things device, and the connection edges are determined according to the collaborative response probability between the corresponding educational Internet of Things devices.

[0023] Step S3, based on the multimodal protocol fingerprint library and the multimodal collaborative network, generates adaptive collaborative control instructions for the educational Internet of Things devices in the current teaching scenario.

[0024] Step S4: Send the adaptive collaborative control instruction to the educational Internet of Things device in the current teaching scene for execution.

[0025] In an embodiment of the present invention, the above-mentioned multimodal protocol refers to a heterogeneous communication protocol supported by educational IoT devices, which may specifically include IoT communication protocols such as Wi-Fi, Zigbee, Modbus, and Bluetooth.

[0026] These communication characteristics may include protocol type, data frame structure, transmission delay, response frequency, and message validation rules. Different protocols have different communication characteristics. The device communication message parsing engine can capture the real-time communication data stream of educational IoT devices and extract characteristic parameters such as protocol type, data frame length, and response delay. Alternatively, log data can be used to extract the real-time communication data stream of educational IoT devices and extract characteristic parameters such as protocol type, data frame length, and response delay.

[0027] The above-mentioned multimodal protocol fingerprint library is used for protocol features (i.e., protocol fingerprints) corresponding to protocol devices with different modalities. Furthermore, each educational IoT device corresponding to a multimodal protocol can correspond to a protocol fingerprint.

[0028] There can be multiple teaching scenarios, and different teaching scenarios can be set up according to the content of different teaching activities, such as digital classroom teaching, music classroom teaching, experimental operation, art exhibition, etc. The current teaching scenario corresponding to the current moment can be determined based on the teaching schedule of the current area (classroom, conference room, multimedia room, playground, etc.). Of course, control terminals can also be set up in each area. The control terminals are equipped with a scene selection interface. Users can select a teaching scenario as the current teaching scenario if they have permission.

[0029] The multimodal collaborative network uses a graph structure to represent the logical topology of device nodes in the teaching scenario. The device nodes are educational IoT devices, and the connecting edges represent the collaborative relationship between devices.

[0030] The collaborative response probability is the probability of the command response density between devices calculated based on historical interaction data. It is used to quantify the weight value of the connection edge and represent the comprehensive influence between various educational IoT devices in the collaborative situation.

[0031] In a possible embodiment, the device nodes connected to the educational Internet of Things network can be polled to collect their communication message samples; the message header and payload structure can be parsed to generate a protocol syntax tree; the transmission delay, packet loss rate and retransmission mechanism characteristics can be extracted, and then the message header, payload structure, protocol syntax tree, transmission delay, packet loss rate and retransmission mechanism can be encoded to obtain a feature vector; the device ID can be bound to the feature vector and stored in a multimodal protocol fingerprint library.

[0032] After determining the current teaching scenario, the current teaching scenario can be matched in the multimodal collaborative network database to match the multimodal collaborative network corresponding to the current teaching scenario. The multimodal collaborative network corresponding to the above current teaching scenario can also be called the target multimodal collaborative network.

[0033] In one possible embodiment, a basic collaborative network framework is loaded from a preset template library according to the scenario type; based on the historical interaction logs of the devices, the collaborative response probability between each device is calculated, the collaborative response probability is used as the connection edge weight, and a weighted multimodal collaborative network is constructed as the target multimodal collaborative network.

[0034] After obtaining the multimodal protocol fingerprint library and the multimodal collaborative network corresponding to the current teaching scenario, the multimodal collaborative network of the current scenario can be input to parse the device node control parameters (such as device status and priority); the communication characteristics of each educational IoT device can be matched according to the protocol fingerprint library to determine the instruction encapsulation format; with the goal of maximizing the collaborative success rate, the instruction issuance path is dynamically planned based on the weight of the multimodal collaborative network; and an adaptive collaborative control instruction set is generated that includes protocol conversion rules, transmission timing, and fault tolerance mechanism.

[0035] The instructions are converted into the protocol format supported by the target educational IoT device through the protocol adaptation gateway; the instructions are issued according to the timing priority of the collaborative network planning; the execution status of the educational IoT device is monitored in real time, and if the response timeout occurs, the backup path is triggered to resend.

[0036] In this embodiment, communication characteristics of educational IoT devices corresponding to multimodal protocols are obtained, and a multimodal protocol fingerprint library is constructed based on the communication characteristics. The multimodal protocol fingerprint library includes protocol fingerprints corresponding to educational IoT devices; a multimodal collaborative network corresponding to the current teaching scenario is obtained, each teaching scenario corresponds to a multimodal collaborative network, and the multimodal collaborative network includes multimodal device nodes and connection edges between device nodes. Each device node corresponds to a control parameter of an educational IoT device, and the connection edges are determined based on the collaborative response probability between the corresponding educational IoT devices; based on the multimodal protocol fingerprint library and the multimodal collaborative network, adaptive collaborative control instructions are generated for the educational IoT devices in the current teaching scenario; and the adaptive collaborative control instructions are issued to the educational IoT devices in the current teaching scenario for execution. The present invention solves the problem of heterogeneous compatibility of device protocols by obtaining multimodal protocol characteristics and constructing a protocol fingerprint library; by establishing a multimodal collaborative network driven by teaching scenarios, the linkage relationship between educational IoT devices under multimodal protocols is quantified; based on the fingerprint library and the collaborative network, adaptive collaborative control instructions are generated to achieve intelligent scheduling of cross-modal devices; and finally, closed-loop control is formed by issuing and executing instructions, significantly improving the device collaboration efficiency and stability of the educational IoT under multi-scenario switching.

[0037] It is understandable that in the specific implementation of this application, communication data, device data, interaction data and other related data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data, as well as the training and use of various models need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0038] Optionally, step S1 specifically includes: Step S101: Obtain communication data packets of educational Internet of Things devices corresponding to the multimodal protocol in different teaching periods.

[0039] Step S102: parsing the communication data to obtain the protocol type, data frame structure, transmission delay, response frequency and message verification rules as communication features.

[0040] Step S103: performing unsupervised clustering on the communication features to generate protocol feature clusters.

[0041] Step S104: extract the cluster center feature vector corresponding to each of the protocol feature clusters as a protocol fingerprint, and store it in a fingerprint library. Each modal protocol corresponds to one protocol fingerprint.

[0042] In an embodiment of the present invention, the teaching period refers to a time interval divided according to the periodic characteristics of the teaching activities, including but not limited to classroom teaching period, experimental operation period, multimedia display period, open class period, extracurricular activity period, etc. Different time periods correspond to educational Internet of Things device combinations with different modal protocols.

[0043] Communication packets are raw data units transmitted by educational IoT devices during communication. They may include protocol header information, payload content, and transmission control fields. A sniffing agent deployed on the educational IoT gateway can capture communication packet streams from all devices based on pre-set teaching time periods (e.g., a 15-minute sampling window). A timestamp mechanism is used to associate packets with their respective teaching time periods, ensuring time-period relevance for feature analysis.

[0044] The packet parsing engine deconstructs the protocol stack structure of communication packets layer by layer, extracting core features such as protocol type, data frame structure, transmission delay, response frequency, and message verification rules. The protocol type can be used to identify the communication protocol standard used by the packet (such as Wi-Fi PHY layer identifiers or Modbus function codes); the data frame structure includes quantized frame header length, payload format, and delimiter rules; transmission delay can be calculated by calculating the end-to-end average delay from command issuance to device response; response frequency can be calculated by counting the number of successful device responses to commands per unit time; and message verification rules can be analyzed by analyzing the CRC check bit distribution and error correction mechanism type.

[0045] Protocol feature clusters are formed by dividing communication features into highly cohesive sets through unsupervised clustering algorithms. Features within the same cluster are highly similar, representing the common behaviors of the same type of protocol modalities.

[0046] The cluster center feature vector is the mean feature point of all samples in the protocol feature cluster, which forms the typical protocol fingerprint of the cluster after vectorized encoding.

[0047] Specifically, the parsed communication feature set can be fed into an unsupervised clustering algorithm (such as K-means++), using Euclidean distance as the similarity metric to automatically aggregate samples with similar feature dimensions. The silhouette coefficient method is then used to determine the optimal number of clusters, generating K protocol feature clusters, each representing a specific protocol behavior pattern. For each protocol feature cluster, the arithmetic mean of its multidimensional features is calculated to generate a cluster center feature vector. This vector is then binary-encoded to form a fixed-length protocol fingerprint string. A mapping relationship between the protocol fingerprint and the device modality is established and stored in a multimodal protocol fingerprint library.

[0048] In this embodiment, data is captured by slicing the teaching period to solve the problem of device communication behavior fluctuating over time and improve the temporal generalization capability of the fingerprint library; the intrinsic correlation of protocol features is automatically discovered through unsupervised clustering to avoid the subjective bias of manually defined fingerprints.

[0049] Optionally, after step S1, the method further comprises the following steps: Step S105: monitor the communication packet loss rate and response delay time of the educational Internet of Things device in real time.

[0050] Step S106 : When the communication packet loss rate is greater than the packet loss rate threshold, or when the response delay time is greater than the delay time threshold, the communication features of the abnormal communication data packet are extracted.

[0051] Step S107 , performing Euclidean distance calculation on the communication feature and the protocol fingerprint in the multimodal protocol fingerprint library. If the calculated minimum Euclidean distance is greater than the distance threshold, the protocol fingerprint corresponding to the minimum Euclidean distance in the multimodal protocol fingerprint library is updated.

[0052] In this embodiment of the present invention, the packet loss rate is the ratio of the number of data packets lost by an educational IoT device to the total number of packets sent per unit time, which indicates the reliability of the link transmission. The response delay is the end-to-end time interval from the issuance of a control command to an educational IoT device to the receipt of a valid response from the device.

[0053] The built-in monitoring agent (sniffer agent) of the education IoT gateway can be used to continuously collect communication performance indicators of each device, calculate the packet loss rate at the transport layer (number of lost packets / total number of sent packets × 100%), and record the response delay time at the application layer (the difference between the time the command is sent and the time the ACK confirmation is received).

[0054] When any performance indicator exceeds the preset threshold, such as the packet loss rate threshold is set to 5% and the delay threshold is set to 200ms, the packet sniffer is triggered to capture the current communication data stream; the feature parsing engine is called to extract the communication feature five-tuple (protocol type, frame structure, delay, frequency, and verification rules) of the abnormal data packet.

[0055] Abnormal communication data packets are communication data packets captured when the packet loss rate or delay time exceeds the threshold. Their communication characteristics reflect the abnormal behavior pattern of the device.

[0056] Euclidean distance calculation involves calculating the geometric distance between the abnormal communication feature vector and the protocol fingerprint vector in a multidimensional feature space to quantify feature similarity. The abnormal communication feature vector can be input into a multimodal protocol fingerprint library. All protocol fingerprints in the library are traversed, and the Euclidean distance between the abnormal feature vector and each fingerprint is calculated. If the minimum Euclidean distance exceeds a distance threshold, the device's protocol behavior is determined to have significantly deviated. The protocol fingerprint corresponding to the minimum distance is located and an update is performed. This update can be a weighted fusion of the abnormal feature vector and the original fingerprint vector (with a new and old weight ratio of 7:3). The generated updated cluster center feature vector overwrites the original protocol fingerprint.

[0057] The distance threshold is a preset tolerance boundary for protocol fingerprint matching. If the value exceeds this threshold, it is determined to be a protocol behavior deviation, and the corresponding communication data packet is an abnormal communication data packet.

[0058] In this embodiment, fingerprint updates are triggered by real-time performance monitoring to solve the problem of protocol feature drift caused by device firmware upgrades or environmental interference; an automatic comparison mechanism between abnormal data packet features and the fingerprint library is used to accurately identify devices with degraded protocol compatibility; a threshold-driven closed-loop update strategy ensures that the fingerprint library remains adaptable to changes in network topology; the fingerprint update process is only initiated when an anomaly is detected, avoiding the overhead of periodic reconstruction of the full fingerprint library.

[0059] Optionally, before step S2, the method further includes: Step S5: for each teaching scenario, identify the activity corresponding to each educational Internet of Things device in the teaching scenario.

[0060] Step S6: Select candidate educational Internet of Things devices whose activity is greater than the activity threshold from among the various educational Internet of Things devices in the teaching scenario, and map the candidate educational Internet of Things devices to device nodes.

[0061] Among them, each device node corresponds to a candidate educational IoT device mapping.

[0062] Step S7: For each candidate educational IoT device, obtain the historical linkage log corresponding to the candidate educational IoT device.

[0063] Among them, the historical linkage target log includes historical communication data when the candidate educational IoT device collaborates with candidate educational IoT devices of other modalities in the teaching scenario.

[0064] Step S8: Determine the probability of collaborative response between candidate educational IoT devices based on historical communication data.

[0065] Step S9: Connect the device nodes corresponding to each candidate educational Internet of Things device through the collaborative response probability between each candidate educational Internet of Things device to obtain a multimodal collaborative network corresponding to each teaching scenario.

[0066] In this embodiment of the present invention, device activity represents the frequency with which educational IoT devices participate in tasks within a teaching scenario. This activity is quantified by the number of triggered command executions per unit time. Device command execution records can be extracted from a teaching scenario event database. The number of valid operations within a preset period (e.g., a single lesson) can be aggregated and counted by the scenario-device dimension.

[0067] The activity threshold is the preset minimum activity standard for educational IoT device nodes to be selected into the multimodal collaborative network (e.g., ≥5 times / class hour), which is used to filter low-frequency devices.

[0068] The candidate educational IoT devices are a subset of devices that meet the activity requirements and serve as the basic building blocks (device nodes) of the multimodal collaborative network.

[0069] Historical linkage logs are a time-series dataset of collaborative events between devices in teaching scenarios, including the timing of command interactions, response status, and communication metadata. You can call the IoT middleware API to retrieve the full collaborative communication logs for candidate devices in the target teaching scenario. Log fields include: initiating device ID, responding device ID, command type, timestamp, and response status code.

[0070] The collaborative response probability is the probability of successful collaboration density between devices based on historical linkage log statistics. It can be calculated by P(A|B) = \frac{N_{A successfully responds to B}}{N_{B sends commands to A}}. When devices A and B have a collaborative relationship (B→A sends a command, A responds), the probability of successful response density of educational IoT device A to the instructions of educational IoT device B based on historical linkage log statistics is calculated. Educational IoT device A and educational IoT device B are two educational IoT devices with different modal protocols.

[0071] In one possible embodiment, statistics are grouped by device pair (Device_A, Device_B): the total number of times Device_B sends commands to Device_A is counted N total（B→A） ; Count the number of successful responses from Device_A, N success（B↔A） ; Calculate the probability of coordinated response: P(A|B) = N success（B↔A） / N total（B→A）。

[0072] Specifically, educational IoT devices with activity levels exceeding an activity threshold are marked as candidate educational IoT devices. A one-to-one mapping relationship between candidate educational IoT devices and device nodes is established, and a set of isolated nodes is initialized. After calculating the collaborative response probability, the collaborative response probability between each device node is obtained. All candidate device pairs are traversed. If the collaborative response probability is greater than 0 (indicating a historical collaborative relationship), connecting edges are added to the graph structure, with the probability value P as the edge weight. This generates a weighted directed graph as the multimodal collaborative network for this teaching scenario.

[0073] In a possible embodiment, two device nodes corresponding to P(A|B)=0 are not connected, or two device nodes whose P(A|B) is less than a preset cooperative response probability threshold are not connected.

[0074] In this embodiment, low-frequency devices are filtered out through activity thresholds to reduce the complexity of the collaborative network; response probabilities are calculated based on real historical interaction data to objectively quantify the impact intensity between devices; and networks are independently constructed for different teaching scenarios to avoid redundant device associations.

[0075] Optionally, step S5 specifically includes: Step S501: determine the working time of each educational Internet of Things device in the teaching scenario, and the number of times each educational Internet of Things device is triggered by the teacher-student interaction.

[0076] Step S502: Based on the working hours and the number of times triggered by teacher-student interactions, the activity level of each educational IoT device in the teaching scenario is determined.

[0077] In an embodiment of the present invention, operating time can be the effective operating time of an educational IoT device in a single teaching scenario, measured from the time the device status switches to "working" to the time the status changes to "standby." The effective operating time of an educational IoT device in a single teaching scenario can be determined by acquiring device operating status change events in real time through a device status monitoring service, or by having the educational IoT device report operating status change events. The total operating time corresponding to the educational IoT device is calculated by aggregation based on the teaching scenario slice.

[0078] The number of teacher-student interaction triggers can be the number of times a teacher or student directly activates a device function through a human-machine interface, such as through a touch screen, voice assistant, or physical button. Scenario-related operation records are extracted from the human-machine interaction log; the number of operations with the event type "interaction" can be counted by device ID.

[0079] Specifically, monitor the device status event stream and record the state transition timestamp; the aggregation calculation by scenario is as follows: ; in, For devices i Total working hours in teaching scenarios, and For the k The start and end time of the working cycle, n is the total number of working cycles.

[0080] Parse the human-computer interaction log and filter the scene-related events; count the number of operations as follows: ; Among them, 1 {} is the indicator function, For interactive events, m is the total number of events.

[0081] The dimension difference is eliminated by data normalization, which is as follows: ; in: ; A collection of educational IoT devices for current teaching scenarios.

[0082] Finally, the activity can be calculated for: ; in, α ∈[0,1] is the weight coefficient, and the default value is α=0.4.

[0083] In this embodiment, the continuity indicator (working hours) and the initiative indicator (number of interactions) are integrated to avoid single-dimensional deviation; normalization processing solves the numerical scale problem caused by differences in device types.

[0084] Optionally, the steps of step S8 specifically include: Step S801: determining the historical success rate between candidate educational IoT devices based on historical communication data; Step S802: determining a protocol compatibility factor based on the protocol fingerprint corresponding to each candidate educational IoT device; Step S803: Determine the collaborative response probability between the candidate educational IoT devices based on the historical success rate, protocol compatibility factor, and bandwidth factor.

[0085] In this embodiment of the present invention, the historical success rate can be understood as the historical probability of successful responses from educational IoT device A to instructions from educational IoT device B, where educational IoT device A and educational IoT device B are two educational IoT devices using different modal protocols. The historical success probability of responses can be calculated using the following formula: ; in, Indicates the total number of times that educational IoT device B sends commands to educational IoT device A. Indicates the number of times that Education IoT device B responds to Education IoT device A after sending a command to B.

[0086] The protocol compatibility factor can be understood as an indicator of communication compatibility between educational IoT devices based on the similarity of protocol fingerprints. Specifically, the protocol compatibility factor can be calculated using the following formula: ; in, Indicates the protocol compatibility factor between educational IoT device A and educational IoT device B. represents the cipher mode vector, The cryptographic pattern vector representing the protocol fingerprint of educational IoT device A, The cryptographic pattern vector representing the protocol fingerprint of educational IoT device B, Indicates the first k The encryption pattern feature vector of the protocol fingerprint.

[0087] The above encryption mode may be AES-128-CTR, ChaCha20-Poly1305, SM4-GCM, etc.; Represents the set of all protocol fingerprints in the multimodal protocol fingerprint library.

[0088] The bandwidth factor is the percentage of available bandwidth on the current network link, reflecting real-time transmission conditions. The bandwidth factor is as follows: ; in, Indicates available bandwidth, indicates total bandwidth .

[0089] The coordinated response probability is a comprehensive prediction of the historical success rate, protocol compatibility factor, and bandwidth factor, as shown below: ; Where ω, μ, and η are the fusion weights corresponding to the fusion historical success rate, protocol compatibility factor, and bandwidth factor, respectively. ω+μ+η=1, and ω, μ, and η are all positive numbers.

[0090] In this embodiment, the limitations of single historical data are overcome by a three-factor model. When the protocol changes, the predicted probability can be automatically reduced. When the network is congested, the transmission risk can be reflected immediately, making the adaptive ability of the coordinated response probability stronger.

[0091] Optionally, step S3 specifically includes: Step S301: Encode the multimodal protocol fingerprint library into a node feature matrix, and use the multimodal collaborative network as the adjacency matrix of the node feature matrix.

[0092] In the node feature matrix, each node represents a protocol fingerprint, and the connecting edges between nodes represent the similarity between the protocol fingerprints.

[0093] In step S302, the node feature matrix and the adjacency matrix are input into a pre-trained graph convolutional network, and control parameters are output with the goal of minimizing the collaborative delay.

[0094] Step S303: convert the control parameters into a device executable instruction set to obtain the adaptive collaborative control instructions corresponding to the educational Internet of Things devices in the current teaching scene.

[0095] In an embodiment of the present invention, the multimodal protocol fingerprint library can be encoded as a matrix X∈R N×D , N is the total number of protocol fingerprints (i.e. the total number of nodes in the node feature matrix), D is the feature dimension of the protocol fingerprint, and each row is X i Corresponding to a coded feature vector of a protocol fingerprint. In a possible embodiment, X i Perform password mode encoding to obtain v enc,i .

[0096] The adjacency matrix corresponding to the above multimodal collaborative network is A∈{0,P(A|B)} N×N , A A,B =0, it means that there is no connection edge between educational IoT devices A and B. A,B = P(A|B) indicates that there is a connection edge between educational IoT devices A and B.

[0097] After obtaining the node feature matrix and adjacency matrix, the node feature matrix and adjacency matrix can be input into the pre-trained graph convolutional network for processing to obtain the prediction results. The prediction results include the instruction sending timing and protocol conversion path of each device node.

[0098] Specifically, the pre-trained graph convolution network includes a parallel first input layer and a second input layer, a feature fusion layer, and an output layer. The input of the first input layer is a node feature matrix, and the input of the second input layer is an adjacency matrix. The first input layer performs graph convolution processing on the node feature matrix to obtain a first feature map. The second input layer performs graph convolution processing on the adjacency matrix to obtain a second feature map. The feature fusion layer performs channel fusion or splicing fusion on the first feature map and the second feature map to obtain fusion features. The fusion features are mapped through the output layer to a control parameter matrix. The control parameter matrix includes the control parameters corresponding to each educational IoT device. The control parameter matrix is parsed according to the device, and the parameter vector is converted into a device executable instruction through the encoder. It should be noted that the structure of the above-mentioned control parameter matrix is the same as that of the adjacency matrix. The difference is that the information corresponding to the device node in the control parameter matrix is control parameter information.

[0099] In this embodiment, the node feature matrix encodes protocol heterogeneity, encodes device collaboration relationships through the adjacency matrix, and uses the graph structure to perceive device topology relationships, optimize the instruction issuance timing and protocol conversion path, and the dual-source input overcomes the problem of protocol and topology separation in traditional methods, thereby better realizing multimodal protocol adaptive control in teaching scenarios.

[0100] Optionally, before step S302, the method further includes: Step S304: construct a training data set and a graph convolutional network to be trained.

[0101] Among them, the training data set includes: the sample node feature matrix obtained by encoding the multimodal protocol fingerprint library, the historical multimodal collaborative network corresponding to different teaching scenarios, and the actual control parameters and real response delay corresponding to the historical multimodal collaborative network. In the sample node feature matrix, each sample node represents a protocol fingerprint, and the connecting edges between each sample node represent the similarity between each protocol fingerprint.

[0102] The graph convolutional network to be trained includes a parallel first input layer and a second input layer, a feature fusion layer, and an output layer.

[0103] In step S305, the sample node feature matrix and the historical multimodal collaborative network are input into the graph convolutional network to be trained to predict the response delay, thereby obtaining a predicted control parameter matrix and a predicted response delay.

[0104] Step S306, calculate the first error loss between the predicted response delay and the actual response delay, and the second error loss between the actual control parameter and the predicted control parameter matrix through the loss function, add the first error loss and the second error loss to obtain the total error loss, and take minimizing the total error loss as the optimization goal, adjust the network parameters of the graph convolutional network to be trained through the back propagation algorithm, and iterate the adjustment process of the network parameters until the training stop condition is met and the training is stopped to obtain the pre-trained graph convolutional network.

[0105] In the embodiment of the present invention, the sample node feature matrix is constructed in the same manner as the node feature matrix, so it is not described here in detail. For the historical multimodal collaborative network, it can also be converted into a sample adjacency matrix. The output of the graph convolutional network is the prediction parameter control matrix and the predicted response delay.

[0106] The total response time (i.e., actual response delay) of a device cluster to complete a collaborative task is calculated using the following formula: ; in, Indicates the actual response delay to complete the collaborative task, k Indicates the first participant in the multimodal collaborative network k Educational IoT devices, Indicates the global start time of the collaborative task, which can be the timestamp of the first control instruction leaving the protocol adaptation gateway. Indicates educational IoT devices k The time when this event started (actual response time), Indicates the slowest responding device.

[0107] The sample node feature matrix is input into the first input layer of the graph convolution network to be trained for graph convolution processing to obtain a first sample feature map. The sample adjacency matrix is input into the second input layer of the graph convolution network to be trained for processing to obtain a second sample feature map. The first sample feature map and the second sample feature map are channel-fused or spliced together through the feature fusion layer of the graph convolution network to be trained to obtain fused features. The fused features are mapped through the output layer to a prediction control parameter matrix and output. At the same time, the predicted response delay corresponding to the prediction control parameter matrix is also output. The control parameter matrix includes the control parameters corresponding to each educational Internet of Things device.

[0108] A weighted mean square error function may be used to calculate a first error loss between the predicted response delay and the actual response delay, and to calculate a second error loss between the actual control parameter and the predicted control parameter matrix.

[0109] Specifically, the above loss function is as follows: ; in, Loss 1 represents the first error loss, M is the sample size, Indicates the i The delayed response weight of samples, Indicates the i The predicted response delay of samples, Indicates the i The actual response delay of samples, Loss 2 represents the first error loss, N is the number of device nodes, Indicates the i The control parameter weights of samples, Indicates the i The first ( k , j ) control parameter predicted values, Indicates the i The actual control parameter matrix of samples ( k , j ) actual values of the control parameters.

[0110] The first error loss and the second error loss are directly added or weighted together to obtain the total error loss. Minimizing the total error loss is then optimized. The network parameters of the graph convolutional network to be trained are adjusted using a backpropagation algorithm. The network parameter adjustment process is iterated until the number of iterations reaches a preset number or the total error loss is less than the preset error loss value. Training is terminated to obtain the pretrained graph convolutional network. The output of the pretrained graph convolutional network is a predictive control parameter matrix. The predicted response delay may or may not be output. If the predicted response delay is output, the predicted response delay can be used as a reference to determine whether the predictive control parameter matrix meets user expectations.

[0111] By constructing a training dataset to train the graph convolutional network to be trained, a pre-trained graph convolutional network for predicting the control parameter matrix can be trained to improve the accuracy of the nonlinear solution of the control parameter matrix.

[0112] Optionally, step S306 specifically includes: calculating the weighted mean square error between the predicted response delay and the actual response delay; determining the multimodal collaborative loss factor based on the historical multimodal collaborative network; determining the feature constraint factor based on the sample node feature matrix; and calculating the error loss between the predicted response delay and the actual response delay based on the weighted mean square error, the multimodal collaborative loss factor, and the feature constraint factor.

[0113] In an embodiment of the present invention, since the actual response delay is highly correlated with the actual control parameter matrix, in order to avoid overfitting, this embodiment improves the loss function of the first error loss, specifically considering the fusion topological relationship penalty factor and prior feature constraints, thereby avoiding overfitting.

[0114] Specifically, the first error loss can be calculated using the following loss function: ; in, Indicates the i Multimodal collaborative network of samples, Indicates the i The set of all connected edges in the multimodal collaborative network of samples, Represents a device node j The predicted response delay, Represents a device node k The predicted response delay, Represents a device node k The degree (with the device node k The number of connected edges), Represents a device node j The degree (with the device node j The number of connected edges), Represents a device node k and device nodes j The historical average collaborative delay difference between Indicates the i The node feature matrix of samples, The Gram matrix representing the node feature matrix, XX T , Indicates the i The benchmark feature matrix of the teaching scenario, Represents the smoothing coefficient, which is used to balance the weights between the two items. represents the trace of the matrix, that is, the sum of the elements on the main diagonal of the matrix, represents the transpose of the node feature matrix, The Laplacian matrix representing the node feature matrix.

[0115] In this embodiment, the topological relationship penalty factor and the prior feature constraint are considered to be integrated to avoid overfitting.

[0116] The present invention also proposes an intelligent collaborative system for an educational Internet of Things based on multimodal protocol adaptation. In embodiments of the present invention, the intelligent collaborative system for an educational Internet of Things based on multimodal protocol adaptation can be a computing device such as a desktop computer, laptop, PDA, or server. The intelligent collaborative system for an educational Internet of Things based on multimodal protocol adaptation can include a processor (e.g., a CPU), a network interface, a user interface, a memory, and a communication bus. The communication bus is used to enable communication between these components. The user interface can include a display and an input unit, such as a keyboard. Optionally, the user interface can also include a standard wired interface or a wireless interface. The network interface can optionally include a standard wired interface or a wireless interface (e.g., a Wi-Fi interface). The memory can be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory can be a storage device independent of the processor.

[0117] Those skilled in the art will understand that the intelligent collaborative system structure of the Internet of Things for Education based on multimodal protocol adaptation proposed in the embodiment of the present invention does not constitute a limitation on the intelligent collaborative system of the Internet of Things for Education based on multimodal protocol adaptation, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0118] The memory as a computer storage medium may include an operating system, a network communication module, a user interface module, and a computer program.

[0119] In the intelligent collaborative system of the educational Internet of Things based on multimodal protocol adaptation, the network interface is mainly used to connect to the background server and communicate data with the background server; the user interface is mainly used to connect to the client (user end) and communicate data with the client; and the processor can be used to call the computer program stored in the memory. When the computer program is called and executed by the processor, the steps of the above-mentioned intelligent collaborative method of the educational Internet of Things based on multimodal protocol adaptation are implemented.

[0120] Based on the educational Internet of Things intelligent collaboration system based on multimodal protocol adaptation proposed in the aforementioned embodiment, the present invention also proposes a storage medium, which stores a computer program. When the computer program is executed by a controller, it implements the educational Internet of Things intelligent collaboration method based on multimodal protocol adaptation recorded in the aforementioned embodiment.

[0121] The educational Internet of Things intelligent collaboration system and storage medium based on multimodal protocol adaptation of the present invention can implement the steps of the above-mentioned educational Internet of Things intelligent collaboration method based on multimodal protocol adaptation, and therefore have at least all the beneficial effects brought about by the technical solutions of the above-mentioned educational Internet of Things intelligent collaboration method based on multimodal protocol adaptation embodiment, and will not be described one by one here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0123] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0124] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0125] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0126] The above description is only a partial or preferred embodiment of the present invention. Neither the text nor the drawings can limit the scope of protection of the present invention. Any equivalent structural transformation made by using the contents of the present invention specification and drawings under the overall concept of the present invention, or direct / indirect application in other related technical fields, is included in the scope of protection of the present invention.

Claims

1. An intelligent collaborative method for educational Internet of Things based on multimodal protocol adaptation, characterized in that: The method comprises: Acquire communication characteristics of an educational Internet of Things device corresponding to a multimodal protocol, and construct a multimodal protocol fingerprint library based on the communication characteristics, wherein the multimodal protocol fingerprint library includes protocol fingerprints corresponding to the educational Internet of Things device; Obtain a multimodal collaborative network corresponding to the current teaching scenario. Each teaching scenario corresponds to a multimodal collaborative network. The multimodal collaborative network includes multimodal device nodes and connection edges between device nodes. Each device node corresponds to a control parameter of an educational Internet of Things device. The connection edges are determined based on the collaborative response probability between the corresponding educational Internet of Things devices. Based on the multimodal protocol fingerprint library and the multimodal collaborative network, generating adaptive collaborative control instructions for the educational Internet of Things devices in the current teaching scenario; The adaptive collaborative control instructions are sent to the educational Internet of Things devices in the current teaching scene for execution.

2. The method for intelligent collaboration of the Internet of Things for education based on multimodal protocol adaptation according to claim 1 is characterized in that: The acquiring of communication characteristics of the educational Internet of Things device corresponding to the multimodal protocol and constructing a multimodal protocol fingerprint library according to the communication characteristics include: Obtain communication data packets of educational IoT devices corresponding to multimodal protocols during different teaching periods; Parsing the communication data packet to obtain the protocol type, data frame structure, transmission delay, response frequency and message verification rules as communication features; Perform unsupervised clustering on communication features to generate protocol feature clusters; The cluster center feature vector corresponding to each of the protocol feature clusters is extracted as the protocol fingerprint and stored in the fingerprint library. Each modal protocol corresponds to one protocol fingerprint.

3. The method for intelligent collaboration of the Internet of Things for education based on multimodal protocol adaptation according to claim 2 is characterized in that: After obtaining the communication characteristics of the educational Internet of Things device corresponding to the multimodal protocol and constructing a multimodal protocol fingerprint library based on the communication characteristics, the method further includes: Real-time monitoring of the communication packet loss rate and response delay time of the educational IoT device; When the communication packet loss rate is greater than a packet loss rate threshold, or the response delay time is greater than a delay time threshold, extracting communication features of the abnormal communication data packet; The communication feature is subjected to Euclidean distance calculation with the protocol fingerprint in the multimodal protocol fingerprint library. If the calculated minimum Euclidean distance is greater than a distance threshold, the protocol fingerprint corresponding to the minimum Euclidean distance in the multimodal protocol fingerprint library is updated.

4. The method for intelligent collaboration of the Internet of Things for education based on multimodal protocol adaptation according to claim 3 is characterized in that: Before the step of obtaining the multimodal collaborative network corresponding to the current teaching scenario, the method further includes: For each teaching scenario, identifying the activity corresponding to each of the educational IoT devices in the teaching scenario; Selecting candidate educational Internet of Things devices whose activity is greater than an activity threshold from each of the educational Internet of Things devices in the teaching scenario, and mapping the candidate educational Internet of Things devices into device nodes, where each device node corresponds to one candidate educational Internet of Things device mapping; For each candidate educational IoT device, obtain a historical linkage log corresponding to the candidate educational IoT device, wherein the historical linkage target log includes historical communication data when the candidate educational IoT device collaborates with candidate educational IoT devices of other modalities in the teaching scenario; Determining, based on the historical communication data, a probability of coordinated response between each of the candidate educational IoT devices; Through the collaborative response probability between each of the candidate educational Internet of Things devices, the device nodes corresponding to each of the candidate educational Internet of Things devices are connected to obtain a multimodal collaborative network corresponding to each of the teaching scenarios.

5. The method for intelligent collaboration of the Internet of Things for education based on multimodal protocol adaptation according to claim 4 is characterized in that: The identifying the activity corresponding to each of the educational IoT devices in the teaching scenario includes: Determine the working time of each of the educational Internet of Things devices in the teaching scenario, and the number of times each of the educational Internet of Things devices is triggered by teacher-student interactions; Based on the working hours and the number of times triggered by teacher-student interactions, the activity corresponding to each of the educational Internet of Things devices in the teaching scenario is determined.

6. The method for intelligent collaboration of educational Internet of Things based on multimodal protocol adaptation according to claim 4 is characterized in that: The determining, based on the historical communication data, the probability of coordinated response between the candidate educational IoT devices includes: Determining a historical success rate between each of the candidate educational IoT devices based on the historical communication data; Determining a protocol compatibility factor based on the protocol fingerprint corresponding to each of the candidate educational IoT devices; The collaborative response probability between each of the candidate educational Internet of Things devices is determined based on the historical success rate, the protocol compatibility factor, and the bandwidth factor.

7. The method for intelligent collaboration of the Internet of Things for education based on multimodal protocol adaptation according to any one of claims 1 to 6, characterized in that: The step of generating adaptive collaborative control instructions for the educational IoT devices in the current teaching scenario based on the multimodal protocol fingerprint library and the multimodal collaborative network includes: Encoding the multimodal protocol fingerprint library into a node feature matrix, and using the multimodal collaborative network as an adjacency matrix of the node feature matrix, wherein each node in the node feature matrix represents a protocol fingerprint, and the connecting edges between the nodes represent the similarity between the protocol fingerprints; Inputting the node feature matrix and the adjacency matrix into a pre-trained graph convolutional network, and outputting control parameters with the goal of minimizing collaborative delay; The control parameters are converted into a device executable instruction set to obtain the adaptive collaborative control instructions corresponding to the educational Internet of Things devices in the current teaching scenario.

8. The method for intelligent collaboration of the Internet of Things for education based on multimodal protocol adaptation according to claim 7 is characterized in that: Before the step of inputting the node feature matrix and the adjacency matrix into a pre-trained graph convolutional network and outputting control parameters with the goal of minimizing collaborative delay, the method further includes: Constructing a training data set and a graph convolutional network to be trained, wherein the training data set includes: a sample node feature matrix obtained by encoding the multimodal protocol fingerprint library, historical multimodal collaborative networks corresponding to different teaching scenarios, and an actual control parameter matrix and a real response delay corresponding to the historical multimodal collaborative network, wherein each sample node in the sample node feature matrix represents a protocol fingerprint, and the connecting edges between the sample nodes represent the similarity between the protocol fingerprints; Inputting the sample node feature matrix and the historical multimodal collaborative network into the to-be-trained graph convolutional network to perform response delay prediction, thereby obtaining a prediction control parameter matrix and a predicted response delay; The first error loss between the predicted response delay and the actual response delay, as well as the second error loss between the actual control parameter and the predicted control parameter matrix are calculated through the loss function. The first error loss and the second error loss are added to obtain the total error loss. With minimizing the total error loss as the optimization goal, the network parameters of the graph convolutional network to be trained are adjusted through the back propagation algorithm, and the network parameter adjustment process is iterated until the training stop condition is met and the training is stopped to obtain a pre-trained graph convolutional network.

9. The method for intelligent collaboration of the Internet of Things for education based on multimodal protocol adaptation according to claim 8 is characterized in that: The calculating a first error loss between the predicted response delay and the actual response delay by using a loss function includes: Calculating a weighted mean square error between the predicted response delay and the actual response delay; determining a multimodal collaboration loss factor based on the historical multimodal collaboration network; Determining a feature constraint factor based on the sample node feature matrix; A first error loss between the predicted response delay and the actual response delay is calculated based on the weighted mean square error, the multimodal synergy loss factor, and the feature constraint factor.

10. An intelligent collaborative system for educational Internet of Things based on multimodal protocol adaptation, characterized in that: The educational Internet of Things intelligent collaboration system based on multimodal protocol adaptation includes a memory, a processor, and a computer program stored on the memory and runnable on the processor. When the computer program is executed by the processor, the steps of the educational Internet of Things intelligent collaboration method based on multimodal protocol adaptation as described in any one of claims 1 to 9 are implemented.

Images