A multimedia device centralized control method based on internet of things

Abstract

The present application relates to the technical field of Internet of Things and multimedia communication, and particularly relates to a multimedia device centralized control method based on Internet of Things, which comprises the following steps: realizing heterogeneous device access by dynamically analyzing protocols through an Internet of Things gateway; collecting multi-source data by using an edge computing node, and detecting anomalies by Kalman filtering fusion and an isolated forest model; generating a dynamic control strategy based on an LSTM neural network and a scene rule base, realizing audio and video stream synchronous switching and resolution adaptive adjustment; guaranteeing control delay by hierarchical caching and a time-sensitive network protocol stack; and supporting multi-terminal control and implementing hierarchical authority management through a role authority matrix. The present application solves the problems of heterogeneous device compatibility difficulty, control real-time deficiency and extensive security management, and effectively improves the collaborative efficiency, response speed and use safety of multimedia devices.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things and multimedia communication, and particularly relates to a multimedia device centralized control method based on Internet of Things. BACKGROUND

[0002] Early multimedia device control mainly relies on manual operation or simple remote controllers, and each device is usually equipped with an independent remote controller or control panel, and the user needs to operate the television, sound, projector and other devices respectively. This scattered control method can meet the basic needs when the number of devices is small, but with the increase of device types and quantity, the operation complexity increases exponentially: the user needs to remember the operation logic of different devices, be familiar with the use method of multiple remote controllers, and the collaborative operation between devices is difficult to achieve, which greatly affects the convenience and experience consistency.

[0003] With the gradual development of computer technology and network communication technology, computer or network-based centralized control methods have emerged. Such solutions send control instructions to multimedia devices through computer serial ports, parallel ports or network interfaces by writing specific control software, thereby achieving unified management of multiple devices. However, these methods have significant application barriers: on the one hand, the system needs to be built and configured by the user with certain professional technical knowledge, involving complex operations such as device protocol parameter configuration and network interface debugging, and has high requirements for the stability of the device network environment and the performance of the computer, making it difficult to be widely popularized among ordinary users; on the other hand, the functions of such control methods are relatively basic, and they can only implement simple operations such as device switching and volume adjustment, and they lack support for complex multimedia scene control and deep linkage between devices, which cannot meet the needs of intelligent scenarios.

[0004] At present, although some commercial multimedia device centralized control systems have appeared in the market, these systems still have many technical bottlenecks: in terms of device compatibility, most systems can only support specific brands or specific types of devices, and have insufficient support for heterogeneous devices using different communication protocols, making it difficult to achieve truly universal centralized control; in terms of intelligence level, existing systems rely on preset rules for control and cannot automatically generate personalized control strategies according to the actual needs of users, real-time environmental parameters and use scenarios, nor can they make forward-looking control adjustments by learning the long-term behavior patterns of users; in terms of performance and security, instruction transmission relies on general network protocols, real-time performance is difficult to guarantee, and when multiple terminals interact, control delays or instruction losses may occur, and the permission management mechanism is relatively rough, making it difficult to achieve fine-grained authorization of device control, parameter modification, system configuration and other operations, and it is difficult to meet the needs of users for efficient, convenient and secure centralized control.

[0005] Therefore, the application discloses a multimedia device centralized control method based on Internet of Things. SUMMARY

[0006] The application aims to solve the problems of complex access, low data processing efficiency, static control strategy, high transmission delay and insufficient security of multi-terminal interaction of heterogeneous multimedia devices in the prior art.

[0007] In order to achieve the above-mentioned purpose, the application adopts the following technical scheme: a multimedia device centralized control method based on Internet of Things, comprising the following steps:

[0008] Step S1, device networking and protocol adaptation, accessing heterogeneous multimedia devices through an Internet of Things gateway, the gateway supporting dynamic analysis and compatible conversion of Modbus, TCP / IP and HTTP protocols;

[0009] Step S2, multi-source data acquisition and processing, real-time acquisition of multi-source data of multimedia devices, data fusion and anomaly detection through edge computing nodes;

[0010] Step S3, dynamic control strategy generation, generating a basic control strategy based on user instructions, optimizing the basic control strategy in combination with scene rules, adjusting the optimized strategy according to a user behavior prediction model, and generating a collaborative strategy including audio and video stream synchronous switching, resolution adaptive adjustment and device parameter linkage control;

[0011] Step S4, low-delay instruction issuing, guaranteeing end-to-end control delay ≤15ms through a hierarchical caching mechanism and a time-sensitive network protocol stack of the edge computing node;

[0012] Step S5, multi-terminal collaborative interaction, supporting remote control of mobile terminals, voice assistants and Web interfaces, and implementing hierarchical permission management through a role permission matrix.

[0013] The technical scheme provided by the application has at least the following beneficial effects:

[0014] The application supports dynamic analysis and compatible conversion of Modbus, TCP / IP and HTTP protocols through an Internet of Things gateway, in combination with a protocol dynamic loading mechanism, effectively solving the problems of difficult access and high adaptation cost of heterogeneous devices in traditional schemes, significantly improving the compatibility and expansion flexibility of the system for different manufacturers of multimedia devices, and realizing plug-and-play of the devices.

[0015] The application breaks through the limitation of traditional data independent processing, accurately fuses multi-dimensional data such as equipment state and environmental parameters, improves the accuracy of data processing and the reliability of fault detection, and realizes intelligent perception and abnormal early warning of equipment operation state through multi-source data fusion and abnormal detection mechanism of edge computing node, Kalman filtering algorithm and isolation forest model.

[0016] The application dynamically generates a cooperative control strategy containing audio and video stream synchronous switching and resolution adaptive adjustment based on the user behavior prediction model and scene rule library of the LSTM neural network, changes the traditional static control mode, enables the system to automatically optimize device operation according to user behavior habits and real-time scenes, and improves control response speed and user interaction experience.

[0017] The application realizes fine classification of control authority and strengthened security authentication of sensitive operation through multi-terminal remote control of supporting mobile terminals, voice assistants and Web interfaces, and combines role permission matrix and dynamic token authentication mechanism, compared with traditional extensive authority management, significantly improves system interaction efficiency and data security protection capability.

[0018] The application organically integrates heterogeneous protocol adaptation, intelligent data processing, dynamic control strategy, low-delay transmission and multi-terminal safe interaction, breaks through the limitation of fragmented control of existing systems, and provides an efficient, intelligent and safe integrated solution for centralized control of Internet of Things multimedia devices. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description, and obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0020] Figure 1 A method flowchart of a multimedia device centralized control method based on Internet of Things is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0021] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the specific implementation, structure, features and effects of a multimedia device centralized control method based on Internet of Things according to the present application are described in detail as follows by combining with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0023] The following examples are for illustrative purposes only and are not intended to limit the scope of the present application.

[0024] The specific scheme of the method for centralized control of multimedia equipment based on Internet of Things provided by the present application is specifically described below in combination with the drawings.

[0025] Referring to Figure 1 , a method flowchart of a method for centralized control of multimedia equipment based on Internet of Things provided by an embodiment of the present application is shown, and the method comprises the following steps:

[0026] Step S1, device networking and protocol adaptation, access heterogeneous multimedia equipment through an Internet of Things gateway, and the gateway supports dynamic parsing and compatible conversion of Modbus, TCP / IP and HTTP protocols;

[0027] In step S1, the following sub-steps are further included:

[0028] S1-1, the Internet of Things gateway extracts and matches the features of Modbus, TCP / IP and HTTP protocol messages based on a pre-constructed protocol feature library; the protocol feature library is used to store feature vector templates of protocol messages, the feature vector includes a combination of a protocol identification field, a function code field and a check field, and protocol matching is completed through a pre-set similarity threshold;

[0029] S1-2, through a protocol conversion engine, heterogeneous protocol data is mapped to a JSON format of a platform internal unified data model based on conversion rules defined by XML mode; the XML mode definition includes the mapping relationship between source protocol fields and target JSON format fields;

[0030] S1-3, a protocol dynamic loading mechanism is established to support loading of protocol analysis plug-ins through a secure sandbox; the plug-ins need to implement a pre-defined data analysis interface, and the legality of the plug-ins is verified through digital signature before loading.

[0031] It should be noted that the function code of the Modbus protocol is its key identification field, for example, function code 0x01 represents reading a coil, 0x02 represents reading a discrete input, 0x03 represents reading a holding register, 0x04 represents reading an input register, 0x05 represents writing a single coil, 0x06 represents writing a single register, 0x0F represents writing multiple coils, and 0x10 represents writing multiple registers. These function codes can indicate that the Modbus device performs corresponding operations.

[0032] Port number is an important identification field of TCP / IP protocol. For example, HTTP protocol uses port 80 and 443 by default. Through port number, TCP / IP protocol can accurately send network packets to the corresponding application program, and realize the differentiation of different services.

[0033] Method field of HTTP protocol is used to identify the request method. For example, GET means to get resources, and POST means to submit data to the server. Different Method fields indicate that the client performs different operations on the resources on the server.

[0034] The role of protocol feature library is to establish the mapping relationship between protocol packets and parsing rules, so that the Internet of Things gateway can judge the communication protocol used by the device by parsing the feature field in the packet, and call the corresponding protocol parsing plug-in according to the identification result, to realize the standardized processing of heterogeneous data.

[0035] Cosine similarity or edit distance algorithm is used to calculate the similarity of protocol packets.

[0036] The execution flow of protocol conversion engine includes:

[0037] In the protocol parsing stage, regular expressions or syntax trees are used to extract source protocol fields;

[0038] In the mapping stage, the field name is replaced and the data type is converted according to the XML rule;

[0039] In the verification stage, it is verified whether the JSON format meets the requirements of the platform data model;

[0040] In the encapsulation stage, timestamp, device ID and other metadata information are added to the converted data, and then output, so that the subsequent data processing and control operation can obtain complete and accurate data information.

[0041] The security sandbox technology uses Docker container or Kata Containers lightweight virtualization technology to limit the system resource access permission of the protocol parsing plug-in, and prevent potential attacks of the plug-in code on the gateway system.

[0042] The plug-in developer uses RSA-2048 algorithm to sign the plug-in code, and the gateway verifies the validity of the signature through CA certificate. If the verification fails, log is recorded and loading is refused, and administrator alarm is triggered.

[0043] Step S2, multi-source data acquisition and processing, real-time acquisition of multi-source data of multimedia devices, data fusion and anomaly detection through edge computing nodes;

[0044] In step S2, the following sub-steps are further included:

[0045] S2-1, collecting multi-source data of the equipment through a distributed sensor network; the multi-source data includes state data, environmental parameters and user position information; the state data includes:

[0046] Power supply status monitored in real time through voltage and current sensors;

[0047] Device surface temperature measured through an infrared sensor;

[0048] Wireless signal strength obtained through RSSI;

[0049] The environmental parameters include ambient light intensity, humidity value and noise decibel value collected through ZigBee terminal nodes; the user position information includes three-dimensional coordinates of the user obtained through ultra-wideband positioning technology;

[0050] S2-2, the edge computing node adopts a Kalman filtering algorithm to perform fusion processing on the multi-source data; wherein, the specific steps include:

[0051] S2-2-1, defining a state vector as a two-dimensional vector of the device surface temperature and the wireless signal strength, and defining an observation vector as a two-dimensional vector of the infrared sensor measurement value and the RSSI measurement value;

[0052] S2-2-2, designing an observation matrix as a two-dimensional non-diagonal matrix:

[0053]

[0054] Wherein, H represents a two-dimensional non-diagonal matrix, and a represents a cross-influence coefficient of temperature on signal strength, with a unit of dBm / ℃;

[0055] S2-2-3, initializing a state covariance matrix based on a 30-day historical data sliding window, and eliminating data noise through iterative calculation;

[0056] S2-2-4, establishing a mapping model of the environmental parameters and the state variable, and dynamically compensating the Kalman filtering output result, wherein:

[0057] The temperature compensation amount AT is calculated through a quadratic polynomial fitting of the humidity value, the logarithmic term of the light intensity and the noise decibel value, the polynomial coefficients are optimized through a gradient descent method, and the loss function is the mean square error of the compensated temperature and the infrared sensor measured value;

[0058] The signal strength compensation amount AS is calculated through a product term of the noise decibel value and the humidity value with an experimental calibration coefficient;

[0059] The compensated temperature is limited to 0℃ to 85℃, and the signal strength is limited to no less than -90dBm;

[0060] S2-3, identifying device failure based on a preset threshold and an isolation forest anomaly detection model; the preset threshold is calculated based on a mean value and a standard deviation of a 60-second sliding window to obtain a ±3σ range, the isolation forest anomaly detection model is used to perform outlier detection on the compensated data, and when the data simultaneously exceeds the preset threshold and is determined as an outlier, a sound-light alarm is triggered.

[0061] α is calibrated by linear regression, and the regression equation is:

[0062]

[0063] wherein, R represents a received signal strength indication; T represents a surface working temperature of the multimedia device; represents a theoretical signal strength when the temperature is 0℃; and α represents a cross-influence coefficient of temperature on signal strength.

[0064] It should be noted that the current sensor and the voltage sensor are both Hall effect sensors, with a measurement accuracy of ±1% FS, and real-time monitoring of the device power supply state.

[0065] The infrared sensor is a non-contact thermal imaging sensor with a resolution of ±0.5℃, covering the device surface temperature acquisition.

[0066] The ZigBee terminal node conforms to the IEEE 802.15.4 standard, and supports real-time collection of ambient light intensity, humidity value, and noise decibel value.

[0067] The ultra-wideband positioning is based on the DW1000 chip, realizing three-dimensional coordinate positioning of users with an accuracy of ≤10cm.

[0068] The multiple sensors access the edge computing node through the Mesh network, support dynamic routing switching, and guarantee the stability of data collection.

[0069] α in the observation matrix is calibrated by a temperature rise experiment, by gradually increasing the temperature from 25℃ to 85℃, recording the signal strength attenuation value and linearly regressing to obtain α.

[0070] The initial value of the state covariance matrix P is calculated based on the variance of 30-day historical data, and is iteratively updated by a prediction equation and an update equation to eliminate noise.

[0071] The initial value calculation formula of the state covariance matrix P is:

[0072]

[0073] wherein, represents the initial value of the state covariance matrix; represents the temperature history variance; represents signal strength history variance; the matrix is in diagonal form, indicating that the two variables of temperature and signal strength are independent of each other and have no coupling relationship in the initial state.

[0074] The calculation formula of temperature compensation is:

[0075]

[0076] wherein ΔT represents the temperature compensation amount, in ℃, for correcting the deviation between the temperature value output by the Kalman filter and the actual measured value; represents the basic compensation value without environmental influence; represents the humidity influence coefficient; H represents the environmental humidity value; represents the light intensity influence coefficient; L represents the environmental light intensity; represents the noise influence coefficient; N represents the environmental noise decibel value.

[0077] The signal strength compensation formula is:

[0078]

[0079] wherein ΔS represents the signal strength compensation amount, for correcting the deviation between the signal strength value output by the Kalman filter and the actual measured value; γ represents the experimental calibration coefficient; N represents the environmental noise decibel value; and A represents the environmental humidity value.

[0080] The isolated forest anomaly detection model generates 100 decision trees by taking the compensated temperature and signal strength sequences as inputs, and randomly selects 256 sample points for each tree to calculate the anomaly score of the sample (the closer the score is to 1, the higher the degree of anomaly), and determines that the device is malfunctioning when the data exceeds the ±3σ threshold value calculated based on the 60-second sliding window mean and standard deviation and the anomaly score is greater than 0.8, thereby triggering an audible and light alarm and uploading the fault information to the cloud.

[0081] Step S3, dynamic control strategy generation, generating a basic control strategy based on user instructions, optimizing the basic control strategy in combination with scene rules, adjusting the optimized strategy according to a user behavior prediction model, and generating a collaborative strategy including audio and video stream synchronous switching, resolution adaptive adjustment, and device parameter linkage control;

[0082] In step S3, the following sub-steps are further included:

[0083] S3-1, constructing a scene rule library for storing rule sets in XML format, each rule including a trigger condition field, a device action field, and a priority weight; the trigger condition field including state data, environmental parameters, and user location information;

[0084] S3-2, a user behavior prediction model is established based on an LSTM neural network, historical operation sequences, current state data and environmental parameters are input, and device operation probability distribution within 5 minutes in the future is output;

[0085] S3-3, the following operations are performed by a strategy optimization engine:

[0086] S3-3-1, the user instruction is parsed into a multimedia control instruction set, which is used to generate a basic control strategy;

[0087] S3-3-2, rules with a weight greater than 0.7 in the scene rule library are matched, and device linkage actions are added to the basic control strategy;

[0088] S3-3-3, the action execution order is adjusted according to the result of the user behavior prediction model, and the video decoder and network bandwidth resources are preloaded for operations with a probability greater than 80%;

[0089] S3-4, audio and video stream synchronous switching adopts RTP / RTCP protocol to realize multi-device clock synchronization, and resolution adaptive adjustment is based on device CPU utilization to select encoding format; when the CPU utilization exceeds the preset threshold, H.264 encoding is adopted, otherwise AV1 encoding is adopted.

[0090] It should be noted that the scene rule library adopts an XML format storage structure rule set, for example, a typical rule example is:

[0091] <rule id="rule_001" priority="0.8">

[0092] <condition>

[0093] <light_intensity>300Lux< / light_intensity>

[0094] <user_position distance="<=2m" / >

[0095] < / condition>

[0096] <action>

[0097] <device id="projector">turnOn< / device>

[0098] <device id="screen">lower< / device>

[0099] < / action>

[0100] < / rule>

[0101] Among them, id is the unique identifier of the rule; priority is the priority weight; the condition field supports logical operators; the action field can define multi-device linkage actions.

[0102] The user behavior prediction model adopts a three-layer architecture, in which:

[0103] The input layer includes historical operation sequences, current state data and environmental parameters;

[0104] The hidden layer includes 2 layers of LSTM, each layer has 128 neurons, and the activation function uses tanh;

[0105] The output layer outputs the probability distribution of each device operation type within 5 minutes in the future through Softmax activation.

[0106] RTP / RTCP protocol is responsible for the transmission and synchronization of audio and video data packets, NTP is used for clock reference alignment between devices to ensure the consistency of RTP timestamp.

[0107] Step S4, low-latency instruction issuance, through the hierarchical cache mechanism of the edge computing node and the time-sensitive network protocol stack, to guarantee the end-to-end control delay ≤15ms;

[0108] In step S4, the following sub-steps are further included:

[0109] S4-1, deploying a hierarchical cache mechanism in the edge computing node, including:

[0110] The primary cache stores high-frequency control instructions with a call frequency >10 times / minute;

[0111] The secondary cache updates device state snapshots based on state change events; the state change events include temperature change ≥2℃, signal strength change ≥5dBm, and voltage fluctuation ≥0.5V;

[0112] S4-2, the time-sensitive network protocol stack based on IEEE 802.1Qbv standard, performs the following operations:

[0113] S4-2-1, assigning the control instructions to the highest priority scheduling queue numbered 7 in the IEEE 802.1Qbv standard, with a scheduling queue bandwidth allocation ratio not less than 90%;

[0114] S4-2-2, in a single-hop network, assigning independent transmission time slots for control instructions through a time-aware shaper, to guarantee the end-to-end transmission delay of control instruction frames ≤2ms;

[0115] S4-3, verifying the end-to-end delay, based on IEEE 1588 PTP protocol synchronization clock, embedding a nanosecond-level timestamp in the instruction header, the receiving end checking the time difference and discarding control instructions with a delay >15ms; the edge computing node triggers a fast retransmission mechanism for instructions with consecutive 3 packet losses, with a retransmission interval of 5ms.

[0116] It should be noted that the edge computing node serves as a carrier for deploying hierarchical cache and protocol stack, for realizing localized fast processing and low-latency forwarding of control instructions.

[0117] The hierarchical cache mechanism manages data in layers through a multi-level cache architecture, differentiates storage according to access frequency and data characteristics, to optimize data processing efficiency.

[0118] The primary cache is implemented based on a Redis in-memory database, dynamically stores control instructions with a call frequency >10 times / minute, and realizes an average retrieval complexity O(1) in an ideal state through a hash table structure, with a cache retrieval delay ≤0.5ms and an instruction parsing delay ≤0.5ms.

[0119] The secondary cache is updated based on state change events, and stores key state parameters of devices in JSON format.

[0120] Time-sensitive network protocol stack is a set of network protocols based on IEEE 802.1 series standards, which introduces deterministic transmission capability for Ethernet through time synchronization, priority scheduling and time slot allocation, ensuring data frame transmission within strict time constraints.

[0121] IEEE 802.1Qbv standard is one of the key standards in the time-sensitive network protocol family, which defines time-aware shapers to achieve deterministic traffic scheduling by allocating independent transmission time slots for different priority data.

[0122] Time-aware shapers are the core components in IEEE 802.1Qbv standard, which allocate independent transmission time slots for different priority data frames based on a global synchronized clock.

[0123] IEEE 1588 PTP protocol, which stands for IEEE 1588 Precision Time Protocol, is a protocol for network device clock synchronization, achieving nanosecond-level precision time synchronization through master-slave clock mechanism

[0124] When the edge node detects 3 consecutive packet losses, it triggers the fast retransmission mechanism to actively resend control instructions to improve transmission reliability.

[0125] Step S5, multi-terminal collaborative interaction, supports mobile terminal, voice assistant and web interface remote control, and implements hierarchical permission management through role permission matrix;

[0126] In step S5, the following sub-steps are included:

[0127] S5-1, the role permission matrix is a two-dimensional permission table, the row dimension defines administrator, ordinary user and visitor roles, the column dimension defines device control, parameter modification and system configuration operation levels, and the permission value is represented by an integer level of 0-3; the permission value includes:

[0128] 0 means forbidden, 1 means read-only, 2 means basic operation, and 3 means full control;

[0129] S5-2, the steps of multi-terminal remote control include:

[0130] S5-2-1, the mobile terminal publishes TLS 1.3 encrypted control instructions through MQTT protocol;

[0131] S5-2-2, the voice assistant converts voice to JSON instructions through ASR engine, and the JSON instructions include device ID, operation type, parameter value and timestamp;

[0132] S5-2-3, the web interface uses WebSocket protocol for bidirectional communication, and verifies through JWT token with a validity period of ≤15 minutes when the connection is established;

[0133] S5-3, the modification request of the device parameter forces to verify the dynamic token generated based on the HMAC-SHA256 algorithm, and the validity period of the dynamic token is ≤30 seconds.

[0134] It should be noted that the structured design of the role permission matrix adopts a 3x3 two-dimensional permission table, and the access permission is defined by the cross mapping of rows and columns. The permission value mapping relationship is as shown in Table 1:

[0135]

[0136] Among them, the administrator (3 / 3 / 3) can perform all operations, including device restart, parameter calibration, and system firmware upgrade; the ordinary user (2 / 1 / 0) can perform basic control and read-only access to parameters, without system configuration permission; the visitor (1 / 0 / 0) can only view the device status and is prohibited from any modification operation.

[0137] 2 represents basic operations, wherein the basic operations include device switching, volume adjustment, and input source switching.

[0138] The mobile terminal adopts the MQTT 5.0 protocol, the QoS 1 level guarantees that the message is delivered at least once, the TLS 1.3 encryption adopts the ECDHE-ECDSA key exchange algorithm and the AES-256-GCM symmetric encryption, the certificate validity period is 3 months, and the bidirectional authentication is supported.

[0139] The MQTT protocol is a lightweight publish / subscribe message transmission protocol based on TCP / IP, and is suitable for low-bandwidth, high-latency, or unreliable network environments.

[0140] TLS 1.3 is the latest generation of transport layer encryption protocol, which reduces the number of handshake round trips, enhances the forward secrecy and attack resistance, and disables unsafe encryption algorithms by default.

[0141] The ASR engine is an automatic speech recognition technology that converts human speech into text. It usually includes three parts: an acoustic model, a language model, and a dictionary. Among them, the acoustic model runs locally, and the language model is processed in the cloud.

[0142] The WebSocket protocol is a full-duplex communication protocol based on TCP, which provides real-time bidirectional communication on a single TCP connection, avoiding the frequent handshake overhead of HTTP short connections.

[0143] The JWT token is a JSON-based open standard for securely transmitting claims between web applications, which includes three parts: header, payload, and signature.

[0144] HMAC-SHA256 is a key-dependent message authentication code using SHA-256 hash algorithm, which generates a fixed-length hash value through a key and a message, used to verify data integrity and identity.

[0145] Dynamic token is a one-time valid token generated based on timestamp, with timeliness, different value generated each time, used to enhance system security.

[0146] Web interface establishes bidirectional communication based on WebSocket protocol, WebSocket handshake phase carries JWT in HTTP header, and the server upgrades the protocol after verification.

[0147] In this way, a multimedia device centralized control method based on the Internet of Things can be implemented.

[0148] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A centralized control method for multimedia devices based on the Internet of Things, characterized in that, The method includes: Step S1: Device networking and protocol adaptation. Connect heterogeneous multimedia devices through an IoT gateway. The gateway supports dynamic parsing and compatible conversion of Modbus, TCP / IP, and HTTP protocols. Step S2: Multi-source data acquisition and processing. Real-time acquisition of multi-source data from multimedia devices, and data fusion and anomaly detection through edge computing nodes. Step S3: Dynamic control strategy generation. A basic control strategy is generated based on user instructions. The basic control strategy is optimized in combination with scene rules. The optimized strategy is adjusted according to the user behavior prediction model to generate a collaborative strategy including audio and video stream synchronous switching, resolution adaptive adjustment and device parameter linkage control. Step S4: Low-latency command issuance. Through the hierarchical caching mechanism of edge computing nodes and the time-sensitive network protocol stack, the end-to-end control latency is guaranteed to be ≤15ms. Step S5: Multi-terminal collaborative interaction, supporting remote control via mobile terminals, voice assistants and web interfaces, and implementing hierarchical permission management through a role-based permission matrix; Step S2 further includes the following sub-steps: S2-1, Collect multi-source data from the device through a distributed sensor network; the multi-source data includes status data, environmental parameters, and user location information; the status data includes: The power supply status is monitored in real time by voltage and current sensors; The surface temperature of the device is measured by an infrared sensor; Wireless signal strength obtained via RSSI; The environmental parameters include ambient light intensity collected by the ZigBee terminal node. Humidity and noise levels in decibels; the user location information includes the user's three-dimensional coordinates obtained through ultra-wideband positioning technology; S2-2, the edge computing nodes use the Kalman filter algorithm to fuse the multi-source data; the specific steps include: S2-2-1, define the state vector as a two-dimensional vector of device surface temperature and wireless signal strength, and define the observation vector as a two-dimensional vector of infrared sensor measurement value and RSSI measurement value; S2-2-2, the observation matrix is ​​designed as a two-dimensional off-diagonal matrix: Where H represents a two-dimensional off-diagonal matrix, and α represents the cross-influence coefficient of temperature on signal strength, in dBm / ℃; S2-2-3 initializes the state covariance matrix based on a sliding window of 30-day historical data and eliminates data noise through iterative calculation. S2-2-4, Establish a mapping model between environmental parameters and state variables, and dynamically compensate for the Kalman filter output, wherein: The temperature compensation amount ΔT is calculated by fitting a quadratic polynomial of the humidity value, the logarithmic term of the light intensity and the noise decibel value. The polynomial coefficients are optimized by the gradient descent method, and the loss function is the mean square error of the compensated temperature and the measured value of the infrared sensor. The signal strength compensation ΔS is calculated by using the product of the noise decibel value and the humidity value through experimental calibration coefficients. The compensated temperature is limited to 0℃ to 85℃, and the signal strength is limited to no less than -90dBm. S2-3, Identify equipment faults based on preset thresholds and an isolated forest anomaly detection model; the preset thresholds are calculated based on the mean and standard deviation of a 60-second sliding window within a range of ±3σ; the isolated forest anomaly detection model is used to perform outlier detection on the compensated data; when the data simultaneously exceeds the preset thresholds and is identified as an outlier, an audible and visual alarm is triggered. Step S3 further includes the following sub-steps: S3-1, Construct a scenario rule base to store rule sets in XML format. Each rule includes a trigger condition field, a device action field, and a priority weight. The trigger condition field includes status data, environmental parameters, and user location information. S3-2, a user behavior prediction model is built based on LSTM neural network. Input historical operation sequence, current state data and environmental parameters, and output the probability distribution of device operation in the next 5 minutes. S3-3 performs the following operations through the policy optimization engine: S3-3-1 parses user commands into a multimedia control instruction set, which is used to generate basic control strategies; S3-3-2, Match the rules with a weight > 0.7 in the scene rule base and add device linkage actions to the basic control strategy; S3-3-3, Adjust the action execution order according to the results of the user behavior prediction model, and preload video decoder and network bandwidth resources for operations with a probability >80%; S3-4, The audio and video stream synchronous switching adopts the RTP / RTCP protocol to realize the clock synchronization of multiple devices, and the resolution adaptive adjustment selects the encoding format based on the device CPU utilization; when the CPU utilization exceeds the preset threshold, H.264 encoding is used, otherwise AV1 encoding is used. Step S4 further includes the following sub-steps: S4-1 deploys a tiered caching mechanism on edge computing nodes, including: The first-level cache stores high-frequency control commands that are accessed more than 10 times per minute; The secondary cache updates the device state snapshot based on state change events; these state change events include temperature changes ≥2℃, signal strength changes ≥5dBm, and voltage fluctuations ≥0.5V. S4-2, the Time-Sensitive Networking Protocol stack, based on the IEEE 802.1Qbv standard, performs the following operations: S4-2-1, allocates the highest priority scheduling queue (numbered 7 in the IEEE 802.1Qbv standard) to the control commands, and the bandwidth allocation ratio of the scheduling queue is not less than 90%; S4-2-2, In a single-hop network, an independent transmission time slot is allocated to the control command through a time-aware shaper to ensure that the end-to-end transmission delay of the control command frame is ≤2ms; S4-3, verify end-to-end delay, synchronize clock based on IEEE 1588 PTP protocol, embed nanosecond-level timestamp in instruction header, the receiving end verifies time difference and discards control instructions with delay >15ms; the edge computing node triggers fast retransmission mechanism for instructions with 3 consecutive packet loss, with retransmission interval of 5ms.

2. The centralized control method for multimedia devices based on the Internet of Things according to claim 1, characterized in that: Step S1 further includes the following sub-steps: S1-1, The IoT gateway performs feature extraction and matching parsing on the Modbus, TCP / IP, and HTTP protocol messages based on a pre-built protocol feature library; the protocol feature library is used to store feature vector templates for each protocol message, and the feature vector includes a combination of protocol identifier field, function code field, and verification field, and protocol matching is completed through a preset similarity threshold; S1-2, through the protocol conversion engine, heterogeneous protocol data is mapped to JSON format of the unified data model within the platform based on the conversion rules defined by the XML schema; the XML schema definition includes the mapping relationship between the source protocol fields and the target JSON format fields; S1-3, Establish a dynamic protocol loading mechanism to support loading protocol parsing plugins through a secure sandbox; the plugins implement predefined data parsing interfaces and verify their legitimacy through digital signatures before loading.

3. The centralized control method for multimedia devices based on the Internet of Things according to claim 1, characterized in that: The α is determined by linear regression, and the regression equation is: Where R represents the received signal strength indicator; T represents the surface operating temperature of the multimedia device; This represents the theoretical signal strength at a temperature of 0℃; α represents the cross-influence coefficient of temperature on signal strength.

4. The centralized control method for multimedia devices based on the Internet of Things according to claim 1, characterized in that: Step S5 further includes the following sub-steps: S5-1, the role-permission matrix is ​​a two-dimensional permission table. The row dimension defines the roles of administrator, regular user, and visitor, and the column dimension defines the operation levels for device control, parameter modification, and system configuration. Permission values ​​are represented by integer levels from 0 to 3. The permission values ​​include: 0 indicates prohibition, 1 indicates read-only, 2 indicates basic operations, and 3 indicates full control; S5-2, the steps for achieving remote control across multiple terminals include: S5-2-1, the mobile terminal publishes TLS 1.3 encrypted control commands via the MQTT protocol; S5-2-2, the voice assistant converts speech into JSON commands through the ASR engine. The JSON commands include device ID, operation type, parameter value, and timestamp. S5-2-3, the web interface uses the WebSocket protocol for bidirectional communication, and the connection is verified by a JWT token with a validity period of ≤15 minutes when it is established; S5-3, Force verification of device parameter modification requests using a dynamic token generated based on the HMAC-SHA256 algorithm, wherein the dynamic token has a validity period of ≤30 seconds.

Images