Multimedia equipment centralized control method based on Internet of Things

Abstract

The invention relates to the technical field of Internet of Things and multimedia communication, in particular to a multimedia equipment centralized control method based on the Internet of Things, which comprises the following steps: realizing heterogeneous equipment access through a gateway dynamic analysis protocol of the Internet of Things; collecting multi-source data by utilizing an edge computing node, and detecting anomaly through Kalman filtering fusion and an isolated forest model; a dynamic control strategy is generated based on an LSTM neural network and a scene rule base, and synchronous switching of audio and video streams and self-adaptive adjustment of resolution are realized; control delay is guaranteed through hierarchical caching and a time-sensitive network protocol stack; and multi-terminal control is supported, and hierarchical authority management is implemented through a role authority matrix. According to the invention, the problems of difficult compatibility, insufficient control real-time performance and extensive safety management of heterogeneous equipment are solved, and the cooperation efficiency, response speed and use safety of multimedia equipment are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things and multimedia communication, and in particular to a centralized control method for multimedia devices based on the Internet of Things. Background Art

[0002] Early multimedia device control relied primarily on manual operation or simple remote controls. Each device typically came with its own remote control or control panel, requiring users to individually control their TV, stereo, projector, and other devices. This decentralized control approach met basic needs when the number of devices was small, but as the number and variety of devices increased, operational complexity increased exponentially: users needed to memorize the operating logic of each device and familiarize themselves with the use of multiple remote controls. Furthermore, coordinated operation between devices proved difficult, significantly impacting ease of use and a consistent user experience.

[0003] With the gradual development of computer technology and network communication technology, centralized control methods based on computers or networks have emerged. Such solutions achieve unified management of multiple devices by writing specific control software and using the computer's serial port, parallel port, or network interface to send control instructions to multimedia devices. However, these methods have significant application barriers: on the one hand, the establishment and configuration of the system requires users to have certain professional technical knowledge, involving complex operations such as device protocol parameter configuration and network interface debugging, and have high requirements for the stability of the device's network environment and computer performance, making it difficult to widely popularize them among ordinary users; on the other hand, the functions of this type of control method are relatively basic, usually only able to implement simple operations such as device switching and volume adjustment. They lack support for complex multimedia scene control and deep linkage between devices, and cannot meet the needs of intelligent scenes.

[0004] At present, although some commercial centralized control systems for multimedia devices have appeared on the market, these systems still have many technical bottlenecks: in terms of device compatibility, most systems can only support specific brands or specific types of devices, and lack support for the access of heterogeneous devices using different communication protocols, making it difficult to achieve truly universal centralized control; in terms of intelligence, existing systems rely on preset rules for control, and are unable to automatically generate personalized control strategies based on users' actual needs, real-time environmental parameters, and usage scenarios, nor can they make forward-looking control adjustments by learning users' long-term behavior patterns; in terms of performance and security, command transmission relies on general network protocols, and real-time performance is difficult to guarantee. Control delays or command losses are prone to occur during collaborative interaction among multiple terminals, and the authority management mechanism is relatively extensive, making it impossible to achieve refined hierarchical authorization for operations such as device control, parameter modification, and system configuration, making it difficult to meet users' needs for efficient, convenient, and secure centralized control.

[0005] Therefore, the present invention discloses a method for centralized control of multimedia devices based on the Internet of Things. Summary of the Invention

[0006] The purpose of the present invention is to solve the problems in the prior art of complex access of heterogeneous multimedia devices, inefficient data processing, static control strategy, high transmission delay and insufficient security of multi-terminal interaction, and to propose a centralized control method for multimedia devices based on the Internet of Things.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: a method for centralized control of multimedia devices based on the Internet of Things, comprising the following steps:

[0008] Step S1: Device networking and protocol adaptation, accessing heterogeneous multimedia devices through an IoT gateway that supports dynamic parsing and compatible conversion of Modbus, TCP / IP, and HTTP protocols;

[0009] Step S2, multi-source data collection and processing, real-time collection of multi-source data from multimedia devices, and data fusion and anomaly detection through edge computing nodes;

[0010] Step S3: Dynamic control strategy generation: Generate a basic control strategy based on user instructions, optimize the basic control strategy based on scenario rules, adjust the optimized strategy based on the user behavior prediction model, and generate a collaborative strategy including synchronous switching of audio and video streams, adaptive resolution adjustment, and coordinated control of device parameters;

[0011] Step S4: Low-latency instructions are issued, and the end-to-end control delay is guaranteed to be ≤15ms through the hierarchical caching mechanism of the edge computing node and the time-sensitive network protocol stack;

[0012] Step S5: Multi-terminal collaborative interaction, supporting remote control by mobile terminals, voice assistants, and web interfaces, and implementing hierarchical authority management through a role authority matrix.

[0013] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0014] The present invention supports dynamic parsing and compatible conversion of Modbus, TCP / IP, and HTTP protocols through the Internet of Things gateway, and combines it with the protocol dynamic loading mechanism to effectively solve the problems of difficult access to heterogeneous devices and high adaptation costs in traditional solutions, significantly improving the system's compatibility and expansion flexibility with multimedia devices from different manufacturers, and realizing plug-and-play of devices.

[0015] This invention uses the multi-source data fusion and anomaly detection mechanism of edge computing nodes, the Kalman filtering algorithm and the isolation forest model to break through the limitations of traditional independent data processing, accurately integrate multi-dimensional data such as equipment status and environmental parameters, improve the accuracy of data processing and the reliability of fault detection, and realize intelligent perception of equipment operating status and abnormal warning.

[0016] Based on the user behavior prediction model and scenario rule library of the LSTM neural network, the present invention dynamically generates a collaborative control strategy that includes synchronous switching of audio and video streams and adaptive adjustment of resolution, changing the traditional static control mode so that the system can automatically optimize device operation according to user behavior habits and real-time scenarios, thereby improving control response speed and user interaction experience.

[0017] The present invention supports multi-terminal remote control of mobile terminals, voice assistants, and Web interfaces, and combines a role permission matrix with a dynamic token authentication mechanism to achieve refined grading of control permissions and enhanced security authentication for sensitive operations. Compared with traditional extensive permission management, it significantly improves system interaction efficiency and data security protection capabilities.

[0018] The present invention organically integrates heterogeneous protocol adaptation, intelligent data processing, dynamic control strategies, low-latency transmission and multi-terminal secure interaction, breaking through the limitations of existing system fragmentation control and providing an efficient, intelligent and secure integrated solution for the centralized control of IoT multimedia devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A method flow chart of a method for centralized control of multimedia devices based on the Internet of Things is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features and effects of a centralized control method for multimedia devices based on the Internet of Things proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable form.

[0022] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0023] The following examples are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0024] The following describes in detail a specific solution of a method for centralized control of multimedia devices based on the Internet of Things provided by the present invention with reference to the accompanying drawings.

[0025] See also Figure 1 , which shows a method flow chart of a method for centralized control of multimedia devices based on the Internet of Things provided by an embodiment of the present invention, the method comprising the following steps:

[0026] Step S1: Device networking and protocol adaptation, access to heterogeneous multimedia devices through the IoT gateway, which supports dynamic parsing and compatible conversion of Modbus, TCP / IP, and HTTP protocols;

[0027] Wherein, step S1 further includes the following sub-steps:

[0028] S1-1, the IoT gateway extracts and matches Modbus, TCP / IP, and HTTP protocol messages based on a pre-built protocol feature library. The protocol feature library stores feature vector templates for each protocol message. The feature vectors include a combination of a protocol identification field, a function code field, and a checksum field. Protocol matching is performed using a preset similarity threshold.

[0029] S1-2, through the protocol conversion engine, maps heterogeneous protocol data into the JSON format of the platform's internal unified data model based on the conversion rules defined in the XML schema; the XML schema definition includes the mapping relationship between the source protocol fields and the target JSON format fields;

[0030] S1-3, establish a dynamic protocol loading mechanism to support loading protocol parsing plug-ins through a secure sandbox; the plug-in must implement a predefined data parsing interface and verify its legitimacy through a digital signature before loading.

[0031] It should be noted that the function code of the Modbus protocol is its key identification field. For example, function code 0x01 indicates reading a coil, 0x02 indicates reading a discrete input, 0x03 indicates reading a holding register, 0x04 indicates reading an input register, 0x05 indicates writing a single coil, 0x06 indicates writing a single register, 0x0F indicates writing multiple coils, and 0x10 indicates writing multiple registers. These function codes can instruct the Modbus device to perform the corresponding operation.

[0032] The port number of the TCP / IP protocol is an important identification field. For example, the HTTP protocol uses ports 80 and 443 by default. Through the port number, the TCP / IP protocol can accurately send network data packets to the corresponding application program, realizing the distinction between different services.

[0033] The HTTP protocol's "Method" field identifies the request method. For example, GET indicates retrieving resources, while POST indicates submitting data to the server. Different "Method" fields instruct the client to perform different operations on resources on the server.

[0034] The role of the protocol feature library is to establish a mapping relationship between protocol messages and parsing rules, so that the IoT gateway can determine the communication protocol used by the device by parsing the characteristic fields in the message, call the corresponding protocol parsing plug-in based on the identification results, and realize the standardized processing of heterogeneous data.

[0035] The cosine similarity or edit distance algorithm is used to calculate the similarity of protocol messages.

[0036] The execution process of the protocol conversion engine includes:

[0037] In the protocol parsing phase, source protocol fields are extracted through regular expressions or syntax trees;

[0038] In the mapping phase, field names are replaced and data types are converted according to XML rules;

[0039] During the verification phase, verify whether the JSON format meets the requirements of the platform data model;

[0040] During the encapsulation phase, metadata information such as timestamps and device IDs are added to the converted data before output, so that subsequent data processing and control operations can obtain complete and accurate data information.

[0041] The security sandbox technology uses Docker containers or Kata Containers lightweight virtualization technology to limit the system resource access rights of the protocol parsing plug-in, preventing potential attacks on the gateway system by the plug-in code.

[0042] The plug-in developer of the digital signature verification uses the RSA-2048 algorithm to sign the plug-in code. The gateway verifies the validity of the signature through the CA certificate. If the verification fails, a log is recorded and the loading is refused, and an administrator alarm is triggered.

[0043] Step S2, multi-source data collection and processing, real-time collection of multi-source data from multimedia devices, and data fusion and anomaly detection through edge computing nodes;

[0044] Wherein, in step S2, the following sub-steps are also included:

[0045] S2-1, collect multi-source data of the device through a distributed sensor network; the multi-source data includes status data, environmental parameters and user location information; the status data includes:

[0046] Real-time monitoring of power status via voltage and current sensors;

[0047] Equipment surface temperature measured by infrared sensors;

[0048] Wireless signal strength obtained through RSSI;

[0049] Environmental parameters include ambient light intensity, humidity, and noise decibel values ​​collected through ZigBee terminal nodes; user location information includes the user's three-dimensional coordinates obtained through ultra-wideband positioning technology;

[0050] S2-2: The edge computing node uses the Kalman filter algorithm to fuse multi-source data. The specific steps include:

[0051] S2-2-1, define the state vector as a two-dimensional vector of the device surface temperature and the wireless signal strength, and define the observation vector as a two-dimensional vector of the infrared sensor measurement value and the RSSI measurement value;

[0052] S2-2-2, design the observation matrix as a two-dimensional non-diagonal matrix:

[0053]

[0054] Where H represents a two-dimensional non-diagonal matrix, α represents the cross-effect coefficient of temperature on signal strength, in dBm / °C;

[0055] S2-2-3, initialize the state covariance matrix based on the 30-day historical data sliding window, and eliminate data noise through iterative calculation;

[0056] S2-2-4, establish a mapping model between environmental parameters and state variables, and dynamically compensate the Kalman filter output results, where:

[0057] The temperature compensation value ΔT is calculated by fitting a quadratic polynomial based on the humidity value, the logarithmic term of the light intensity, and the noise decibel value. The polynomial coefficients are optimized using the gradient descent method, and the loss function is the mean square error between the compensated temperature and the actual value measured by the infrared sensor.

[0058] The signal strength compensation ΔS is calculated by multiplying the noise decibel value and the humidity value using the experimental calibration coefficient;

[0059] Limit the compensated temperature to 0°C to 85°C and the signal strength to no less than -90dBm;

[0060] S2-3, identifies equipment faults based on preset thresholds and the isolation forest anomaly detection model; the preset threshold is calculated based on the mean and standard deviation of a 60-second sliding window within a ±3σ range, and the isolation forest anomaly detection model is used to perform outlier detection on the compensated data. When the data exceeds the preset threshold and is determined to be an outlier at the same time, an audible and visual alarm is triggered.

[0061] α is calibrated by linear regression, and the regression equation is:

[0062] R=α0+α·T

[0063] Wherein, R represents the received signal strength indicator; T represents the surface operating temperature of the multimedia device; α0 represents the theoretical signal strength when the temperature is 0°C; and α represents the temperature influence coefficient.

[0064] It should be noted that both the current sensor and the voltage sensor use Hall effect sensors with a measurement accuracy of ±1% FS, and monitor the power status of the device in real time.

[0065] The infrared sensor uses a non-contact thermal imaging sensor with a resolution of ±0.5°C, covering the surface temperature of the equipment.

[0066] ZigBee terminal nodes comply with the IEEE 802.15.4 standard and support real-time collection of ambient light intensity, humidity, and noise decibel values.

[0067] Ultra-wideband positioning is based on the DW1000 chip, which enables users to locate their three-dimensional coordinates with an accuracy of ≤10cm.

[0068] Multiple types of sensors are connected to edge computing nodes through Mesh networks, supporting dynamic routing switching to ensure the stability of data collection.

[0069] The α in the observation matrix is ​​calibrated through a temperature rise experiment. By gradually increasing the temperature from 25°C to 85°C, the signal intensity attenuation value is recorded and linear regression is performed to obtain α.

[0070] The initial value of the state covariance matrix P is calculated based on the variance of 30-day historical data and is iteratively updated through the prediction equation and update equation to eliminate noise.

[0071] The initial value calculation formula of the state covariance matrix P is:

[0072]

[0073] Among them, P0 represents the initial value of the state covariance matrix; represents the temperature history variance; Represents the historical variance of signal strength; the matrix is ​​in diagonal form, indicating that the temperature and signal strength variables are independent of each other in the initial state and have no coupling relationship.

[0074] The calculation formula for temperature compensation is:

[0075] ΔT=β0+β1H+β2ln(L)+β3N 2

[0076] Where ΔT is the temperature compensation value (in °C), which is used to correct the deviation between the temperature value output by the Kalman filter and the actual measurement value; β0 is the basic compensation value when there is no environmental influence; β1 is the humidity influence coefficient; H is the ambient humidity value; β2 is the light intensity influence coefficient; L is the ambient light intensity; β3 is the noise influence coefficient; and N is the ambient noise decibel value.

[0077] The signal strength compensation formula is:

[0078] ΔS=γ·(N×A)

[0079] Wherein, ΔS represents the signal strength compensation, which is used to correct the deviation between the signal strength value output by the Kalman filter and the actual measured value; γ represents the experimental calibration coefficient; N represents the decibel value of the ambient noise; and A represents the ambient humidity value.

[0080] The Isolation Forest Anomaly Detection Model uses the compensated temperature and signal strength sequences as input to generate 100 decision trees. Each tree randomly selects 256 sample points to calculate the sample's anomaly score (the closer the score is to 1, the higher the degree of anomaly). When the data exceeds the ±3σ threshold calculated based on the mean and standard deviation of a 60-second sliding window and the anomaly score is greater than 0.8, it is determined to be a device failure, triggering an audible and visual alarm and uploading the fault information to the cloud.

[0081] Step S3: Dynamic control strategy generation: Generate a basic control strategy based on user instructions, optimize the basic control strategy based on scenario rules, adjust the optimized strategy based on the user behavior prediction model, and generate a collaborative strategy including synchronous switching of audio and video streams, adaptive resolution adjustment, and coordinated control of device parameters;

[0082] Wherein, in step S3, the following sub-steps are also included:

[0083] S3-1, build a scenario rule library to store XML-formatted rule sets. Each rule includes a trigger condition field, a device action field, and a priority weight. The trigger condition field includes status data, environmental parameters, and user location information.

[0084] S3-2, builds a user behavior prediction model based on the LSTM neural network, inputs historical operation sequences, current status data and environmental parameters, and outputs the probability distribution of device operations in the next 5 minutes;

[0085] S3-3, the policy optimization engine performs the following operations:

[0086] S3-3-1, parsing user instructions into multimedia control instruction sets for generating basic control strategies;

[0087] S3-3-2, matching the rules with weights > 0.7 in the scenario rule library, and adding device linkage actions to the basic control strategy;

[0088] S3-3-3, adjust the action execution order based on the results of the user behavior prediction model, and preload video decoders and network bandwidth resources for operations with a probability greater than 80%;

[0089] In S3-4, the synchronous switching of audio and video streams uses the RTP / RTCP protocol to achieve multi-device clock synchronization, and the resolution is adaptively adjusted to select the encoding format based on the device CPU utilization; when the CPU utilization exceeds the preset threshold, H.264 encoding is used, otherwise AV1 encoding is used.

[0090] It should be noted that the scenario rule library uses XML format to store structured rule sets. For example, a typical rule example is:

[0091]

[0092]

[0093] Among them, id is the unique identifier of the rule; priority is the priority weight; the condition field supports logical operators; and the action field can define multi-device linkage actions.

[0094] The user behavior prediction model adopts a three-layer architecture, where:

[0095] The input layer includes historical operation sequences, current state data, and environmental parameters;

[0096] The hidden layer consists of 2 LSTM layers, each with 128 neurons, and the activation function uses tanh;

[0097] The output layer uses Softmax activation to output the probability distribution of each device operation type in the next 5 minutes.

[0098] The RTP / RTCP protocol is responsible for the transmission and synchronization of audio and video data packets, and NTP is used to align clock references between devices to ensure the consistency of RTP timestamps.

[0099] Step S4: Low-latency instructions are issued, and the end-to-end control delay is guaranteed to be ≤15ms through the hierarchical caching mechanism of the edge computing node and the time-sensitive network protocol stack;

[0100] Wherein, in step S4, the following sub-steps are also included:

[0101] S4-1, deploy a hierarchical caching mechanism on edge computing nodes, including:

[0102] The first-level cache stores high-frequency control instructions with a call frequency of more than 10 times per minute;

[0103] The L2 cache updates the device status snapshot based on status change events. Status change events include temperature change ≥ 2°C, signal strength change ≥ 5dBm, and voltage fluctuation ≥ 0.5V.

[0104] S4-2, the time-sensitive networking protocol stack is based on the IEEE 802.1Qbv standard and performs the following operations:

[0105] S4-2-1: Assign the highest priority scheduling queue numbered 7 in the IEEE 802.1Qbv standard to control instructions, and the bandwidth allocation ratio of the scheduling queue shall not be less than 90%;

[0106] S4-2-2, in a single-hop network, a time-aware shaper is used to allocate independent transmission time slots for control instructions, ensuring that the end-to-end transmission delay of control instruction frames is ≤2ms;

[0107] S4-3 verifies end-to-end latency by synchronizing clocks based on the IEEE 1588PTP protocol and embedding nanosecond timestamps in the instruction header. The receiving end verifies the time difference and discards control instructions with a delay greater than 15ms. The edge computing node triggers a fast retransmission mechanism for instructions that have been lost three times in a row, with a retransmission interval of 5ms.

[0108] It should be noted that edge computing nodes serve as carriers for hierarchical caching and protocol stack deployment, and are used to achieve localized rapid processing and low-latency forwarding of control instructions.

[0109] The hierarchical cache mechanism manages data in layers through a multi-level cache architecture, and stores data differently based on access frequency and data characteristics to optimize data processing efficiency.

[0110] The first-level cache is implemented based on the Redis in-memory database, dynamically storing control instructions with a call frequency of >10 times / minute. The hash table structure is used to achieve an average retrieval complexity of O(1) under ideal conditions, with a cache retrieval delay of ≤0.5ms and an instruction parsing delay of ≤0.5ms.

[0111] The secondary cache triggers updates based on status change events and stores key device status parameters in JSON format.

[0112] The Time-Sensitive Networking (TSN) protocol stack is a set of network protocols based on the IEEE 802.1 series of standards. It introduces deterministic transmission capabilities to Ethernet through time synchronization, priority scheduling, and time slot allocation, ensuring that data frames are transmitted within strict time constraints.

[0113] The IEEE 802.1Qbv standard is one of the key standards in the Time-Sensitive Networking (TSN) protocol suite. It defines a time-aware shaper that implements deterministic traffic scheduling by allocating independent transmission time slots to data of different priorities.

[0114] The time-aware shaper is a core component of the IEEE 802.1Qbv standard. It allocates independent transmission time slots for data frames of different priorities based on a globally synchronized clock.

[0115] IEEE 1588PTP is the full name of IEEE 1588 Precision Time Protocol, a protocol used for clock synchronization of network devices. It achieves nanosecond-level precision time synchronization through a master-slave clock mechanism.

[0116] When an edge node detects three consecutive packet losses, it triggers the fast retransmission mechanism and actively resends the control instructions to improve transmission reliability.

[0117] Step S5: Multi-terminal collaborative interaction, supporting remote control via mobile terminals, voice assistants, and web interfaces, and implementing hierarchical authority management through a role authority matrix;

[0118] Wherein, in step S5, the following sub-steps are also included:

[0119] S5-1, the role permission matrix is ​​a two-dimensional permission table. The row dimension defines the administrator, ordinary user, and guest roles, and the column dimension defines the device control, parameter modification, and system configuration operation levels. The permission values ​​are expressed as integer levels from 0 to 3. The permission values ​​include:

[0120] 0 means prohibited, 1 means read-only, 2 means basic operations, and 3 means full control;

[0121] S5-2, the steps for implementing remote control of multiple terminals include:

[0122] S5-2-1, the mobile terminal publishes TLS1.3 encrypted control instructions via the MQTT protocol;

[0123] S5-2-2, the voice assistant converts the voice into JSON instructions through the ASR engine. The JSON instructions include device ID, operation type, parameter value and timestamp;

[0124] S5-2-3, the web interface uses the WebSocket protocol for two-way communication, and the connection is established through the JWT token with a validity period of ≤ 15 minutes for verification;

[0125] S5-3: Requests to modify device parameters must be verified using a dynamic token generated using the HMAC-SHA256 algorithm. The dynamic token is valid for ≤ 30 seconds.

[0126] It should be noted that the structured design of the role permission matrix adopts a 3×3 two-dimensional permission table, which defines access rights through cross-mapping of rows and columns. The permission value mapping relationship is shown in Table 1:

[0127] Table 1

[0128]

[0129] Among them, the administrator (3 / 3 / 3) can perform all operations, including device restart, parameter calibration, and system firmware upgrade; ordinary users (2 / 1 / 0) can perform basic control, read-only access parameters, and have no system configuration permissions; guests (1 / 0 / 0) can only view the device status and are prohibited from any modification operations.

[0130] 2 represents basic operations, including device power on / off, volume adjustment, and input source switching.

[0131] The mobile terminal uses the MQTT 5.0 protocol, QoS1 level ensures that messages are delivered at least once, TLS1.3 encryption uses the ECDHE-ECDSA key exchange algorithm and AES-256-GCM symmetric encryption, the certificate is valid for 3 months, and supports two-way authentication.

[0132] The MQTT protocol is a lightweight publish / subscribe message transmission protocol based on TCP / IP and is suitable for low-bandwidth, high-latency or unreliable network environments.

[0133] TLS1.3 is the latest generation of transport layer encryption protocol. TLS1.3 reduces the number of handshake round trips, enhances forward secrecy and anti-attack capabilities, and disables insecure encryption algorithms by default.

[0134] The ASR engine is an automatic speech recognition technology, a system that converts human speech into text. It usually consists of three parts: an acoustic model, a language model, and a dictionary. The acoustic model runs locally, and the language model is processed in the cloud.

[0135] The WebSocket protocol is a full-duplex communication protocol based on TCP, providing real-time two-way communication over a single TCP connection, avoiding the frequent handshake overhead of HTTP short connections.

[0136] The JWT token is an open standard based on JSON, used to securely transmit statements between network applications. It consists of three parts: header, payload, and signature.

[0137] HMAC-SHA256 is a key-dependent message authentication code that uses the SHA-256 hash algorithm. It generates a fixed-length hash value using a key and a message to verify data integrity and identity.

[0138] A dynamic token is a one-time valid token generated based on a timestamp. It is time-sensitive and generates a different value for each request to enhance system security.

[0139] The web interface establishes two-way communication based on the WebSocket protocol. During the WebSocket handshake phase, JWT is carried in the HTTP header, and the protocol is upgraded after the server verifies it.

[0140] In this way, a centralized control method for multimedia devices based on the Internet of Things can be realized.

[0141] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for centralized control of multimedia devices based on the Internet of Things, characterized in that: The method includes: Step S1: Device networking and protocol adaptation, accessing heterogeneous multimedia devices through an IoT gateway that supports dynamic parsing and compatible conversion of Modbus, TCP / IP, and HTTP protocols; Step S2, multi-source data collection and processing, real-time collection of multi-source data from multimedia devices, and data fusion and anomaly detection through edge computing nodes; Step S3: Dynamic control strategy generation: Generate a basic control strategy based on user instructions, optimize the basic control strategy based on scenario rules, adjust the optimized strategy based on the user behavior prediction model, and generate a collaborative strategy including synchronous switching of audio and video streams, adaptive resolution adjustment, and coordinated control of device parameters; Step S4: Low-latency instructions are issued, and the end-to-end control delay is guaranteed to be ≤15ms through the hierarchical caching mechanism of the edge computing node and the time-sensitive network protocol stack; Step S5: Multi-terminal collaborative interaction, supporting remote control by mobile terminals, voice assistants, and web interfaces, and implementing hierarchical authority management through a role authority matrix.

2. The method for centralized control of multimedia devices based on the Internet of Things according to claim 1, characterized in that: Wherein, step S1 further includes the following sub-steps: S1-1, the IoT gateway performs feature extraction and matching analysis on the Modbus, TCP / IP, and HTTP protocol messages based on a pre-built protocol feature library; the protocol feature library is used to store feature vector templates of each protocol message, and the feature vector includes a combination of a protocol identification field, a function code field, and a checksum field, and protocol matching is completed using a preset similarity threshold; S1-2, using a protocol conversion engine, maps heterogeneous protocol data into a JSON format of a unified data model within the platform based on conversion rules defined in an XML schema; the XML schema definition includes a mapping relationship between source protocol fields and target JSON format fields; S1-3, establish a protocol dynamic loading mechanism to support loading protocol parsing plug-ins through a secure sandbox; the plug-in implements a predefined data parsing interface and verifies its legitimacy through a digital signature before loading.

3. The method for centralized control of multimedia devices based on the Internet of Things according to claim 1, characterized in that: Wherein, in step S2, the following sub-steps are also included: S2-1, collecting multi-source data of the device through a distributed sensor network; the multi-source data includes status data, environmental parameters and user location information; the status data includes: Real-time monitoring of power status via voltage and current sensors; Equipment surface temperature measured by infrared sensors; Wireless signal strength obtained through RSSI; The environmental parameters include the ambient light intensity, humidity value and noise decibel value collected by the ZigBee terminal node; the user location information includes the user's three-dimensional coordinates obtained by ultra-wideband positioning technology; S2-2, the edge computing node uses the Kalman filter algorithm to fuse the multi-source data; wherein the specific steps include: S2-2-1, define the state vector as a two-dimensional vector of the device surface temperature and the wireless signal strength, and define the observation vector as a two-dimensional vector of the infrared sensor measurement value and the RSSI measurement value; S2-2-2, design the observation matrix as a two-dimensional non-diagonal matrix: Where H represents a two-dimensional non-diagonal matrix, α represents the cross-effect coefficient of temperature on signal strength, in dBm / °C; S2-2-3, initialize the state covariance matrix based on the 30-day historical data sliding window, and eliminate data noise through iterative calculation; S2-2-4, establish a mapping model between environmental parameters and state variables, and dynamically compensate the Kalman filter output results, where: The temperature compensation value ΔT is calculated by fitting a quadratic polynomial based on the humidity value, the logarithmic term of the light intensity, and the noise decibel value. The polynomial coefficients are optimized using the gradient descent method, and the loss function is the mean square error between the compensated temperature and the actual value measured by the infrared sensor. The signal strength compensation ΔS is calculated by multiplying the noise decibel value and the humidity value using the experimental calibration coefficient; Limit the compensated temperature to 0°C to 85°C and the signal strength to no less than -90dBm; S2-3, identifying equipment failures based on a preset threshold and an isolation forest anomaly detection model; the preset threshold is calculated based on the mean and standard deviation of a 60-second sliding window within a ±3σ range, and the isolation forest anomaly detection model is used to perform outlier detection on the compensated data. When the data exceeds the preset threshold and is determined to be an outlier at the same time, an audible and visual alarm is triggered.

4. The method for centralized control of multimedia devices based on the Internet of Things according to claim 3, characterized in that: The α is calibrated by linear regression, and the regression equation is: R=α0+α·T Wherein, R represents the received signal strength indicator; T represents the surface operating temperature of the multimedia device; α0 represents the theoretical signal strength when the temperature is 0°C; and α represents the temperature influence coefficient.

5. The method for centralized control of multimedia devices based on the Internet of Things according to claim 1, characterized in that: Wherein, in step S3, the following sub-steps are also included: S3-1, building a scenario rule library for storing rule sets in XML format. Each rule includes a trigger condition field, a device action field, and a priority weight. The trigger condition field includes state data, environmental parameters, and user location information. S3-2, builds a user behavior prediction model based on the LSTM neural network, inputs historical operation sequences, current status data and environmental parameters, and outputs the probability distribution of device operations in the next 5 minutes; S3-3, the policy optimization engine performs the following operations: S3-3-1, parsing user instructions into multimedia control instruction sets for generating basic control strategies; S3-3-2, matching rules with a weight greater than 0.7 in the scenario rule library, and adding device linkage actions to the basic control strategy; S3-3-3, adjusting the order of action execution based on the results of the user behavior prediction model, and preloading video decoders and network bandwidth resources for operations with a probability greater than 80%; S3-4, synchronous switching of audio and video streams uses RTP / RTCP protocol to achieve multi-device clock synchronization, and resolution adaptive adjustment selects the encoding format based on the device CPU utilization; when the CPU utilization exceeds the preset threshold, H.264 encoding is used, otherwise AV1 encoding is used.

6. The method for centralized control of multimedia devices based on the Internet of Things according to claim 1, characterized in that: Wherein, in step S4, the following sub-steps are also included: S4-1, deploy a hierarchical caching mechanism on edge computing nodes, including: The first-level cache stores high-frequency control instructions with a call frequency of more than 10 times per minute; The L2 cache updates the device status snapshot based on status change events; the status change events include temperature change ≥ 2°C, signal strength change ≥ 5dBm, and voltage fluctuation ≥ 0.5V; S4-2, the time-sensitive networking protocol stack is based on the IEEE 802.1Qbv standard and performs the following operations: S4-2-1, allocating the highest priority scheduling queue numbered 7 in the IEEE 802.1Qbv standard to the control instruction, with the bandwidth allocation ratio of the scheduling queue being no less than 90%; S4-2-2, in a single-hop network, allocate independent transmission time slots for control instructions through a time-aware shaper to ensure that the end-to-end transmission delay of the control instruction frame is ≤ 2ms; S4-3 verifies end-to-end latency by synchronizing clocks based on the IEEE 1588PTP protocol, embedding nanosecond timestamps in the instruction header, and the receiving end verifies the time difference and discards control instructions with a delay greater than 15ms. The edge computing node triggers a fast retransmission mechanism for instructions that have been lost three times in a row, with a retransmission interval of 5ms.

7. The method for centralized control of multimedia devices based on the Internet of Things according to claim 1, characterized in that: Wherein, in step S5, the following sub-steps are also included: S5-1, the role permission matrix is ​​a two-dimensional permission table. The row dimension defines the administrator, ordinary user, and guest roles, and the column dimension defines the device control, parameter modification, and system configuration operation levels. The permission values ​​are expressed as integer levels from 0 to 3. The permission values ​​include: 0 means prohibited, 1 means read-only, 2 means basic operations, and 3 means full control; S5-2, the steps for implementing remote control of multiple terminals include: S5-2-1, the mobile terminal publishes TLS1.3 encrypted control instructions via the MQTT protocol; S5-2-2, the voice assistant converts the voice into a JSON command through the ASR engine, wherein the JSON command includes the device ID, operation type, parameter value and timestamp; S5-2-3, the web interface uses the WebSocket protocol for two-way communication, and the connection is established through the JWT token with a validity period of ≤ 15 minutes for verification; S5-3, the request to modify the device parameters is forced to verify the dynamic token generated based on the HMAC-SHA256 algorithm, and the validity period of the dynamic token is ≤30 seconds.

Images