Intelligent equipment data interaction method for realizing web and hardware information synchronization

Abstract

The invention discloses an intelligent equipment data interaction method for realizing web and hardware information synchronization, which comprises a desktop application end, the desktop application end comprises a system audio interface, an audio capture module is used for receiving audio of external equipment, the audio passes through an FFT feature analysis module and a zstd data compression module in sequence, and a data compression module is used for compressing data; the data are respectively transmitted to the WebSocket sending module and the local HID control module through a channel I and a channel II; the webpage control end comprises a WebSocket receiving module for decompressing the audio data and transmitting the decompressed audio data to the advanced effect engine to realize generation of an enhanced instruction; the peripheral end comprises output and input hardware, and the output and input hardware is used for state feedback of the WebSocket receiving module; compared with the prior art, the method has the advantages that a two-way communication channel between the desktop application and the webpage application is established, and the limitation of a browser sandbox is broken through; a low-delay audio feature extraction algorithm is designed, the delay is less than 50 ms, a dynamic light effect configuration protocol standard is created, and a cross-platform (Windows / macOS / Linux) unified control frame is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of computer human-computer interaction, and specifically to an intelligent device data interaction method for realizing web and hardware information synchronization. Background Art

[0002] Intelligent device data interaction refers to the ability of different devices to exchange and communicate data. In today's digital age, various intelligent devices, Internet of Things devices, computer systems, etc. need to cooperate with each other and share information, which involves intelligent device data interaction.

[0003] In the prior art, the methods for realizing intelligent device data interaction are as follows: First, pure hardware solutions such as Razer Chroma rely on dedicated drivers, and have problems such as poor device compatibility and high update and maintenance costs; Second, the Web Audio API solution based on the browser is limited by the browser security policy and cannot directly access the system-level audio stream; Third, local application programs (such as Wallpaper Engine) lack cross-platform cooperation capabilities and cannot realize dynamic configuration on the web page side.

[0004] And the prior art also has the following disadvantages: 1. The browser sandbox mechanism restriction causes the inability to directly obtain system audio data; 2. Existing cross-platform solutions have data synchronization delays (>200ms); 3. The hardware solution has too high a coupling degree with the software environment; 4. Lack of an extensible lighting effect configuration interface.

[0005] The information disclosed in this background art section is only intended to enhance the overall understanding of the present invention and should not be regarded as an admission or any form of implication that this information constitutes the prior art already known to those of ordinary skill in the art. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the above technical defects, and provide an intelligent device data interaction method for realizing web and hardware information synchronization. By establishing a two-way communication channel between the desktop application and the web application, the browser sandbox restriction is broken through; a low-latency audio feature extraction algorithm (<50ms delay) is designed; a dynamic lighting effect configuration protocol standard is created; and a cross-platform (Windows / macOS / Linux) unified control framework is realized.

[0007] To solve the above problems, the technical solution of the present invention is an intelligent device data interaction method for realizing web and hardware information synchronization, including: Desktop application side, the desktop application side includes a system audio interface, an audio capture module, an FFT feature analysis module, and a zstd data compression module. The system audio interface receives and connects to an external device, and the audio capture module receives the audio of the external device. The audio passes through the FFT feature analysis module and the zstd data compression module in sequence, and is transmitted to the WebSocket sending module through Channel 1 and to the local HID control module through Channel 2 respectively; Web control side, the web control side includes a WebSocket receiving module. The WebSocket sending module sends the processed audio to the WebSocket receiving module, decompresses the audio data, and conveys it to the advanced effect engine to generate enhancement instructions; Peripheral side, the peripheral side includes input / output hardware. The enhancement instruction production is connected to the input / output hardware through the USB HID protocol. The local HID control module is directly connected to and controls the input / output hardware. The input / output hardware is used for the status feedback of the WebSocket receiving module.

[0008] Furthermore, it also includes a protocol stack. The protocol stack includes an application layer, a transport layer, a data compression layer, a network transport layer, and a physical layer in sequence. The dual-channel WebSocket protocol is used to connect the application layer and the transport layer. The transport layer transfers data to the data compression layer through a control channel or a data stream channel. The compressed data layer transfers data to the network transport layer through the zstd compression algorithm. The network transport layer transfers data to the physical layer through the TCP / IP protocol stack.

[0009] Furthermore, the physical layer is the bottom layer, and the WASAPI / ASIO audio capture module implemented in C++ is adopted; the network layer is the middle layer, and the FFT feature extraction and data compression module is implemented by the zstd algorithm. The transport layer is the dual-channel WebSocket protocol, including a control channel and a data stream channel.

[0010] Furthermore, in the desktop application side, Channel 1 is for feature data, and Channel 2 is for basic instructions.

[0011] Furthermore, in the peripheral side, the input / output hardware includes keyboard lighting.

[0012] Furthermore, the desktop application starts WASAPI loopback capture, calculates the audio spectrum features in real time (256-band analysis), pushes the compressed feature data packets through WebSocket (one data frame every 20 ms). The web side parses the data and generates RGB control instructions to control the device lighting through the USB HID protocol.

[0013] Further, when the system starts the service, if the connection is successful during operation, and an exception is detected, a reconnection is performed. After the reconnection is successfully restored, it is normally closed; when the retry limit is exceeded, manual intervention or forced termination is performed. Forced termination means shutting down the system. After manual intervention, maintenance is carried out, and after the maintenance is completed, the running connection is resumed. The advantages of the present invention compared with the existing technologies are as follows: 1. The present invention establishes a two-way communication channel between the desktop application and the web application, breaking through the browser sandbox limitation; designs a low-latency audio feature extraction algorithm with a latency of less than 50 ms, creates a dynamic lighting effect configuration protocol standard, and realizes a unified control box across platforms (Windows / macOS / Linux).

[0014] 2. The present invention includes a hierarchical audio processing architecture and a dynamic load balancing mechanism, automatically switches the data sampling rate according to the network latency, and realizes an audio-light synchronization error of <3 ms for the web-side rendering engine through an intelligent caching strategy.

[0015] 3. Compared with the traditional hardware solution, the present invention reduces the development cost by 80%, reduces the data latency by 5 times compared with the existing cross-platform solution, supports multi-device cascaded control. Through actual test verification: the system resource occupancy rate is reduced to 1 / 3 of the traditional solution (CPU occupancy <5%), the passing rate of the cross-platform compatibility test is 100%, the lighting response latency reaches the professional e-sports level standard, and supports dynamic lighting strategy hot update. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is the system architecture block diagram of the present invention.

[0017] Figure 2 is the communication protocol stack hierarchy diagram of the present invention.

[0018] Figure 3 is the timing flowchart of the present invention.

[0019] Figure 4 Abnormal handling state machine diagram. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] In order to make the content of the present invention more clearly understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0021] Such as Figures 1-4As shown in the figure, a method for intelligent device data interaction to achieve web and hardware information synchronization, a desktop application end, the desktop application end includes a system audio interface, an audio capture module, an FFT feature analysis module, and a zstd data compression module. The system audio interface receives and connects to an external device, and receives the audio of the external device through the audio capture module. The audio passes through the FFT feature analysis module and the zstd data compression module in sequence, and is transmitted to the WebSocket sending module through channel one and to the local HID control module through channel two respectively; the channel one is feature data, and the channel two is a basic instruction.

[0022] A web control end, the web control end includes a WebSocket receiving module. The WebSocket sending module sends the processed audio to the WebSocket receiving module, decompresses the audio data, and conveys it to the advanced effect engine to achieve enhanced instruction generation; A peripheral end, the peripheral end includes input / output hardware, and the input / output hardware includes keyboard lighting. The enhanced instruction production is connected to the input / output hardware through the USB HID protocol. The local HID control module is directly connected to and controls the input / output hardware, and the input / output hardware is used for the status feedback of the WebSocket receiving module.

[0023] It also includes a protocol stack. The protocol stack includes an application layer, a transport layer, a data compression layer, a network transport layer, and a physical layer in sequence. The dual-channel WebSocket protocol is used to connect the application layer and the transport layer. The transport layer transfers to the data compression layer through a control channel or a data stream channel. The compressed data layer transfers to the network transport layer through the zstd compression algorithm. The network transport layer transfers to the physical layer through the TCP / IP protocol stack. The physical layer is the bottom layer and is a WASAPI / ASIO audio capture module implemented in C++; the network layer is the middle layer and implements the FFT feature extraction and data compression module using the zstd algorithm. The transport layer is the dual-channel WebSocket protocol, including a control channel and a data stream channel.

[0024] The server controls the keyboard lighting through HID instructions, and the keyboard lighting confirms the response. The browser transmits signals to the server through spectrum data, and the server gives a status feedback to the browser. When disconnected, the connection between the browser and the server is lost, and a reconnection request is made. The server restores the status of the controller.

[0025] When the system starts the service, the running connection is successful. After detecting an abnormality, a reconnection is made. When the reconnection is successfully restored, it is normally closed; when the retry limit is exceeded, manual intervention or forced termination is performed. Forced termination means shutting down the system. After manual intervention, maintenance is carried out, and after the maintenance is completed, it returns to the running connection.

[0026] Example 1: Basic implementation framework. The desktop application starts WASAPI loopback capture, calculates the audio spectrum features in real time (256-band analysis), and pushes the compressed feature data packets through WebSocket (one data frame every 20 ms). The web page parses the data and generates RGB control instructions to control the device lights through the USB HID protocol.

[0027] Example 2: Advanced application scenarios Game scenario: Achieve millisecond-level synchronization of explosion special effects and light vibrations Music visualization: Support custom frequency band mapping rules Office assistance: Trigger preset light modes according to system notification types Intelligent interaction scenario between USB and keyboard: Achieve two-way data communication between the keyboard hardware and the host, support composite functions such as real-time feedback of device status, cloud synchronization of macro keys, and cross-device configuration migration, breaking through the limitations of traditional one-way keyboard input.

[0028] Wireless integration scenario: Build a cross-protocol collaboration system between the keyboard and wireless headphones / smart watches to achieve intelligent linkage between physical input devices, biosensors, and spatial audio.

[0029] The above describes the present invention and its implementation manners, and this description is not restrictive. Generally speaking, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments without creative efforts without departing from the spirit of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. An intelligent device data interaction method for realizing web and hardware information synchronization, characterized in that Including: Desktop application terminal, the desktop application terminal includes a system audio interface, an audio capture module, an FFT feature analysis module, and a zstd data compression module. The system audio interface receives and connects to an external device, and the audio capture module receives the audio of the external device. The audio passes through the FFT feature analysis module and the zstd data compression module in sequence, and is transmitted to the WebSocket sending module through channel one and to the local HID control module through channel two respectively; Web control terminal, the web control terminal includes a WebSocket receiving module. The WebSocket sending module sends the processed audio to the WebSocket receiving module, decompresses the audio data, and conveys it to the advanced effect engine to realize enhanced instruction generation; Peripheral device terminal, the peripheral device terminal includes input / output hardware. The enhanced instruction production is connected to the input / output hardware through the USB HID protocol. The local HID control module is directly connected to and controls the input / output hardware, and the input / output hardware is used for the status feedback of the WebSocket receiving module.

2. The intelligent device data interaction method for realizing web and hardware information synchronization according to claim 1, characterized in that: It also includes a protocol stack. The protocol stack sequentially includes an application layer, a transport layer, a data compression layer, a network transport layer, and a physical layer. The dual-channel WebSocket protocol is used to connect the application layer and the transport layer. The transport layer transfers data to the data compression layer through a control channel or a data stream channel. The compressed data layer transfers data to the network transport layer through the zstd compression algorithm. The network transport layer transfers data to the physical layer through the TCP / IP protocol stack.

3. A method for data interaction of an intelligent device for synchronizing web and hardware information according to claim 2, characterized in that: The physical layer is the bottom layer, and the WASAPI / ASIO audio capture module implemented in C++ is used; the network layer is the middle layer, and the FFT feature extraction and data compression module is implemented using the zstd algorithm. The transport layer is the dual-channel WebSocket protocol, including a control channel and a data stream channel.

4. A method for intelligent device data interaction to achieve web and hardware information synchronization according to claim 1, characterized in that: In the desktop application terminal, channel one is feature data, and channel two is basic instructions.

5. A method for intelligent device data interaction to achieve web and hardware information synchronization according to claim 1, characterized in that: In the peripheral device terminal, the input / output hardware includes keyboard lighting.

6. A method for intelligent device data interaction to achieve web and hardware information synchronization according to claim 1, characterized in that: The desktop application starts WASAPI loopback capture, calculates the audio spectrum features in real time (256-band analysis), and pushes the compressed feature data packets through WebSocket (one data frame every 20ms). The web side parses the data and generates RGB control instructions to control the device lighting through the USB HID protocol.

7. A method for intelligent device data interaction to achieve web and hardware information synchronization according to claim 1, characterized in that: When the system starts the service, if the connection is successful during operation, reconnect after detecting an anomaly. When the reconnection is successful, it shuts down normally; when the retry limit is exceeded, manual intervention or forced termination is performed. Forced termination means shutting down the system. After manual intervention, maintenance is carried out, and after the maintenance is completed, it returns to the running connection.

Images