Multimodal feedback vr device bluetooth interaction system

Abstract

The application relates to the technical field of virtual reality interaction, and discloses a multi-modal feedback VR device Bluetooth interaction system, which comprises a virtual reality host device, a Bluetooth communication link, a microcontroller unit and a tactile and olfactory feedback execution module. The microcontroller unit receives virtual physical event data by using a ring buffer which is maintained with a read-write pointer, and through an overlap strategy when a write pointer chases a read pointer and a timeout reset mechanism, the real-time performance and stability of instruction processing are ensured. The microcontroller unit runs a segmented control algorithm, maps a wind speed parameter into a pulse width modulation signal, and eliminates a motor starting dead zone. Meanwhile, a hardware timer is used to load a synchronous delay parameter, and a physical response time difference of tactile and olfactory feedback is accurately compensated. The application solves the problems of communication lag, control nonlinearity and sensory timing asynchronization in multi-modal feedback.

Description

Technical Field

[0001] This invention relates to the field of virtual reality interaction technology, specifically to a multimodal feedback Bluetooth interaction system for VR devices. Background Technology

[0002] With the popularization of virtual reality (VR) technology, users' demand for immersive experiences has expanded from single visual feedback to the synergy of multimodal perception, including touch and smell. Existing VR peripherals typically communicate with the host device wirelessly via Bluetooth to control actuators such as fans and scent generators to simulate environmental physical characteristics. However, in practical applications, the hardware and software design of multimodal feedback systems still faces many technical bottlenecks.

[0003] Specifically, existing lower-level systems often employ simple first-in-first-out (FIFO) queues or blocking reception logic when receiving high-frequency interactive data. When the frequency of commands sent by the host exceeds the processing speed of the lower-level machine, buffer backlog can easily occur, causing the system to prioritize processing outdated commands and fail to respond to the latest interactive events in real time. Furthermore, signal fluctuations or bit errors are inevitable in wireless communication environments. Traditional data parsing schemes lack robust fault tolerance mechanisms; once incomplete data frames are received, the parsing program can easily get stuck in a deadlock or waiting state, preventing the device from responding to subsequent commands and affecting the system's robustness.

[0004] Furthermore, commonly used haptic feedback devices such as DC fans exhibit nonlinear physical characteristics, resulting in a dead zone during startup due to static friction. Existing control methods mostly employ simple linear mapping, leading to motors failing to start or producing noticeable vibrations when simulating low-speed scenarios such as gentle breezes, failing to accurately reproduce subtle physical quantities in the virtual environment. Simultaneously, the transmission characteristics of touch (wind sensation) and smell (odor) in physical space are drastically different; there is a significant time difference between the mechanical inertia of the fan motor and the diffusion rate of odor molecules. Existing control schemes typically use synchronous triggering modes, ignoring the differences in response delays between different modes. This results in a discrepancy between the arrival time of the wind sensation and the odor when the user visually perceives the event, creating a sensory disconnect.

[0005] Furthermore, VR interactive peripherals typically pursue miniaturization and high integration, leading to the RF communication module and motor drive circuit often being located on the same printed circuit board. Existing designs sometimes neglect electromagnetic compatibility issues, failing to adequately clear or isolate the RF antenna area, causing electromagnetic noise from the motor to interfere with the Bluetooth signal, resulting in unstable connections or disconnections. Additionally, while directly using pulse-width modulation (PWM) square waves to drive the motor is simple to implement, high-frequency harmonic components can easily induce electromagnetic noise and mechanical vibration within the motor, further reducing the quality of the feedback experience. Summary of the Invention

[0006] To achieve the above objectives, the present invention provides the following technical solution: a multimodal feedback VR device Bluetooth interaction system, comprising: a virtual reality host device, a Bluetooth communication link, a microcontroller unit, a haptic feedback execution module, and an olfactory feedback execution module;

[0007] The microcontroller unit interacts with the virtual reality host device via the Bluetooth communication link. The virtual reality host device is used to run a virtual reality application configured with physical event mapping logic, and to convert the environmental interaction events generated by the virtual reality application into discrete virtual physical event data, and to send the virtual physical event data through the Bluetooth communication link; The virtual physical event data includes wind speed level parameters, odor type parameters, odor concentration parameters, and duration parameters; The Bluetooth communication link is a wireless radio frequency channel established based on the Bluetooth serial port specification, used to transmit the virtual physical event data. The microcontroller unit integrates a Bluetooth baseband and a processor, and has a first general-purpose input / output port and a second general-purpose input / output port, used to receive and parse the virtual physical event data and convert it into hardware driver instructions; The haptic feedback execution module is connected to the first general-purpose input / output port; The microcontroller unit outputs a pulse width modulation signal with a variable duty cycle to the haptic feedback execution module, controlling the haptic feedback execution module to generate simulated physical airflow; The olfactory feedback execution module is connected to the second general-purpose input / output port; the microcontroller unit outputs a switch signal to the olfactory feedback execution module to control the olfactory feedback execution module to release the odor medium.

[0008] Preferably, the haptic feedback execution module includes a signal conditioning circuit, a motor drive circuit, and a DC fan motor; The signal conditioning circuit is connected between the first general-purpose input / output port and the control input terminal of the motor drive circuit. It adopts a first-order RC low-pass filter structure to filter out high-frequency components in the pulse width modulation signal and convert the digital pulse signal into an analog voltage control signal. The motor drive circuit is connected between the signal conditioning circuit and the DC fan motor, and is used to amplify the power of the analog voltage control signal to drive the DC fan motor. The olfactory feedback execution module includes a power drive circuit, a miniature solenoid valve, and an odor storage container; the power drive circuit is connected between the second general-purpose input / output port and the miniature solenoid valve, and a switching circuit is constructed using a metal-oxide-semiconductor field-effect transistor; The miniature solenoid valve is installed at the release channel port of the odor storage container and controls the opening and closing of the release channel in response to the conduction or cutoff action of the power drive circuit.

[0009] Preferably, the microcontroller unit runs a protocol stack firmware based on the Bluetooth serial port specification and is configured with an automatic reconnection mechanism; The microcontroller unit determines the current link status by reading the connection status flag variable; When the connection status flag variable indicates a disconnected state, the microcontroller unit executes a reconnection procedure; The reconnection process employs a fixed time interval delay strategy: the microcontroller unit performs a delay operation of 2.0 seconds before executing the next connection attempt; After the delay operation ends, the microcontroller unit resets the state machine of the Bluetooth protocol stack and executes the device scan and connection request again.

[0010] Preferably, the Bluetooth communication link adopts a serial communication protocol with a hybrid data frame structure, supporting binary command frames and text command frames; The microcontroller unit runs an instruction parsing program, which determines the frame type by reading the first byte of the data packet: when the first byte is a value in the range of 0x00 to 0x0F, it is determined to be the binary instruction frame; When the first byte is the character '{', it is determined to be the text instruction frame; the binary instruction frame adopts a fixed-length structure of command code + parameter + check word + terminator; The check character is located two bytes before the end character and stores a 16-bit cyclic redundancy check code. The microcontroller unit uses the check word to perform integrity verification on the binary instruction frame; The text instruction frame is in JSON format, and the microcontroller unit calls the JSON parsing logic to verify the format validity of the text instruction frame.

[0011] Preferably, the microcontroller unit is configured with a circular buffer in memory and maintains write pointers and read pointers for temporarily storing asynchronous serial data from the Bluetooth communication link; The microcontroller unit enables a receive interrupt, writes the received data into the circular buffer at the address pointed to by the write pointer, and updates the write pointer. If the write pointer value catches up with the read pointer value, the overwrite strategy is executed, and the latest received data overwrites the earliest unprocessed data; The microcontroller unit also maintains a receive timeout mechanism and is configured with a receive timer: If the receive timer times out and no valid end-of-frame character is detected, the microcontroller unit forcibly resets the protocol parsing state machine to the initial state and clears the current parsing buffer.

[0012] Preferably, the microcontroller unit runs a segmented control algorithm program, which is preset with proportional gain coefficients for low-speed sections, proportional gain coefficients for medium-speed sections, proportional gain coefficients for high-speed sections, a start-up compensation constant, a first correction intercept, and a second correction intercept, used to map the wind speed level parameter to the duty cycle of the pulse width modulation signal. The segmented control algorithm program takes the wind speed level parameter as the input variable and divides the input variable into a start-up compensation zone, a linear response zone, and a saturation decay zone: In the start-up compensation zone, the input variable is less than the start-up threshold, and the target duty cycle calculated by the microcontroller unit is equal to the input variable multiplied by the proportional gain coefficient of the low-speed section and the start-up compensation constant. In the linear response region, the input variable is between the start-up threshold and the saturation threshold, and the target duty cycle calculated by the microcontroller unit is equal to the input variable multiplied by the proportional gain coefficient of the medium speed range plus the first correction intercept. The value of the first correction intercept satisfies the continuity condition of the control curve at the start-up threshold; in the saturation decay region, the input variable is greater than the saturation threshold, and the target duty cycle calculated by the microcontroller unit is equal to the input variable multiplied by the proportional gain coefficient of the high-speed section plus the second correction intercept; The proportional gain coefficient of the high-speed section is less than that of the medium-speed section.

[0013] Preferably, the microcontroller unit uses a low-frequency pulse width modulation control method to adjust the odor concentration; The microcontroller unit has a preset mapping table between odor concentration level and duty cycle; When the microcontroller unit receives the odor concentration level parameter from the virtual physical event data, it adjusts the on-time ratio of the olfactory feedback execution module within the control cycle to change the odor release duration per unit time.

[0014] Preferably, the microcontroller unit is equipped with a hardware timer; The text instruction frame encapsulates a composite instruction containing timestamp synchronization information; the microcontroller unit extracts the target pulse width modulation duty cycle, the target olfactory channel identifier, and the synchronization delay parameter from the composite instruction. The microcontroller unit drives the tactile feedback execution module to run according to the target pulse width modulation duty cycle, and determines the triggering timing of the olfactory feedback execution module according to the synchronization delay parameter: when the synchronization delay parameter is equal to 0, the microcontroller unit executes the synchronization triggering logic and directly activates the olfactory feedback execution module corresponding to the target olfactory channel identifier; When the synchronization delay parameter is greater than 0, the microcontroller unit executes asynchronous delay logic, starts the hardware timer, and loads the synchronization delay parameter into the hardware timer; When the hardware timer overflows and triggers an interrupt, the microcontroller unit activates the olfactory feedback execution module corresponding to the target olfactory channel identifier in the interrupt service routine.

[0015] Preferably, a multimodal feedback VR device Bluetooth interaction method, implemented based on the aforementioned multimodal feedback VR device Bluetooth interaction system, includes the following steps: The virtual reality host device runs a virtual reality application and converts the environmental interaction events generated by the virtual reality application into virtual physical event data; The virtual physical event data includes wind speed level parameters, odor type parameters, odor concentration parameters, and duration parameters; The virtual reality host device sends the virtual physical event data to the microcontroller unit via the Bluetooth communication link; The microcontroller unit receives the virtual physical event data and parses it to obtain hardware driver instructions; According to the hardware driving instructions, the microcontroller unit outputs a pulse width modulation signal to the tactile feedback execution module through the first general-purpose input / output port to simulate physical airflow, and outputs a switch signal to the olfactory feedback execution module through the second general-purpose input / output port to release the odor medium.

[0016] Preferably, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multimodal feedback Bluetooth interaction method for VR devices as described above.

[0017] This invention provides a multimodal feedback Bluetooth interaction system for VR devices. It has the following beneficial effects: (1) This invention effectively improves the real-time performance and robustness of Bluetooth communication by configuring a circular buffer with an overlay strategy and a receive timeout reset mechanism in the microcontroller unit. The overlay logic of the write pointer catching up with the read pointer in the circular buffer ensures that the VR interaction system always prioritizes processing the latest physical event data, avoiding feedback delays caused by data backlog; combined with CRC16 check and timeout reset logic, it can automatically filter transmission errors and prevent parsing program deadlock caused by frame breaks, ensuring stable operation of the device under continuous high-frequency instruction streams.

[0018] (2) By setting the start-up compensation constant and linear correction intercept in the control algorithm, the present invention eliminates the dead zone effect of the DC fan motor in the low-speed range and realizes a smooth transition from standstill to high speed. At the same time, by using a hardware timer to load the synchronization delay parameter, the physical time difference between the mechanical inertia of the fan motor acceleration and the transmission delay of the odor gas diffusion is accurately compensated, ensuring the time synchronization of the two physical media of touch and smell at the target point.

[0019] (3) By setting a radio frequency antenna clearance area and a π-type power supply filter network on the printed circuit board, the present invention effectively isolates the electromagnetic interference of high-frequency digital logic circuits to Bluetooth radio frequency signals, thereby improving the communication distance and connection stability. In addition, the first-order RC low-pass filter circuit connected to the PWM output terminal can convert discrete digital pulse signals into continuous and smooth analog voltage signals, thereby reducing electromagnetic noise and mechanical vibration during motor operation. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is a schematic diagram of the system architecture of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figure 1 - Figure 2This invention provides a multimodal feedback VR device Bluetooth interaction system. The multimodal feedback VR device Bluetooth interaction system mainly includes a virtual reality host device, a Bluetooth communication link, a microcontroller unit (CONTROLLER), a haptic feedback execution module, and an olfactory feedback execution module. The microcontroller unit acts as a lower-level receiver and interacts with the virtual reality host device, which acts as an upper-level transmitter, through the Bluetooth communication link.

[0023] The virtual reality host device runs a virtual reality application, which is configured with physical event mapping logic. This logic can convert environmental interaction events in the virtual environment into discrete virtual physical event data. The virtual physical event data is a set of digital signals that include wind speed level parameters, odor type parameters, and duration parameters.

[0024] The Bluetooth communication link is a wireless radio frequency channel established based on the Bluetooth Serial Port Specification (SPP). The Bluetooth communication link connects the virtual reality host device and the microcontroller unit and is used to transmit virtual physical event data.

[0025] The microcontroller unit is the computing and control center of this system. Its core adopts a system-on-a-chip (SoC) that integrates a Bluetooth baseband and a 32-bit processor, such as the ESP32 series chip. The microcontroller unit has a main control module built inside. The main control module reads data through the universal asynchronous receiver / transmitter (UART) interface and uses the built-in event parsing algorithm to solve the virtual physical event data into hardware drive instructions of different priorities.

[0026] The haptic feedback execution module corresponds to the attached Figure 2 The hardware circuit containing the FanModel includes a haptic feedback execution module comprising a pulse width modulation drive circuit and a DC fan motor. The first general purpose input / output port (GPIO) of the main control module is connected to the haptic feedback execution module, outputting a pulse width modulation (PWM) signal with a variable duty cycle to control the speed of the fan motor, thereby simulating physical airflow at different speeds.

[0027] The olfactory feedback execution module corresponds to the attached Figure 2 The hardware circuit containing the SeemModel (odor model) includes an olfactory feedback execution module, which includes an odor storage container and an electronically controlled valve (such as a solenoid valve or a piezoelectric ceramic plate). The second general-purpose input / output port of the main control module is connected to the olfactory feedback execution module, and outputs high and low level switching signals to control the opening and closing of the electronically controlled valve, thereby realizing the quantitative release of a specific odor.

[0028] In terms of hardware connectivity, the microcontroller unit is also connected to a power management circuit and a USB-to-serial communication circuit for program debugging. The main control module coordinates the tactile feedback execution module and the olfactory feedback execution module through an internal hardware timer to form a closed-loop multimodal physical feedback loop.

[0029] Furthermore, this embodiment details the overall workflow of the multimodal feedback VR device Bluetooth interaction system. This workflow is implemented based on the data interaction logic between the microcontroller unit's internal firmware and the virtual reality host device. Specifically: After the system starts up, the microcontroller unit executes the initialization program and enters the automatic identification and connection stage. The microcontroller unit broadcasts a scanning signal through the Bluetooth communication link to obtain the device identification information within the current signal range. The microcontroller unit then matches the obtained device identification information with a preset target address list.

[0030] If it is determined that the device is not connected or the connection establishment failed (i.e.) Figure 1 If the "Is the device connected?" message is incorrect, the microcontroller unit 130 returns to the automatic identification state and performs scanning and connection operations in a loop according to the preset retry cycle until the connection handshake is successful.

[0031] If it is determined that the device is connected (i.e.) Figure 1 (If the "device is connected" condition is met, the microcontroller unit enters the data processing state and waits to receive instructions. At this time, the virtual reality host device generates virtual physical event data according to the virtual scene logic, encapsulates it into a data frame that conforms to the Bluetooth serial communication protocol, and sends it through the Bluetooth communication link.)

[0032] The microcontroller unit receives data frames and performs data parsing. The main control module uses check bits (such as CRC16) to verify the integrity of the data frames.

[0033] If the data validation fails (i.e.) Figure 1 If the "data validity" is determined to be no, the microcontroller unit executes a discard operation. The main control module directly ignores the erroneous data frame and does not generate any electrical signal output to prevent erroneous instructions from causing malfunctions.

[0034] If the data validation passes (i.e.) Figure 1 If the "data validity" is determined to be yes, the microcontroller unit executes the instruction distribution. The main control module generates a pulse width modulation signal for the haptic feedback execution module and a switch control signal for the olfactory feedback execution module based on the specific content of the virtual physical event data. The haptic feedback execution module and the olfactory feedback execution module respond to the above signals to perform physical actions, thereby realizing the feedback reproduction of the virtual environment.

[0035] The microcontroller unit uses the ESP32 series system-on-chip (SoC) as the core processing component. The ESP32 series SoC integrates a 2.4GHz Wi-Fi and Bluetooth dual-mode baseband and a 32-bit microprocessor, and supports a deep sleep low-power operating mode. In deep sleep mode, the quiescent current of the ESP32 series SoC is less than 20 microamps.

[0036] The microcontroller unit also includes a Universal Serial Bus (USB) to Universal Asynchronous Receiver / Transmitter (UART) interface circuit, which uses a CH340C level converter chip. The CH340C level converter chip is connected to the second Universal Asynchronous Receiver / Transmitter interface (UART2) of the ESP32 series on-chip system.

[0037] In terms of hardware connectivity, the 16th general-purpose input / output pin (GPIO16) of the ESP32 series on-chip system is used as the receiver (RXD), and the 17th general-purpose input / output pin (GPIO17) is used as the transmitter (TXD). The data transmission pin of the CH340C level conversion chip is connected to GPIO16, and the data reception pin is connected to GPIO17. This interface circuit is used for firmware burning and serial output of system running data.

[0038] The printed circuit board (PCB) of the microcontroller unit adopts a partitioned layout structure of radio frequency modules and digital logic circuits. The PCB has a clearance area in the area corresponding to the radio frequency antenna of the ESP32 series on-chip system. The physical distance between the edge of the clearance area and the radio frequency antenna is set to be no less than 2 mm. No copper foil conductive layer and signal traces are laid within the clearance area to reduce electromagnetic interference.

[0039] A π-type filter network is connected to the power input pin of the microcontroller unit. The π-type filter network consists of parallel filter capacitors and series inductors. The π-type filter network is used to filter out high-frequency noise signals in the power supply line.

[0040] Furthermore, the haptic feedback execution module is connected to the main control module of the microcontroller unit. The haptic feedback execution module includes a signal conditioning circuit, a motor drive circuit, and a DC fan motor.

[0041] The microcontroller unit is equipped with a pulse width modulation (PWM) controller, which generates a pulse signal with a frequency of 50 Hz and outputs it through the 27th general purpose input / output pin (GPIO27). The microcontroller unit controls the high-level duration of the output signal by adjusting the duty cycle register value of the PWM controller.

[0042] The signal conditioning circuit is connected between the 27th general purpose input / output pin and the control input terminal of the motor drive circuit. The signal conditioning circuit adopts a first-order RC low-pass filter structure, which consists of a series resistor and a parallel capacitor. In this embodiment, the resistance of the resistor is 10 kΩ and the capacitance of the capacitor is 10 μF. This signal conditioning circuit is used to filter out the high-frequency components in the pulse width modulation signal and convert the digital pulse signal into an analog voltage control signal with variable amplitude.

[0043] The motor drive circuit is connected between the signal conditioning circuit and the DC fan motor. The motor drive circuit is used to amplify the analog voltage control signal. The DC fan motor is the actuator, and its operating voltage or operating current is controlled by the output of the motor drive circuit.

[0044] When the microcontroller unit changes the duty cycle of the pulse width modulation signal, the analog voltage control signal output by the signal conditioning circuit changes linearly, thereby controlling the motor drive circuit to adjust the speed of the DC fan motor. The speed change range of the DC fan motor corresponds to the duty cycle change range of the pulse width modulation signal from 0% to 100%, thereby generating physical airflows of different velocities to simulate tactile effects.

[0045] Furthermore, the olfactory feedback execution module corresponds to the attached... Figure 2 The odor control logic unit, identified as "SeemModel", is an olfactory feedback execution module used to execute odor release commands and control the timing and amount of release of specific odor media.

[0046] The olfactory feedback execution module mainly includes a power drive circuit, a miniature solenoid valve, and an odor storage container. The miniature solenoid valve is installed at the release channel port of the odor storage container. As an execution component, the miniature solenoid valve controls the opening and closing of the release channel through the mechanical displacement of the valve core.

[0047] The microcontroller unit is connected to the olfactory feedback execution module through the second general-purpose input / output pin. Since the drive current of the micro solenoid valve is usually greater than the output capability of the microcontroller unit pin, the power drive circuit is connected between the second general-purpose input / output pin and the control terminal of the micro solenoid valve. In this embodiment, the power drive circuit uses an N-channel enhancement-mode metal-oxide-semiconductor field-effect transistor (MOSFET) to construct the switching circuit.

[0048] The microcontroller unit controls the power drive circuit to turn on and off by outputting high and low level signals. When the second general-purpose input / output pin outputs a high-level valid signal, the power drive circuit is turned on, the coil of the miniature solenoid valve is energized and attracted, the release channel is opened, and the odor medium diffuses to the outside. When a low-level signal is output, the power drive circuit is turned off, the miniature solenoid valve is de-energized and reset, and the release channel is closed.

[0049] For odor concentration adjustment, the microcontroller unit adopts low-frequency pulse width modulation (PWM) control. The microcontroller unit has a preset mapping table between odor concentration level and duty cycle. When it receives the odor concentration level parameter (e.g., level 1 to 8), the microcontroller unit adjusts the opening time ratio of the micro solenoid valve within the control cycle according to the parameter. By changing the odor release duration per unit time, quantitative control of the ambient odor concentration is achieved.

[0050] Furthermore, the microcontroller unit runs a protocol stack firmware based on the Bluetooth Serial Port Specification (SPP). After the system is powered on and reset, the microcontroller unit initializes the Bluetooth controller and registers Bluetooth event callback functions. The microcontroller unit then enters MasterScanMode and periodically receives Bluetooth broadcast packets from the surrounding environment.

[0051] The microcontroller unit has a pre-installed target device identifier list in its non-volatile memory. This list stores the media access control (MAC) address or Bluetooth device name of the virtual reality host device. The microcontroller unit compares the source address field or name field in the scanned broadcast packet with the target device identifier list.

[0052] When the comparison results match, the microcontroller unit stops scanning and initiates a connection request to the virtual reality host device, attempting to establish a Logical Link Control and Adaptation Protocol (L2CAP) link and serial port communication channel.

[0053] To address connection establishment failures or unexpected link interruptions during operation, the microcontroller unit implements an automatic reconnection mechanism in the firmware main loop, such as... Figure 1 As shown in the negative branch of the "Is the device connected?" determination node, the microcontroller unit determines the current link status by reading the connection status flag variable. When the connection status flag variable indicates a disconnected state, the microcontroller unit triggers the reconnection process.

[0054] The reconnection process employs a fixed time interval delay strategy. Before executing the next connection attempt, the microcontroller unit performs a delay operation with a duration of 2.0 seconds. This delay operation is implemented through a software delay function or a hardware timer interrupt. After the delay ends, the microcontroller unit resets the state machine of the Bluetooth protocol stack and executes the device scan and connection request again.

[0055] The fixed reconnection interval of 2.0 seconds is designed to match the state switching timing of the Bluetooth protocol stack. This time interval ensures that the Bluetooth controller initiates a new radio frequency operation only after completing the cleanup work of the previous connection request, preventing protocol stack congestion or instruction execution errors caused by high-frequency continuous calls to the connection interface.

[0056] Once the Bluetooth communication link is successfully established, the Bluetooth protocol stack triggers a connection success event callback. The microcontroller unit updates the connection status flag variable to the connected state in the callback function. At this time, the system state switches from the connection establishment stage to the data pass-through stage. The microcontroller unit keeps the radio frequency receiving circuit on to receive virtual physical event data sent by the virtual reality host device in real time.

[0057] The Bluetooth communication link between the virtual reality host device and the microcontroller unit adopts a serial communication protocol with a hybrid data frame structure, which supports both binary instruction frames and text instruction frames.

[0058] The binary instruction frame adopts a fixed-length structure of "command code + parameters + checksum + end character", where: Command code: A single-byte hexadecimal number used to identify the function of the instruction; Parameters: These are fixed-length data used to transmit control values; Checksum: This is a two-byte data string used to store the CRC16 checksum. End character: uniformly set to 0x0D (carriage return character), used to mark the end of the frame.

[0059] The microcontroller unit's main control module runs an instruction parsing program. The instruction parsing program determines the frame type by reading the first byte of the data packet. When the first byte is a value in the range of 0x00 to 0x0F, it is determined to be a binary instruction frame; when the first byte is the character '{', it is determined to be a text instruction frame.

[0060] The communication protocol divides instructions into real-time control instructions (high priority) and configuration instructions (low priority) based on their execution timeliness.

[0061] Real-time control commands use a binary command frame format and mainly include haptic control and status query. Tactile intensity control command: The command code is set to 0x03, and the parameter is a single-byte integer (0-100), corresponding to the pulse width modulation duty cycle of the fan motor, for example: The instruction sequence [0x03,0x32,CRC_L,CRC_H,0x0D] indicates that the fan speed is set to 50%.

[0062] Status query command: The command code is set to 0x05, with no parameter bytes. After receiving this command, the microcontroller unit immediately returns the current operating status data of the device.

[0063] The configuration instructions use a binary instruction frame format and are primarily used for olfactory module control.

[0064] Olfactory switch command: The command code is set to 0x01. The parameters include the channel number and the switch bit. For example, the command sequence [0x01,0x01,CRC_L,CRC_H,0x0D] means to open odor channel 1.

[0065] Olfactory concentration adjustment command: Command code is set to 0x04, parameter is concentration level (1-8).

[0066] For multimodal collaborative scenarios, the protocol supports text command frames based on JSON format. This format is used to encapsulate composite commands containing timestamp synchronization information. For example, the data packet content is a string of {"t":1,"o":2,"d":50}, where the field "t" represents the tactile channel, "o" represents the olfactory channel, and "d" represents the synchronization delay time (milliseconds). The JSON parsing library (such as cJSON) integrated inside the microcontroller unit extracts the above fields and sets the hardware timer accordingly to achieve millisecond-level timing synchronization between tactile feedback and olfactory feedback.

[0067] Furthermore, the microcontroller unit's main control module runs a segmented control algorithm program. This algorithm program is used to establish a mapping relationship between physical quantities in the virtual environment and the output quantities of real physical actuators. The main control module reads input variables from the virtual reality host device. ,in, This represents the wind speed value in the virtual scene, measured in meters per second (m / s). The main control module performs floating-point operations using the arithmetic logic unit to calculate the output variable. ,in, This represents the duty cycle percentage of the target pulse width modulation signal, ranging from 0 to 100.

[0068] The main control module uses the physical characteristic curve of the DC fan motor to input variables. The domain is divided into three control intervals, and the algorithm presets two critical speed thresholds: the start-up threshold and the start-up threshold. With saturation threshold In this embodiment, Set to 3 meters per second, Set to 10 meters per second.

[0069] First Interval: Compensation Zone Activation When the input variables satisfy At that time, the algorithm performs the start-up compensation operation, and the operation formula is as follows:

[0070] In the formula: This represents the proportional gain coefficient in the low-speed range, used to set the response sensitivity at low wind speeds. This represents the start-up compensation constant (intercept). The numerical setting must correspond to the minimum starting voltage of the DC fan motor to ensure that when When greater than 0, the output is It can generate a driving force sufficient to overcome the static friction torque of the motor rotor, preventing the motor from being in a locked state while energized.

[0071] Second interval: Linear response region: When the input variables satisfy At that time, the algorithm performs a linear scaling operation, and the calculation formula is as follows:

[0072] In the formula: This represents the proportional gain coefficient in the medium-speed range, corresponding to the interval where the motor speed and voltage have the best linear relationship; This represents the first correction intercept, used to ensure the control curve is within the specified range. To maintain continuity and prevent abrupt changes in the output signal, The value of must satisfy the continuity condition: .

[0073] Third interval: Saturation decay region: When the input variables satisfy At that time, the algorithm performs saturation decay calculation, and the calculation formula is as follows:

[0074] In the formula: This represents the proportional gain coefficient for the high-speed section. Taking into account the decreased tactile sensitivity of human skin to high-velocity airflow and the physical limits of motor speed, the coefficient is set to satisfy... The decreasing relationship; This represents the second corrected intercept. Similarly, to ensure the control curve is within... Continuity at the point, The value of must meet the following conditions: .

[0075] Hardware quantization mapping: The main control module calculates the floating-point duty cycle percentage. Then, it is converted into an integer value suitable for hardware registers. The pulse width modulation controller of the microcontroller unit is configured as follows: In this embodiment, the bit resolution is... The value is set to 10, which means a resolution of 1024 levels. The quantization conversion formula is as follows:

[0076] In the formula: This represents the integer control value written to the pulse width modulation comparator register, with a value range of 0 to 1023; This represents the integer division function; Indicates the bit width resolution of the hardware timer; This indicates the maximum value of the hardware timer counter (i.e., 1023).

[0077] The main control module will calculate the result Write to the comparator register corresponding to the 27th general-purpose input / output pin to control the high-level duration of the output waveform, thereby driving the haptic feedback execution module to generate a continuous and smoothly changing physical airflow.

[0078] Furthermore, the microcontroller unit's main control module runs instruction parsing logic, which is used to process composite control instructions containing timing constraints. The main control module reads and identifies virtual physical event data from the receive buffer. When the main control module parses a composite instruction frame containing multimodal parameters, it starts the synchronous cooperative processing program.

[0079] The main control module extracts three key control parameters from the composite command frame: target pulse width modulation duty cycle. Target olfactory channel identifier and synchronization delay parameters .

[0080] The parameters are defined as follows: : Indicates the target output intensity of the haptic feedback execution module, with a value range of 0 to 100, and its physical meaning is consistent with the pulse width modulation duty cycle defined in the previous embodiment; : Indicates the specific odor channel number to be activated in the olfactory feedback execution module, corresponding to a specific general-purpose input / output pin; : Represents the timing offset in milliseconds (ms). This parameter defines the lag trigger time of the olfactory feedback action relative to the tactile feedback action.

[0081] The main control module first executes the haptic feedback action, based on... The numerical update of the pulse width modulation controller's comparison register immediately drives the haptic feedback execution module to run. Subsequently, the main control module updates the synchronization delay parameter. Numerical judgments are made to determine when to trigger olfactory feedback.

[0082] when When the value equals 0, the main control module executes the synchronization trigger logic. Within the current instruction cycle, the main control module directly sends signals to the corresponding device. Write the enable level signal to the corresponding general-purpose input / output pin.

[0083] when When the value is greater than 0, the main control module executes asynchronous delay logic, and the main control module identifies the target olfactory channel. Write global state variables to firmware During temporary storage, the main control module starts a hardware timer, and based on the microcontroller unit's clock frequency parameters, the main control module sets the time parameters... Converted to timer clock count value and will Write to the overload register of the hardware timer to enable timer interrupts.

[0084] When the hardware timer's count reaches zero, the microcontroller unit triggers the timer interrupt service routine (ISR), which reads the global status variables. The system temporarily stores the channel identifier data and addresses the corresponding hardware pin accordingly. It outputs a high-level enable signal to activate the olfactory feedback execution module. After the interrupt service routine is completed, it automatically shuts down the hardware timer and clears the interrupt flag.

[0085] Through the above mechanism, the main control module can precisely control the physical start-up time difference between the tactile feedback execution module and the olfactory feedback execution module. This mechanism is used to compensate for the mechanical inertia delay caused by the acceleration of the fan motor and the transmission delay caused by the diffusion of odor gas, ensuring that the two physical media, wind and odor, arrive at the target spatial point synchronously in time.

[0086] Furthermore, the microcontroller unit is configured with a ring buffer in memory. The ring buffer is used to receive and temporarily store asynchronous serial data from the Bluetooth communication link. The microcontroller unit enables the receive interrupt of the general asynchronous transceiver interface. When byte data is received, the interrupt service routine writes the data to the address pointed to by the write pointer of the ring buffer and makes the write pointer automatically increment. If the write pointer value catches up with the read pointer value, the interrupt service routine executes the overwrite strategy, overwriting the oldest unprocessed data with the latest received data, so as to ensure that the system always processes the latest interactive instructions.

[0087] The main control module performs data parsing tasks in the main program loop. The main control module determines whether there is data to be processed in the buffer by comparing the values ​​of the write pointer and the read pointer. When there is data to be processed, the main control module reads the data stream byte by byte and uses the state machine to retrieve the end-of-frame character (0x0D).

[0088] When the state machine detects the end-of-frame marker, the main control module extracts the byte sequence between the current end-of-frame marker and the previous end-of-frame marker to form a data frame to be verified. According to the communication protocol of this embodiment, the structure of the data frame to be verified is modified to "command code + parameter + check word + end marker". The check word is located two bytes before the end marker and stores a 16-bit cyclic redundancy check code (CRC16) calculated and generated by the virtual reality host device.

[0089] The main control module extracts the command code and parameter fields from the data frame to be verified, using a polynomial algorithm consistent with that of the sending end (e.g., ...). The local checksum is calculated, and the main control module compares the local checksum with the checksum in the data frame.

[0090] If the two match, the data frame is deemed valid, and the main control module executes the corresponding hardware control operation based on the command code.

[0091] If the two are inconsistent, the data frame is determined to have a bit error. Given the real-time requirements of virtual reality feedback, the main control module executes a discard strategy. The main control module directly clears the current data frame without performing any action or sending a retransmission request to the host, in order to avoid the accumulation of signal delay caused by the retransmission mechanism.

[0092] To address the deadlock issue caused by data transmission interruptions, the main control module introduces a receive timeout mechanism. The main control module maintains a receive timer, which is reset whenever new byte data is received in the circular buffer. If the timer expires (e.g., more than 100 milliseconds) and no valid end-of-frame character is detected, the main control module determines that the current frame reception is incomplete. At this time, the main control module forcibly resets the protocol parsing state machine to the initial state and clears the current parsing buffer, waiting for the arrival of the next frame data.

[0093] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal feedback Bluetooth interaction system for VR devices, characterized in that, include: Virtual reality host device, Bluetooth communication link, microcontroller unit, haptic feedback execution module and olfactory feedback execution module; The microcontroller unit interacts with the virtual reality host device via the Bluetooth communication link. The virtual reality host device is used to run a virtual reality application configured with physical event mapping logic, convert the environmental interaction events generated by the virtual reality application into discrete virtual physical event data, and send the virtual physical event data through the Bluetooth communication link; The virtual physical event data includes wind speed level parameters, odor type parameters, odor concentration parameters, and duration parameters; The Bluetooth communication link is a wireless radio frequency channel established based on the Bluetooth serial port specification, used to transmit the virtual physical event data. The microcontroller unit integrates a Bluetooth baseband and a processor, and has a first general-purpose input / output port and a second general-purpose input / output port, used to receive and parse the virtual physical event data and convert it into hardware driver instructions; The haptic feedback execution module is connected to the first general-purpose input / output port; The microcontroller unit outputs a pulse width modulation signal with a variable duty cycle to the haptic feedback execution module, controlling the haptic feedback execution module to generate simulated physical airflow; The olfactory feedback execution module is connected to the second general-purpose input / output port; the microcontroller unit outputs a switch signal to the olfactory feedback execution module to control the olfactory feedback execution module to release the odor medium.

2. The multimodal feedback Bluetooth interaction system for VR devices according to claim 1, characterized in that: The tactile feedback execution module includes a signal conditioning circuit, a motor drive circuit, and a DC fan motor; The signal conditioning circuit is connected between the first general-purpose input / output port and the control input terminal of the motor drive circuit. It adopts a first-order RC low-pass filter structure to filter out high-frequency components in the pulse width modulation signal and convert the digital pulse signal into an analog voltage control signal. The motor drive circuit is connected between the signal conditioning circuit and the DC fan motor, and is used to amplify the power of the analog voltage control signal to drive the DC fan motor. The olfactory feedback execution module includes a power drive circuit, a miniature solenoid valve, and an odor storage container; the power drive circuit is connected between the second general-purpose input / output port and the miniature solenoid valve, and a switching circuit is constructed using a metal-oxide-semiconductor field-effect transistor; The miniature solenoid valve is installed at the release channel port of the odor storage container and controls the opening and closing of the release channel in response to the conduction or cutoff action of the power drive circuit.

3. The multimodal feedback Bluetooth interaction system for VR devices according to claim 1, characterized in that: The microcontroller unit runs a protocol stack firmware based on the Bluetooth serial port specification and is configured with an automatic reconnection mechanism. The microcontroller unit determines the current link status by reading the connection status flag variable; When the connection status flag variable indicates a disconnected state, the microcontroller unit executes a reconnection procedure; The reconnection process employs a fixed time interval delay strategy: the microcontroller unit performs a delay operation of 2.0 seconds before executing the next connection attempt; After the delay operation ends, the microcontroller unit resets the state machine of the Bluetooth protocol stack and executes the device scan and connection request again.

4. The multimodal feedback Bluetooth interaction system for VR devices according to claim 1, characterized in that: The Bluetooth communication link adopts a serial communication protocol with a hybrid data frame structure, supporting binary command frames and text command frames. The microcontroller unit runs an instruction parsing program, which determines the frame type by reading the first byte of the data packet: when the first byte is a value in the range of 0x00 to 0x0F, it is determined to be the binary instruction frame; When the first byte is the character '{', it is determined to be the text instruction frame; the binary instruction frame adopts a fixed-length structure of command code + parameter + check word + terminator; The check character is located two bytes before the end character and stores a 16-bit cyclic redundancy check code. The microcontroller unit uses the check word to perform integrity verification on the binary instruction frame; The text instruction frame is in JSON format, and the microcontroller unit calls the JSON parsing logic to verify the format validity of the text instruction frame.

5. The multimodal feedback Bluetooth interaction system for VR devices according to claim 4, characterized in that: The microcontroller unit is configured with a circular buffer in memory and maintains write and read pointers for temporarily storing asynchronous serial data from the Bluetooth communication link. The microcontroller unit enables a receive interrupt, writes the received data into the circular buffer at the address pointed to by the write pointer, and updates the write pointer. If the write pointer value catches up with the read pointer value, the overwrite strategy is executed, and the latest received data overwrites the earliest unprocessed data; The microcontroller unit also maintains a receive timeout mechanism and is configured with a receive timer: If the receive timer times out and no valid end-of-frame character is detected, the microcontroller unit forcibly resets the protocol parsing state machine to the initial state and clears the current parsing buffer.

6. The multimodal feedback Bluetooth interaction system for VR devices according to claim 1, characterized in that: The microcontroller unit runs a segmented control algorithm program, which is preset with proportional gain coefficients for low-speed sections, medium-speed sections, and high-speed sections, a start-up compensation constant, a first correction intercept, and a second correction intercept, used to map the wind speed level parameter to the duty cycle of the pulse width modulation signal. The segmented control algorithm program takes the wind speed level parameter as the input variable and divides the input variable into a start-up compensation zone, a linear response zone, and a saturation decay zone: In the start-up compensation zone, the input variable is less than the start-up threshold, and the target duty cycle calculated by the microcontroller unit is equal to the input variable multiplied by the proportional gain coefficient of the low-speed section and the start-up compensation constant. In the linear response region, the input variable is between the start-up threshold and the saturation threshold, and the target duty cycle calculated by the microcontroller unit is equal to the input variable multiplied by the proportional gain coefficient of the medium speed range plus the first correction intercept. The value of the first corrected intercept satisfies the continuity condition of the control curve at the start-up threshold; In the saturation decay region, if the input variable is greater than the saturation threshold, the target duty cycle calculated by the microcontroller unit is equal to the input variable multiplied by the proportional gain coefficient of the high-speed section plus the second correction intercept. The proportional gain coefficient of the high-speed section is less than that of the medium-speed section.

7. The multimodal feedback Bluetooth interaction system for VR devices according to claim 1, characterized in that: The microcontroller unit uses a low-frequency pulse width modulation control method to adjust the odor concentration; The microcontroller unit has a preset mapping table between odor concentration level and duty cycle; When the microcontroller unit receives the odor concentration level parameter from the virtual physical event data, it adjusts the on-time ratio of the olfactory feedback execution module within the control cycle to change the odor release duration per unit time.

8. The multimodal feedback Bluetooth interaction system for VR devices according to claim 4, characterized in that: The microcontroller unit is equipped with a hardware timer; The text instruction frame encapsulates a composite instruction containing timestamp synchronization information; the microcontroller unit extracts the target pulse width modulation duty cycle, the target olfactory channel identifier, and the synchronization delay parameter from the composite instruction. The microcontroller unit drives the tactile feedback execution module to run according to the target pulse width modulation duty cycle, and determines the triggering timing of the olfactory feedback execution module according to the synchronization delay parameter: when the synchronization delay parameter is equal to 0, the microcontroller unit executes the synchronization triggering logic and directly activates the olfactory feedback execution module corresponding to the target olfactory channel identifier; When the synchronization delay parameter is greater than 0, the microcontroller unit executes asynchronous delay logic, starts the hardware timer, and loads the synchronization delay parameter into the hardware timer; When the hardware timer overflows and triggers an interrupt, the microcontroller unit activates the olfactory feedback execution module corresponding to the target olfactory channel identifier in the interrupt service routine.

9. A multimodal feedback Bluetooth interaction method for VR devices, characterized in that, The implementation of the VR device Bluetooth interaction system based on the multimodal feedback as described in claim 1 includes the following steps: The virtual reality host device runs a virtual reality application and converts the environmental interaction events generated by the virtual reality application into virtual physical event data; The virtual physical event data includes wind speed level parameters, odor type parameters, odor concentration parameters, and duration parameters; The virtual reality host device sends the virtual physical event data to the microcontroller unit via the Bluetooth communication link; The microcontroller unit receives the virtual physical event data and parses it to obtain hardware driver instructions; According to the hardware driving instructions, the microcontroller unit outputs a pulse width modulation signal to the tactile feedback execution module through the first general-purpose input / output port to simulate physical airflow, and outputs a switch signal to the olfactory feedback execution module through the second general-purpose input / output port to release the odor medium.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multimodal feedback Bluetooth interaction method for VR devices as described in claim 9.

Images