Synchronous measurement system and method for human-computer interaction multi-parameter response time of intelligent household electrical appliance

Abstract

The invention discloses an intelligent household electrical appliance man-machine interaction multi-parameter response time-oriented synchronous measurement system, which comprises a clock synchronization basic module, a central control unit and data processing module, a trigger simulation module, an environment simulation module and a data acquisition module, and is characterized in that the clock synchronization basic module is used as a time reference in the whole measurement system; the central control unit and the data processing module are responsible for coordinating the operation of the measurement system; the trigger simulation module is used for automatically sending a multi-mode interaction trigger signal to the tested household appliance; the environment simulation module is used for simulating a multi-dimensional interference condition of a real use environment of the tested household appliance; the data acquisition module is used for detecting a multi-dimensional response signal and stamping a timestamp on each response event of the detected household appliance; according to the invention, the response time difference from the trigger event to the household appliance response execution can be accurately captured, and the objectivity and repeatability of the test result are improved.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction testing for smart home appliances, specifically a synchronous measurement system and method for multi-parameter response time of human-computer interaction in smart home appliances. Background Technology

[0002] Current human-computer interaction performance testing for smart home appliances typically focuses on testing a specific interaction mode, such as touch, voice, or a pure software interface. This approach suffers from the following problems:

[0003] 1. The start and end points of the response time cannot be unified based on a time standard.

[0004] Existing testing solutions typically consist of multiple independent hardware devices and software tools, lacking a unified system time base and synchronization mechanism. Because the command triggering unit (such as a voice player or mechanical actuator) and the response capture unit (such as a camera or microphone) are physically separated and have independent clocks, the start point (the moment the command takes effect) and the end point (the moment the device responds) of the response time measurement cannot be accurately marked and aligned on a high-precision timeline. This makes it impossible to compare and analyze the response times of different modalities under various interaction modes such as gestures, voice, apps, and local operations.

[0005] 2. The test environment is highly susceptible to interference, resulting in unstable results.

[0006] When tests are conducted in a typical office environment, they are easily affected by factors such as ambient light (affecting gesture and visual detection), background noise (affecting speech recognition and acoustic detection), and network fluctuations (affecting remote control). The test results fluctuate greatly, failing to accurately reflect the device's performance and providing no stable and reliable data for research and development optimization.

[0007] 3. The testing environment is too idealized and deviates from the actual user experience.

[0008] The tests were conducted in an ideal laboratory environment, but lacked the ability to simulate common challenging factors in real-world user environments, such as noise interference, network interference, and lighting. As a result, the test results deviated from the actual usage scenarios, making it impossible for the evaluation results to fully predict the user experience of the product in the actual market, and making R&D optimization lack effective targeting.

[0009] 4. The testing process relies on manual labor, which is inefficient and highly subjective.

[0010] Current testing methods heavily rely on manual operation (such as using a stopwatch) and subjective judgment (such as visually observing the screen lighting up or listening to prompts). This approach introduces significant human delays and interpretation biases, making it difficult to achieve measurement accuracy beyond the millisecond level. Furthermore, test results vary from person to person, resulting in poor repeatability. Simultaneously, manual testing is extremely inefficient and cannot achieve automated batch execution of test cases, automatic data collection, and analysis.

[0011] As smart home appliances evolve towards multimodal interaction via four parallel channels—voice, gesture, app, and local touch—the industry has come to regard end-to-end response time—from the user issuing a command to the device starting to execute it—as a core performance indicator. However, the multimodal nature of smart home appliances presents unprecedented challenges to their performance testing. Summary of the Invention

[0012] One of the objectives of this invention is to provide a synchronous measurement system for multi-parameter response time of human-computer interaction in smart home appliances. By establishing a unified time reference and constructing a controlled environment, the system enables response time testing of various interaction modes such as voice control, APP control, and gesture control, accurately capturing the response time difference from the "trigger event" to the "home appliance execution response".

[0013] One of the objectives of this invention is achieved through the following technical solution:

[0014] A synchronous measurement system for multi-parameter response time of human-computer interaction in smart home appliances includes a clock synchronization module, a central control unit and data processing module, a trigger simulation module, an environmental simulation module, and a data acquisition module.

[0015] The clock synchronization module serves as the time reference for the entire measurement system, synchronizing the time references of each module within the system.

[0016] The central control unit and data processing module are responsible for coordinating the operation of the measurement system, sending control commands to each module, and coordinating the start, stop and timing of the actions of the trigger simulation module, the environment simulation module and the data acquisition module.

[0017] The trigger simulation module is connected to the central control unit and data processing module, and is used to automatically send multimodal interactive trigger signals to the tested home appliance, including a voice trigger module, an APP simulation control module and a gesture trigger module;

[0018] The environmental simulation module, connected to the central control unit and data processing module, is used to simulate multi-dimensional interference conditions in the real-world usage environment of the tested home appliance, including a noise generator, a network latency simulator, a temperature and humidity controller, and an adjustable lighting system.

[0019] The data acquisition module, connected to the central control unit and the data processing module, is used for multi-dimensional response signal detection and to timestamp each response event of the tested home appliance. It includes a digital power meter, a video capture device, an audio capture device, and a digital oscilloscope.

[0020] This invention constructs a controlled environment using a unified time benchmark to evaluate the response time of smart home appliances under conditions close to real-world usage scenarios (including noise interference and network fluctuations). It incorporates multiple interaction methods such as voice control, APP control, and gesture control into the same testing system and uses a unified time benchmark for synchronous measurement, achieving comparability and unified analysis of cross-modal response times. The trigger simulation module automatically configures trigger events and accurately captures the response time difference between the trigger event and the appliance's execution response, improving the objectivity and repeatability of the test results. It is also applicable to various application scenarios such as laboratory verification, production line quality inspection, and after-sales diagnostics.

[0021] Preferably, the clock synchronization base module uses a PTP master clock and continuously distributes high-precision clock signals to each module in the measurement system via an Ethernet port using the IEEE 1588 Precision Time Protocol.

[0022] Preferably, the voice triggering module uses a speaker with a frequency response of 20Hz to 20kHz, a maximum sound pressure level of 85dB, and an artificial mouth; the APP simulation control module uses a mobile terminal simulator; and the gesture triggering module uses a multi-degree-of-freedom robotic arm capable of waving left and right and up and down.

[0023] Preferably, the noise generator is an audio device with a frequency response of 20Hz to 20kHz, a sound pressure level of 30 to 90dB, and capable of playing white noise and simulating real environmental noise; the network latency simulator is an adjustable network impairment instrument with a latency of 0 to 1000ms and a packet loss rate of 0 to 30%; and the adjustable lighting system is a dimming device with an illuminance of 50 to 2000 Lux and a color temperature of 2700 to 6500K.

[0024] Preferably, the digital power meter has a sampling rate of 100kHz and an accuracy of ±0.1%, used to collect the power status of the tested home appliance in real time; the video capture device uses 1 to 6 cameras to form a monitoring network to capture the real-time response status of the tested home appliance; the audio capture device uses multiple omnidirectional microphones to capture the audio response information of the tested home appliance; and the digital oscilloscope has a sampling rate of 100kHz and an accuracy of ±0.1%, used to capture the voltage / current signal changes of the tested home appliance in real time.

[0025] A second objective of this invention is to provide a synchronous measurement method for multi-parameter response time of human-computer interaction in smart home appliances. This method employs the measurement system described above and includes the following steps:

[0026] S1. The time synchronization basic module synchronizes the reference clock to each module of the measurement system;

[0027] S2. The central control unit and data processing module are configured with the actual usage environment parameters, trigger mode parameters, data acquisition method parameters, and number of test cycles for the tested home appliance;

[0028] S3. The central control unit and data processing module send control commands to each module of the measurement system, and then...

[0029] The environment simulation module starts and executes the configured noise, lighting, and network environment;

[0030] The simulation module is triggered to start and execute trigger control commands: play voice, perform APP simulation operations, or perform gesture actions;

[0031] The tested appliance generates a response after receiving the trigger control command;

[0032] The data acquisition module collects and records the end time T1 of each trigger control command action of the trigger simulation module, and collects and records the response time T2 of the tested home appliance;

[0033] The central control unit and data processing module calculate the response processing time of the tested home appliance, ΔT = T2 - T1;

[0034] S4. Repeat step S3 until the test cycle number is reached. Calculate the average, standard deviation, and variance of the response processing time of the tested home appliance to obtain the average response time, maximum response time, minimum response time, median response time, and response success rate of the tested home appliance.

[0035] Each slave device of the environment simulation module, the trigger simulation module, and the data acquisition module has its own built-in microcontroller unit. The microcontroller unit parses the control instructions of the central control unit and the data processing module and executes the corresponding actions of the control instructions. After execution, it returns the execution status and timestamp through the network interface.

[0036] The present invention has the following beneficial effects:

[0037] 1. This invention overcomes the limitations of single-module testing, achieving cross-modal unification and constructing a unified testing system for multiple interaction modalities. By introducing a hardware-level clock synchronization module, a microsecond-level unified time reference is established for all trigger commands and response acquisition devices, fundamentally solving the time alignment problem of multi-sensor data and enabling accurate calculation and comparability analysis of cross-modal response times. Based on a unified clock reference trigger coordination mechanism and standardized parameter settings, it provides a one-stop evaluation for the multimodal interaction performance of smart home appliances.

[0038] 2. This invention can simulate complex real-world usage environments and accurately simulate multi-dimensional environmental interference factors (such as background noise, complex lighting, and network impairment) in a quantifiable and reproducible manner, thereby precisely reconstructing real user scenarios. By injecting multiple interference parameters into a multi-dimensional real environment, the usability of test results can be improved.

[0039] 3. This invention adopts a multi-dimensional comprehensive judgment method. The data acquisition module combines the intelligent fusion of multi-dimensional signals such as power consumption, video, and audio to determine the device response time of smart home appliances. By using multiple acquisition devices such as digital power meters, video capture devices, audio capture devices, and digital oscilloscopes, the electrical, visual, and acoustic responses of home appliances are monitored and collected simultaneously, avoiding the one-sidedness of judgment based on a single result.

[0040] 4. This invention enables standardized testing, providing an optimized benchmark for the technological development of smart home appliances. Specifically, this includes: standardization of trigger signals (voice command specifications, gesture libraries, network command protocols); standardization of the testing environment (noise, network, lighting, spatial arrangement); and standardization of response time definition (time node division, multimodal response determination).

[0041] 5. This invention can solve the problem of unattended automated testing. This invention automates the testing process, including automatic debugging of the actual operating environment, automatic data acquisition, and automatic monitoring of time feedback. This allows the measurement system to automatically complete testing and record raw data without the need for dedicated personnel after calibration and debugging. It enables precise control and efficient management of the testing process, improves the level of testing automation, and helps reduce testing costs and increase testing efficiency. Attached Figure Description

[0042] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0043] Figure 1 This is a schematic diagram of a synchronous measurement system for multi-parameter response time of human-computer interaction in smart home appliances.

[0044] Figure 2 This is a schematic diagram of the component module framework of a synchronous measurement system for multi-parameter response time of human-computer interaction in smart home appliances;

[0045] Figure 3 This is a flowchart of a synchronous measurement method for multi-parameter response time in human-computer interaction for smart home appliances. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0047] It should be understood that the invention can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, providing these embodiments will make the disclosure thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0048] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0049] A synchronous measurement system for multi-parameter response time in human-computer interaction of smart home appliances, such as... Figure 1 and Figure 2 As shown, it includes a clock synchronization basic module, a central control unit and data processing module, a trigger simulation module, an environmental simulation module, and a data acquisition module.

[0050] The clock synchronization module serves as the time reference for the entire measurement system, synchronizing the time references of each module within the system.

[0051] The central control unit and data processing module are responsible for coordinating the operation of the measurement system, sending control commands to each module, and coordinating the start, stop and timing of the actions of the trigger simulation module, the environment simulation module and the data acquisition module.

[0052] The trigger simulation module is connected to the central control unit and data processing module, and is used to automatically send multimodal interactive trigger signals to the tested home appliance, including a voice trigger module, an APP simulation control module and a gesture trigger module;

[0053] The environmental simulation module, connected to the central control unit and data processing module, is used to simulate multi-dimensional interference conditions in the real-world usage environment of the tested home appliance, including a noise generator, a network latency simulator, a temperature and humidity controller, and an adjustable lighting system.

[0054] The data acquisition module, connected to the central control unit and the data processing module, is used for multi-dimensional response signal detection and to timestamp each response event of the tested home appliance. It includes a digital power meter, a video capture device, an audio capture device, and a digital oscilloscope.

[0055] 1. In this embodiment, the clock synchronization base module adopts a PTP master clock and continuously distributes high-precision clock signals to each module in the measurement system through an Ethernet port using the IEEE 1588 precision time protocol.

[0056] The clock synchronization module serves as the foundation for establishing a unified, accurate, and reliable time reference for the entire test system, forming the basis for all measurements. It has the following characteristics:

[0057] 1) Time distribution: As the PTP master clock, it continuously distributes high-precision clock signals to all devices in the network via the Ethernet port using the IEEE 1588 Precision Time Protocol (PTP).

[0058] 2) Clock source management: Built-in high-stability clock source (TCXO temperature-controlled crystal oscillator) to ensure long-term stability and low jitter of its output clock signal.

[0059] 3) Synchronization Management: Processes synchronization requests from slave devices of all modules, calculates and compensates for network transmission delays, and ensures that the time synchronization accuracy of the entire system reaches the microsecond level.

[0060] 4) Status monitoring: Monitor the synchronization status of all devices in the network, such as offset and latency, and provide alarms when synchronization is lost.

[0061] 2. In this embodiment, the central control unit and data processing module (hereinafter referred to as the central control unit) are the core modules of the measurement system control. The central control unit is responsible for coordinating and controlling the operation of the entire system, sending control commands to all modules, and coordinating the entire testing process. It also manages automated test cases and reports, processes test data, and generates test reports.

[0062] 1) Test process management: Provides a human-computer interaction interface for creating, editing, saving, and automating the execution of complex test cases;

[0063] 2) Device control and coordination: Send control commands to all other modules through various interfaces (Ethernet, USB, GPIB, etc.) to precisely coordinate the start, stop and timing of actions such as triggering simulation, environmental simulation and data acquisition.

[0064] 3) Data aggregation and storage: Receive and store massive amounts of raw data (such as images, audio, and waveforms) transmitted back from the data acquisition module, and associate these data with precise timestamps.

[0065] 4) Data Analysis and Calculation: Using algorithms such as image recognition, acoustic spectrum analysis, and signal processing, the system automatically determines the start and end points of the response, calculates the response time of the tested home appliance, and performs statistical analysis (such as average, standard deviation, variance, maximum, and minimum).

[0066] 5) Report generation: Automatically generate well-structured and visually appealing test reports, including data charts and performance conclusions.

[0067] 3. In this embodiment, the trigger simulation module is used to automatically generate multimodal interactive trigger signals. The core components and technical specifications are as follows.

[0068] Voice trigger module:

[0069] 1) Loudspeaker group: frequency response 20Hz~20kHz, maximum sound pressure level 85dB;

[0070] 2) Voice command library: covering acoustic parameter control for multiple languages ​​including Mandarin, dialects, and English;

[0071] 3) Pitch +200 cents, speech rate adjustable from 0.5 to 2.0 times;

[0072] 4) Distance simulation: The sound pressure level is automatically adjusted at different distances from the guide rail, from 0.5m to 2m.

[0073] APP simulation control module:

[0074] 1) Mobile terminal emulator: Supports Android / iOS systems;

[0075] 2) Network interface: Supports Wi-Fi 2.4G / 5G, Bluetooth 4.0 / 5.0, and Zigbee 3.0;

[0076] 3) Command transmission method control: precision +1ms;

[0077] Gesture triggering module:

[0078] 1) Robotic arm system: 6 degrees of freedom, repeatability +0.1mm;

[0079] 2) The programmable control system controls the robotic arm to swing left and right, up and down;

[0080] 4. In this embodiment, the environment simulation module is used to simulate multi-dimensional interference conditions in real-world usage environments, and its core components are as follows.

[0081] 1) Noise generator: 20Hz~20kHz, sound pressure level adjustable from 30 to 90dB, can play various types of noise signals: white noise, real environment (kitchen noise, living room TV sound), etc.

[0082] 2) Network latency simulator: 0-1000ms latency, 0-30% packet loss rate adjustable;

[0083] 3) Temperature and humidity controller: Temperature adjustable from 15 to 40℃, humidity adjustable from 30 to 90%RH;

[0084] 4) Adjustable lighting system: 50-2000 Lux illuminance, color temperature adjustable from 2700K to 6500K.

[0085] 5. In this embodiment, the data acquisition module is used for multi-dimensional response signal detection and to accurately timestamp each event. Its core components are as follows.

[0086] 1) Digital power meter: sampling rate 100kHz, accuracy ±0.1%, real-time acquisition of device power status;

[0087] 2) Video capture equipment: 1 to 6 high-speed cameras form a monitoring network for image acquisition. The high-speed cameras have a resolution of 1920×1080 and a frame rate of 1000fps to capture real-time response status.

[0088] 3) Audio capture equipment: Eight omnidirectional microphones collect audio information in real time and perform acoustic spectrum analysis;

[0089] 4) Digital oscilloscope: sampling rate 100kHz, accuracy ±0.1%, real-time capture of voltage / current signal changes of the tested household appliance.

[0090] A synchronous measurement method for multi-parameter response time of human-computer interaction in smart home appliances, employing the measurement system described above in this invention, such as... Figure 3 As shown, the measurement method includes the following steps:

[0091] S1. The time synchronization basic module synchronizes the reference clock to each module of the measurement system;

[0092] S2. The central control unit and data processing module are configured with the actual usage environment parameters, trigger mode parameters, data acquisition method parameters, and number of test cycles for the tested home appliance;

[0093] S3. The central control unit and data processing module send control commands to each module of the measurement system, and then...

[0094] The environment simulation module starts and executes the configured noise, lighting, and network environment;

[0095] The simulation module is triggered to start and execute trigger control commands: play voice, perform APP simulation operations, or perform gesture actions;

[0096] The tested appliance generates a response after receiving the trigger control command;

[0097] The data acquisition module collects and records the end time T1 of each trigger control command action of the trigger simulation module, and collects and records the response time T2 of the tested home appliance;

[0098] The central control unit and data processing module calculate the response processing time of the tested home appliance, ΔT = T2 - T1;

[0099] S4. Repeat step S3 until the test cycle number is reached. Calculate the average, standard deviation, and variance of the response processing time of the tested home appliance to obtain the average response time, maximum response time, minimum response time, median response time, and response success rate of the tested home appliance.

[0100] The measurement method in this embodiment starts the clock synchronization base module, the central control unit, and the data processing module to establish a unified time base, and each module performs hardware time synchronization. After the system starts, the clock synchronization base module first establishes a unified time base for the entire system. Specifically, the PTP master clock (based on the IEEE 1588 precision time protocol) distributes high-precision clock signals to the central control unit, the trigger analog module, and the data acquisition module through a switch. After all slave devices of all modules complete time synchronization calibration, the system enters the "ready" state. This step ensures the consistency of timestamps between modules, achieving microsecond-level time alignment.

[0101] Each slave device of the environment simulation module, the trigger simulation module, and the data acquisition module has its own built-in microcontroller unit. The microcontroller unit parses the control instructions of the central control unit and the data processing module and executes the corresponding actions of the control instructions. After execution, it returns the execution status and timestamp through the network interface.

[0102] Test case:

[0103] 1. The test object is a household air conditioner. Taking the scenarios of "voice power-on" and "mobile APP control" as examples, the overall response time of voice command and APP remote control command "turn on the air conditioner" is tested.

[0104] Under the unified scheduling of the central control unit, the testing system works in concert with the clock synchronization module, environmental simulation module, trigger simulation module and data acquisition module to achieve full-process measurement and analysis from voice triggering to device action response.

[0105] 2. Hardware Equipment List:

[0106] Device Type Model / Spec Quantity Purpose Remarks GPS clock source Meinberg M1000 1 PTP master clock Time source of the whole system, provides authoritative UTC time PTP master clock switch EDS-G205-1GTXSFP-T 1 Clock distribution Support IEEE 1588 PTP Industrial computer Advantech ARK-3500 Series 1 Central control unit Support IEEE 1588 PTP Noise player (power amplifier + speaker) BOSE S1 Pro 1 Environmental noise simulation Support PTP audio interface control. Sound level meter B&K 2250 1 Environmental noise monitoring Digital power meter WT5000 Series 1 Power response detection Support IEEE 1588 PTP Digital oscilloscope ZDL6000 1 Signal waveform analysis and waveform measurement Support IEEE 1588 PTP High-speed camera MV-GES240GC 2 LED state detection Support IEEE 1588 PTP Microphone array XMOS XVF3510 1 Voice response detection Support IEEE 1588 PTP Robotic arm Universal Robots UR5e, UR10e Repeatability accuracy < 0.1 mm 1 Gesture simulation Support IEEE 1588 PTP Artificial mouth simulator B&K 4227 Artificial mouth 1 Standardized voice output -

[0107] 3. Prototype preparation and module configuration before testing.

[0108] 1) Connection and setup of digital power meter and digital oscilloscope.

[0109] The power input terminal of the air conditioner prototype under test is connected to the AC power circuit through a digital power meter. The power meter serves as an intermediate measurement node to collect the power change signal of the air conditioner in real time.

[0110] The power meter input terminal (IN) is connected to the power supply output terminal;

[0111] Connect the power meter output (OUT) to the prototype power input.

[0112] The power meter is connected to the central control unit via an Ethernet or USB interface to enable real-time power data transmission.

[0113] This configuration ensures that the power curve accurately reflects the air conditioner's response time when the device starts up or its operating status changes.

[0114] The digital power meter connects to the central control unit via an Ethernet interface, transmits real-time power data using the UDP / IP protocol, and supports data packet timestamp synchronization.

[0115] 2) Connecting and setting up video capture equipment.

[0116] Position of the high-speed camera and the air conditioner prototype: The video capture equipment consists of 3 high-speed cameras, each positioned at a 45° angle from the front and 30cm horizontally away from the air conditioner, to ensure that the changes in the air conditioner indicator light can be captured from all directions.

[0117] The high-speed camera is connected to the central control unit via an Ethernet or USB interface. After the high-speed camera is started, it enters real-time acquisition mode. After the central control unit issues a "start acquisition" command, it records the video stream synchronously and aligns it with the command issuance time through timestamp calibration to calculate the visual response time.

[0118] 3) Connection and setup of audio capture device and voice trigger module.

[0119] The audio system consists of two parts: a voice trigger module and an audio capture device.

[0120] Voice trigger module: Set the distance between the artificial mouth and the air conditioner prototype to 1m, and use the artificial mouth to play the wake-up phrase such as "Xiao Mei, Xiao Mei, turn on the air conditioner." Record the number of times the air conditioner prototype is woken up and the time. The artificial mouth playback device is connected to the central control unit via an Ethernet interface, and the central control unit controls the playback sequence and records the number of wake-ups.

[0121] Audio capture equipment: A three-microphone array is arranged around the air conditioner's voice pickup microphone to capture wake-up voice input time and the device's acoustic response (such as prompts or voice feedback).

[0122] The voice trigger module and audio capture device are connected to the central control unit via Ethernet or USB interface. The audio capture device transmits the collected data back in real time. The central processing unit ensures the consistency of trigger time and response time through a unified time synchronization protocol (PTP protocol).

[0123] 4) Connection and settings of the APP simulation control module.

[0124] After the air conditioner prototype completes its network connection, the APP simulation control module (consisting of a smart terminal and a robotic arm) receives test commands through the central control unit. The robotic arm simulates the user's touch operations on the mobile APP interface, executing commands such as "power on" and "power off." The smart terminal and robotic arm communicate with the central control unit via Ethernet or USB interface, and send back execution status and operation confirmation signals after the command is executed.

[0125] 5) Connection and settings of the gesture triggering module.

[0126] The gesture triggering module uses a robotic arm to simulate user gestures (such as waving to turn on the device). The robotic arm is connected to the central control unit via an Ethernet interface, receives central control commands, executes the gesture actions, and simultaneously sends back a signal indicating that the action is complete.

[0127] 6) Connection and settings of the environment simulation module.

[0128] The environment simulation module is used to construct different external interference conditions (noise, light, network environment, etc.), and achieves unified management and parameter feedback through the central control unit.

[0129] 4. Test parameters for the environment simulation module.

[0130] 1) Noise parameters: Play a noise signal (white noise) through the speakers. The frequency of the sound signal is controlled between 300Hz and 2000Hz, and the noise range is 40 to 60dB. The distance between the speakers and the air conditioner prototype is set to 1m.

[0131] 2) Lighting parameters: The LED lighting system is adjusted to 800 Lux and a color temperature of 5000 K to simulate common indoor ambient light.

[0132] 3) Network parameters: The network environment is switched to wireless connection mode with random packet loss mode and a packet loss probability of 0.5%.

[0133] The environmental simulation module is connected to the central control unit via an Ethernet or USB interface. The central control unit can read and record environmental parameters such as noise, indoor light intensity, humidity, and packet loss rate in real time, and ensure feedback on command execution status.

[0134] 5. Data synchronization and calibration.

[0135] All modules are managed uniformly through a central control unit and use a unified time base (PTP synchronization).

[0136] Before the test begins, the system automatically performs a synchronization calibration to ensure that the timestamp error of each module's data is less than 10ms, thus guaranteeing the accuracy of response time measurement.

[0137] 6. Test parameter configuration.

[0138] In the central control unit and data processing module, the test scenario and parameter settings are first performed.

[0139] Test scenario: Voice power-on;

[0140] Voice command: "Turn on the air conditioner";

[0141] Number of tests: 10;

[0142] Ambient noise range: 40–60 dB;

[0143] Network conditions: Wireless connection mode, packet loss rate <0.5%;

[0144] Lighting conditions: 800 Lux, color temperature 5000 K;

[0145] Temperature conditions: 25±1℃;

[0146] This parameter is entered and saved through the human-computer interaction interface, and the system automatically generates a test task file and distributes it to each sub-module.

[0147] 7. Test the execution system process.

[0148] 1) System initialization starts the basic clock synchronization module.

[0149] The PTP switch synchronizes the time base to the central control unit and data processing module, trigger simulation module (voice playback), and data acquisition module (high-speed camera, microphone array, power meter, and oscilloscope);

[0150] 2) Parameter configuration and test task setting.

[0151] Users configure test parameters through the human-machine interface at the upper level of the central control unit.

[0152] Test scenario: Voice power-on, voice command: turn on the air conditioner; artificial mouth position: 1 meter away from the air conditioner; calibration microphone position: at the air conditioner microphone;

[0153] Number of tests: 10; Environmental parameters: background noise 40-60dB, ambient light 800Lux, color temperature 5000K, temperature and humidity 25±1℃; Network conditions: wireless connection mode, random packet loss rate <0.5%. After the settings are completed, the central control unit generates a test task configuration file (.json or .xml) and distributes it to each module.

[0154] 3) Simultaneous preheating stage of environment and equipment.

[0155] The central control unit sends a "prepare for test" command to the environmental simulation module, the trigger simulation module, and the data acquisition module, and simultaneously initiates synchronization time verification;

[0156] The environmental simulation module performs preheating operations (e.g., white noise signal loading, lighting system brightness adjustment, temperature and humidity stabilization), while the data acquisition modules (camera, microphone, power meter) enter standby sampling mode and send a "ready" signal back. After all modules respond, the central control unit records the system synchronization completion timestamp.

[0157] 4) Environment configuration execution.

[0158] Upon receiving the control command, the environmental simulation module plays 50dB white noise via a noise generator. The network environment switches to a random packet loss mode in wireless connection mode with a packet loss probability of 0.5%. The lighting system is adjusted to 800Lux, color temperature 5000K, and temperature 25±1℃. Environmental parameters are uploaded to the central control unit in real time for verification and recording.

[0159] 5) Test the start-up and trigger command issuance.

[0160] The central control unit broadcasts a "Start Test" signal (including test round, PTP timestamp, and environmental snapshot), and all modules begin recording.

[0161] When the trigger simulation module receives the instruction, the artificial mouth executes the voice command ("Xiao Mei, Xiao Mei, turn on the air conditioner"); at the same time, the central control unit marks the start time T1_start and the end time T1_end of the trigger command (based on the end time of the voice signal detected by the calibration microphone, T1 = T1_end).

[0162] 6) Data acquisition and time stamping.

[0163] Upon receiving the instruction, the data acquisition module activates the high-speed camera to detect LED status, the microphone array to detect voice response, and the power meter and oscilloscope to monitor power and perform signal waveform analysis and measurement.

[0164] Microphone array: Acquires audio frames and extracts features, then sends feature data packets with PTP timestamps to the central control unit via UDP protocol;

[0165] The data processing module of the central control unit starts the detection algorithm after receiving the sample from T1_start.

[0166] High-speed camera: monitors changes in the status of the LEDs on the air conditioning panel, and after being encoded and compressed by the image acquisition card, sends the image with a timestamp to the central control unit via UDP.

[0167] The power meter and oscilloscope are used for power monitoring, signal waveform analysis and waveform measurement. The power curve and electrical waveform changes of the prototype are monitored in real time. The real-time acquired data with PTP timestamp is sent to the central control unit via UDP through the acquisition card.

[0168] After receiving the T1 trigger sample, the data processing module starts the detection algorithm to perform feature recognition and event detection on each channel.

[0169] 7) Response time and marking.

[0170] The test is executed, and the start and end times are marked.

[0171] (1) Execute the trigger command and record the trigger time T1.

[0172] Voice command start time: T1_start, voice command end time (based on the voice command received by the calibration microphone): T1_end, reference time: T1=T1_end;

[0173] (2) Data acquisition time stamp.

[0174] Power and signal waveform detection, recording time: T2a = the moment when the power change was detected;

[0175] Visual detection detects a change in the status of the LEDs on the air conditioning panel and records the time: T2b = the time when the visual change first occurs;

[0176] Acoustic detection (T2c), fan operating noise, recording time: T2c = the time when the acoustic feature first appears;

[0177] 8) Calculate the response time.

[0178] The trigger simulation module and data acquisition module transmit the tagged data to the central control unit and data processing module. After receiving the data, the module performs data analysis and calculates the response time.

[0179] Electrical response time ΔT_electric = T2a - T1 (power change);

[0180] Visual response time ΔT_visual = T2b - T1 (indicator light change);

[0181] Acoustic response time ΔT_voice = T2c - T1 (fan operation);

[0182] The final total response time for the test is: ΔT_total1 = min(T2a, T2b, T2c) - T1;

[0183] 9) Statistically analyze the parameters and generate a test report.

[0184] The system automatically generates statistical results for 10 tests:

[0185] Average response time: μ = Σ(ΔTᵢ) / n;

[0186] Standard deviation: σ = √[Σ(ΔTᵢ-μ)² / (n-1)];

[0187] Variance: σ²=Σ(ΔTᵢ-μ)² / (n-1);

[0188] Maximum response time: max(ΔTᵢ);

[0189] Minimum response time: min(ΔTᵢ);

[0190] Median response time: median(ΔTᵢ);

[0191] Response success rate: (Number of correctly identified and successfully responded to / Total number of commands) × 100%;

[0192] Command recognition accuracy: Number of correctly recognized commands / Total number of commands × 100%.

[0193] 10) Test result output and report generation.

[0194] The central control unit summarizes the results from each module and outputs a test report (PDF / CSV) containing the following information.

[0195] (1) Test configuration information (scenario, environment parameters, network conditions);

[0196] (2) Distribution and trend of response time for each channel;

[0197] (3) Response time statistics;

[0198] (4) Anomaly detection logs (such as records of speech recognition failure, communication timeout, and packet loss retransmission);

[0199] Automatically generated test summaries and data visualization results (response time bar chart, box plot, trend curve).

[0200] This test case uses quantifiable and reproducible simulations of multi-dimensional environmental interference factors (such as background noise, complex lighting, and network impairment) to accurately reconstruct real user scenarios. Multiple interference parameters are injected into the multi-dimensional real environment. A multi-dimensional comprehensive judgment method is adopted. The data acquisition module intelligently fuses power consumption, video, and audio signals to determine the device response time of smart home appliances. Multiple acquisition devices, including digital power meters, video capture devices, audio capture devices, and digital oscilloscopes, simultaneously monitor and collect the electrical, visual, and acoustic responses of the appliances, avoiding the bias of judging from a single result.

[0201] The above embodiments of the present invention are not intended to limit the scope of protection of the present invention. The implementation of the present invention is not limited thereto. All other modifications, substitutions or alterations made to the above structure of the present invention based on the above content of the present invention, in accordance with ordinary technical knowledge and common practice in the field, without departing from the basic technical idea of ​​the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A multi-parameter response time synchronization measurement system for human-machine interaction of smart home appliances, characterized in that, The system comprises a clock synchronization base module, a central control unit and data processing module, a trigger simulation module, an environment simulation module and a data acquisition module, wherein, The clock synchronization base module synchronizes the time reference of each module in the measurement system as the time reference of the entire measurement system. The central control unit and data processing module are responsible for coordinating the operation of the measurement system, sending control instructions to each module, and coordinating the start and stop of the actions of the trigger simulation module, the environment simulation module and the data acquisition module. The trigger simulation module is connected to the central control unit and data processing module, and is used to automatically send multi-modal interaction trigger signals to the measured household appliance, including a voice trigger module, an APP simulation control module and a gesture trigger module. The environment simulation module is connected to the central control unit and data processing module, and is used to simulate the multi-dimensional interference conditions of the real use environment of the measured household appliance, including a noise generator, a network delay simulator, a temperature and humidity controller and an adjustable lighting system. The data acquisition module is connected to the central control unit and data processing module, and is used for multi-dimensional response signal detection and time stamping for each response event of the measured household appliance, including a digital power meter, a video capture device, an audio capture device and a digital oscilloscope.

2. The system for synchronously measuring the response time of multiple parameters of human-computer interaction of smart home appliances according to claim 1, characterized in that: The clock synchronization base module uses a PTP master clock to continuously distribute high-precision clock signals to each module in the measurement system through an Ethernet port using the IEEE 1588 precision time protocol.

3. The system for synchronously measuring the response time of multiple parameters of human-computer interaction of intelligent household appliances according to claim 1, characterized in that: The voice trigger module uses a speaker with a frequency response of 20Hz-20kHz, a maximum sound pressure level of 85dB and an artificial mouth; the APP simulation control module uses a mobile terminal simulator; and the gesture trigger module uses a multi-degree-of-freedom mechanical arm that can swing left and right and up and down.

4. The system for synchronously measuring the response time of multiple parameters of human-computer interaction of intelligent household appliances according to claim 1, characterized in that: The noise generator uses a sound device with a frequency response of 20Hz-20kHz, a sound pressure level of 30-90dB, and can play white noise and simulate real environment noise; the network delay simulator uses a adjustable network impairment instrument with a delay of 0-1000ms and a packet loss rate of 0-30%; and the adjustable lighting system uses a dimming device with an illuminance of 50-2000Lux and a color temperature of 2700-6500K.

5. The system for synchronous measurement of response time of multiple parameters of human-machine interaction of smart home appliances according to claim 1, characterized in that: The digital power meter has a sampling rate of 100kHz and an accuracy of ±0.1%, and is used to collect the power state of the measured household appliance in real time; the video capture device uses 1-6 cameras to form a monitoring network to capture the real-time response state of the measured household appliance; The audio capture device uses multiple omnidirectional microphones to capture the audio response information of the measured household appliance; The digital oscilloscope has a sampling rate of 100kHz and an accuracy of ±0.1%, and is used to capture the voltage / current signal changes of the measured household appliance in real time.

6. A method for synchronously measuring response time of multiple parameters of human-computer interaction of smart home appliances, characterized in that, The measurement method comprises the following steps: S1. The time synchronization base module synchronizes the reference clock to each module in the measurement system; S2. The central control unit and data processing module configure the real use environment parameters, trigger mode parameters, data acquisition method parameters and test cycle number of the measured household appliance. S3. The central control unit and data processing module send control instructions to each module of the measurement system, then, The environmental simulation module is started to execute the configured noise, illumination, and network environment; The trigger simulation module is started to execute the trigger control instructions: playing voice, executing APP simulation operation, or executing gesture action; The measured household appliance produces a response after receiving the trigger control instructions; The data acquisition module collects and records the end time T1 of each trigger control instruction action of the trigger simulation module, and collects and records the response time T2 of the measured household appliance; The central control unit and data processing module calculate the response processing time ΔT = T2-T1 of the measured household appliance; S4. Repeat steps S3 to the test cycle number, and statistically calculate the average value, standard deviation, and variance of the response processing time of the measured household appliance to obtain the average response time, maximum response time, minimum response time, median response time, and response success rate of the measured household appliance.

7. The method of claim 6, wherein the method further comprises: Each slave device of the environmental simulation module, the trigger simulation module, and the data acquisition module is internally provided with a micro control unit, which analyzes the control instructions of the central control unit and data processing module and executes the corresponding actions of the control instructions, and returns the execution status and time stamp after the execution is completed through the network interface.

Images