A multi-modal sensing and dual-branch human touch and load electrical appliance distinguishing detection system
By combining multimodal sensing with a dual-branch neural network, accurate differentiation between human touch and load operating status is achieved, solving the problems of high misjudgment rate and insufficient robustness in existing technologies, and improving the safety and reliability of low-voltage power distribution systems.
Patent Information
- Application Number
- CN202610432975.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-03
- Publication Date
- 2026-07-10
AI Technical Summary
Existing low-voltage power distribution safety protection schemes have difficulty accurately distinguishing between human touch and load operation status in humid environments, power grid fluctuations, and load start-up impacts, resulting in a high misjudgment rate and difficulty in achieving both real-time performance and robustness.
Multimodal sensors are used to synchronously collect features such as capacitance, pressure, and environment from human touch and load electrical appliances. These features are then combined with a dual-branch neural network for feature modeling and fusion decision-making. The final result is output through attention fusion and temporal collaborative decision-making, and power-off and alarm are executed when a dangerous state is determined.
It achieves reliable differentiation between human touch and load operation status in complex environments, reduces the false judgment rate, improves the robustness of the system in high humidity, high temperature and power grid disturbance environments, and maintains basic protection capabilities when a single sensor malfunctions.
Smart Images
Figure CN122361936A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of low-voltage power distribution safety protection and intelligent identification technology, and in particular to a system, method, electronic device and storage medium that distinguishes and detects human touch state and load electrical appliance working state based on multimodal sensing and dual-branch neural network and outputs safety control results. Background Technology
[0002] Existing low-voltage power distribution safety protection schemes mostly employ fixed threshold methods, leakage current discrimination methods, or single feature identification methods. For example, judging solely based on leakage current, capacitance changes, or current surges, while simple to implement, can easily lead to misjudging normal load operation as human contact in humid environments, power grid fluctuations, inductive load startup impacts, or capacitive load charging transients, resulting in malfunctions. Conversely, in scenarios where human contact and load operation overlap, missed detections may also occur.
[0003] On the other hand, human touch and load operation differ fundamentally in their physical mechanisms. Human touch typically manifests as changes in capacitance, pressure, and contact impedance, as well as unstable active power and aperiodic current fluctuations. In contrast, resistive, inductive, and capacitive loads exhibit different harmonic distributions, power factors, phase differences, impedance, and startup impact characteristics. If these multiple characteristics cannot be modeled collaboratively, relying solely on a single dimension for discrimination makes it difficult to simultaneously ensure accuracy, real-time performance, and environmental robustness.
[0004] Therefore, a technical solution is needed that can simultaneously collect, jointly model, and fuse human touch biometrics and load electrical characteristics for decision-making, so as to more reliably distinguish between human touch and normal load operation in low-voltage power distribution scenarios such as household sockets, industrial distribution boxes, and AC charging interfaces. Summary of the Invention
[0005] Technical problems to be solved The purpose of this invention is to provide a detection system and method for distinguishing between human touch and load electrical appliances based on multimodal sensing and dual-branch technology, so as to solve the problems in the prior art that it is difficult to accurately distinguish between human touch and normal load operation, the high misjudgment rate in complex environments, and the difficulty in balancing real-time performance and robustness.
[0006] To achieve the above objectives, the present invention provides the following technical solution: a detection system for distinguishing between human touch and load electrical appliances based on multimodal sensing and dual-branch sensing, comprising a multimodal sensing module for simultaneously acquiring features related to human touch such as capacitance, pressure, environment, and current difference, as well as features related to load such as current, voltage, power factor, and impedance.
[0007] A data preprocessing module is set up to perform unified clock-triggered sampling, time alignment, filtering and denoising, standardization and feature construction on the above multimodal data; A dual-branch neural network inference module is set up, which uses the human body branch and the load branch respectively to infer the human touch characteristics and the load electrical characteristics; Set up a fusion decision module to perform attention fusion and temporal collaborative decision-making on the dual-branch output results; When a dangerous situation is determined to be caused by human contact, the execution control module will execute power cut-off and / or alarm.
[0008] Compared with existing technologies, the present invention provides a multimodal sensing and dual-branch human touch and load electrical appliance differentiation detection system, which has the following beneficial effects:
[0009] 1. By simultaneously collecting multi-source information such as capacitance, pressure, environment, current, voltage and power, and constructing human body characteristics and load characteristics, a multi-dimensional joint characterization of human body touch and load working state can be achieved, which can reduce misjudgment caused by single threshold judgment.
[0010] 2. By modeling the two types of features separately through human body branches and load branches, and then using attention fusion and temporal collaborative decision-making to output the final result, a more reliable distinction can be made between human touch, normal load operation, and human body and load superposition scenarios.
[0011] 3. By introducing environmental correction mechanisms such as humidity, temperature and light, as well as judgment conditions such as start-up impact duration, power threshold and current difference threshold, the robustness of the system in complex environments such as high humidity, high temperature and power grid disturbance can be improved.
[0012] 4. By setting up an execution control module and redundant decision paths, it can quickly output on / off and alarm control when a dangerous state of human touch is detected, and can still maintain basic protection capabilities when a single sensor is abnormal. Attached Figure Description
[0013] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 This is a schematic diagram of the multimodal sensing deployment of the present invention; Figure 3 This is a schematic diagram of the dual-branch neural network inference structure of the present invention; Figure 4 This is a schematic diagram of the signal acquisition and execution control connection of the present invention; Figure 5 This is a flowchart illustrating the model training, optimization, and deployment process of this invention. Figure 6 This is a schematic diagram illustrating the recognition effect of the present invention under different environmental conditions.
[0014] Explanation of reference numerals in the attached figures 100. System; 110. Multimodal Sensing Module; 111. Capacitive Sensing Unit; 112. Pressure Sensing Unit; 113. Current Sensing Unit; 114. Voltage Sensing Unit; 115. Power Factor / Power Detection Unit; 116. Environmental Parameter Sensing Unit; 117. Temperature Sensing Unit; 118. Synchronization Triggering and Frame Alignment Unit; 120. Data Preprocessing Module; 130. Dual-Branch Neural Network Inference Module; 131. Human Body Branch; 132. Load Branch; 133. Attention Fusion Layers; 134. Classification Output; 140. Fusion Decision Module; 150. Execution Control Module; 151. Driver Unit; 152. Switching Unit; 153. Alarm Unit; 210. Signal Acquisition Circuit; 220. Edge Processor; 230. Load Terminal; 310. Sample Acquisition Unit; 320. Synchronous Labeling and Preprocessing Unit; 330. Dual-Branch Pre-training / Joint Training Unit; 340. Quantization Deployment Unit; 350. Online Inference and Confidence Screening Unit; 360. Cloud Incremental Training Unit Detailed Implementation
[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Equivalent substitutions and conventional modifications made by those skilled in the art without departing from the concept of the present invention should all fall within the scope of protection of the present invention.
[0016] Example 1: System Overall Structure.
[0017] like Figure 1 As shown, the system 100 provided in this embodiment includes a multimodal sensing module 110, a data preprocessing module 120, a dual-branch neural network inference module 130, a fusion decision module 140, and an execution control module 150.
[0018] The multimodal sensing module 110 is used to collect touch signals and environmental signals related to human touch, as well as electrical signals in the load circuit; the data preprocessing module 120 is used to process the raw data into standardized features that can be called by the model; the dual-branch neural network inference module 130 includes a human body branch 131 and a load branch 132; the fusion decision module 140 obtains the final judgment conclusion based on the dual-branch output results; and the execution control module 150 is used for output drive, on / off and alarm control.
[0019] Example 2: Multimodal sensing deployment.
[0020] like Figure 2As shown, the capacitance sensing unit 111 preferably adopts a 3×3 array capacitor structure and is arranged in the vicinity of the live wire terminal and / or the neutral wire terminal to sense the capacitance change generated when a human body approaches or touches the terminal; the pressure sensing unit 112 preferably adopts a thin-film pressure sensor to obtain the touch force; the current sensing unit 113 is arranged in series with the live wire circuit, the voltage sensing unit 114 is connected across the live wire and the neutral wire, the power factor / power detection unit 115 is used to collect the power factor and active power when the load is working, or to calculate the power factor and active power based on the synchronously collected voltage and current signals; the environmental parameter sensing unit 116 is used to collect at least one parameter among temperature, humidity and light; the temperature sensing unit 117 is arranged close to the load terminal to assist in the calculation of the impedance temperature coefficient.
[0021] Each sensing unit is uniformly driven by the synchronization triggering and frame alignment unit 118. In the preferred embodiment, the sampling rate is 1kHz and the timestamp accuracy is no greater than 1ms.
[0022] Example 3: Data preprocessing and feature construction.
[0023] After time alignment of the signals from each channel, the data preprocessing module 120 employs appropriate filtering and feature extraction methods.
[0024] For capacitance signals, a 50Hz notch filter is preferably applied first, followed by three-layer wavelet soft thresholding denoising based on a db4 wavelet basis. For pressure signals, a moving average filter with a window size of 5 is preferably applied. For current and voltage signals, a Hanning window Fourier transform is preferably performed within a 1024-point sampling window to extract the RMS current value, RMS voltage value, 3rd / 5th / 7th harmonic proportions, phase difference, and fundamental impedance. For power signals, active power and power factor are preferably calculated within a 100ms integration window. Outliers can be removed using the 3σ criterion, where the 3σ criterion refers to using three times the standard deviation above and below the mean as the anomaly detection range.
[0025] In this embodiment, the human body touch characteristics preferably include capacitance change amplitude, number of activated electrodes, electrode activation rate, pressure peak value, pressure rise slope, touch duration, human body contact impedance, environmental correction factor, and current difference; the load characteristics preferably include current RMS value, voltage RMS value, 3rd / 5th / 7th harmonic ratio, phase difference, power factor, fundamental impedance, impedance temperature coefficient, peak impulse current, impulse duration, and power stability.
[0026] Optionally, the touch features are 20-dimensional, the load features are 18-dimensional, and the environment features are 4-dimensional, and then dimensionality reduction is performed to form fused features for model use.
[0027] Example 4: Human Feature Extraction.
[0028] To improve the stability of human touch recognition, capacitance and pressure features can be fused into touch confidence.
[0029] The confidence scores for capacitance characteristics C_conf, pressure characteristics F_conf, and fusion touch confidence scores H_conf can be calculated according to equations (5) to (7), respectively.
[0030] Furthermore, human touch can also be further verified through the difference in current.
[0031] The current difference ΔI can be calculated according to formula (8). Wherein, ΔI represents the difference between the live wire current and the neutral wire current, I_L represents the live wire current, and I_N represents the neutral wire current. When ΔI is greater than the preset human safety threshold and H_conf reaches the effective touch threshold, it can be determined as a dangerous state of human touch. In a preferred embodiment, the human safety threshold can be 5mA; when the humidity is greater than or equal to 80% RH, the current difference trigger threshold can be lowered; where RH represents relative humidity.
[0032] Example 5: Two-branch neural network inference.
[0033] Figure 3 An optional dual-branch neural network inference structure is shown. The human body branch 131 preferably comprises two layers of a Convolutional Neural Network (CNN) and two layers of a Bidirectional Long Short-Term Memory (BiLSTM) network, while the load branch 132 preferably comprises two layers of a CNN and one layer of a BiLSTM network. The human body branch is used to extract the spatial distribution and temporal evolution of touch data, while the load branch is used to extract frequency domain patterns and temporal patterns from current, voltage, and power features. The attention fusion layer 133 preferably employs a 4-head multi-head attention mechanism, dynamically weighting the first confidence S1 output from the human body branch and the second confidence S2 output from the load branch to obtain fused features and output four categories of results: human touch, resistive load, inductive load, and capacitive load.
[0034] In one optional implementation, the human body branch has 32 convolutional kernels, 64 BiLSTM hidden units, and a Dropout probability of 0.2; where Dropout represents a regularization operation that randomly deactivates some neurons during training. The load branch can also have 32 convolutional kernels and 64 BiLSTM hidden units. During training, human body touch samples, load samples, and overlay scene samples can be divided into training, validation, and test sets in an 8:1:1 ratio.
[0035] Example 6: Integrating Decision and Enforcement Control.
[0036] Figure 4 The connection relationship between the signal acquisition circuit 210, the edge processor 220, and the execution control module 150 is shown. The execution control module 150 includes a drive unit 151, a switching unit 152, and an alarm unit 153. After receiving the dual-branch inference results, the fusion decision module 140 can make a judgment according to the following logic: when the confidence level of human touch is not lower than the first preset threshold and the current difference is not lower than the second preset threshold, it is determined to be a dangerous state of human touch; when the load confidence level is not lower than the third preset threshold and the active power is not lower than the fourth preset threshold, it is determined to be a normal working state of the load; when the confidence level of human touch and the load confidence level both fall into the fuzzy range, time series analysis is continued, and the duration of touch and the duration of load impact are used as secondary discrimination criteria.
[0037] In one optional implementation, the first preset threshold can be between 0.8 and 0.9, the second preset threshold can be 5mA, the third preset threshold can be above 0.9, the fourth preset threshold can be 10W, and the human touch duration threshold can be 100ms. If the confidence level of human touch is high and the duration exceeds 100ms, even if the load characteristic confidence level is high, a human touch danger state is output first to meet the safety priority principle. The switching unit 152 can be any one or more combinations of a relay, an electronic switch, or a circuit breaker mechanism.
[0038] In a preferred embodiment, after determining a dangerous situation involving human contact, the fusion decision module 140 outputs an on / off control signal to the drive unit 151 and synchronously drives the alarm unit 153 to output an audible and visual alarm when necessary. When a critical sensor fails, the system can automatically switch to a redundant decision path, such as using power factor, impedance, and temperature characteristics to replace current difference characteristics, thereby maintaining basic protection functions.
[0039] Example 7: Training, Optimization, and Deployment.
[0040] Figure 5 A training and deployment workflow is illustrated. First, human touch samples, load samples, and human-load superimposed scenario samples are collected. Then, simultaneous annotation, preprocessing, and feature construction are performed. Next, the human branch and load branch are pre-trained separately, and the overall dual-branch model is jointly trained. After training, further quantization, pruning, and / or distillation are performed to adapt to edge processor deployment. During online runtime, low-confidence samples can be filtered and uploaded back to the cloud for incremental training before updated parameters are distributed to edge devices.
[0041] Optionally, the training samples can include 27,000 human touch samples and 30,000 load samples, with approximately 10,000 samples each for resistive, inductive, and capacitive loads. During training, multi-class cross-entropy can be used as the loss function, and the optimizer can be Adam; where Adam represents an adaptive moment estimation optimization algorithm. During incremental training, the human branch parameters can be frozen, and only the load branch and fusion layer parameters are updated to reduce training time.
[0042] Example 8: Environmental Adaptation and Performance Verification.
[0043] Figure 6 The diagram illustrates the recognition performance under different environmental conditions. Based on the test results of the disclosed samples, the overall system accuracy remains high under standard, high humidity, high temperature, and power grid disturbance conditions. Optionally, under conditions of -25℃, 95% RH, and with ±10% voltage fluctuations and electrical fast transient / burst interference as specified in GB / T 17626.4, the system accuracy decrease can be controlled to within 1%.
[0044] In field applications such as home kitchens and industrial workshops, the system can distinguish between the normal operating state of different loads such as electric kettles, table fans, LED power supplies, motors, welding machines, and chargers, and the dangerous state of accidental or continuous human contact with live parts. For example, when a 100W table fan starts, even if there is a starting inrush current, the system can still identify it as a normally operating load based on conditions such as the inrush duration being shorter than a preset threshold and the power factor meeting the characteristics of an inductive load. When a person continuously touches the live casing, the system can output trip control based on conditions such as increased confidence in human characteristics, current difference reaching a threshold, and contact duration exceeding 100ms.
[0045] Example 9: Electronic device and storage medium.
[0046] The electronic device described above may include a processor, a memory, an input / output interface, and a communication interface. The memory stores program instructions for executing the steps of the method for distinguishing between human touch and electrical load detection based on multimodal sensing and a dual-branch neural network. After invoking the program instructions, the processor executes steps S1 to S5, thereby achieving the distinguishing detection and safety control of human touch and electrical load. The computer-readable storage medium described above may be flash memory, a solid-state drive, a read-only memory, or a random access memory. The program stored thereon, when executed by the processor, can also implement the above method.
[0047] Formula (1): H_n = (I_n / I_1) × 100%.
[0048] Where H_n represents the proportion of the nth harmonic, I_n represents the amplitude of the nth harmonic current, and I_1 represents the amplitude of the fundamental current.
[0049] Formula (2): P ≈ (1 / N) × Σ[u(n) · i(n)], n = 1, 2, ..., N.
[0050] Where P represents active power, N represents the number of sampling points in the integration window, u(n) represents the voltage value of the nth sampling point, and i(n) represents the current value of the nth sampling point.
[0051] Equation (3): PF = P / (U_rms × I_rms).
[0052] Where PF represents the power factor, P represents the active power, U_rms represents the effective voltage value, and I_rms represents the effective current value.
[0053] Formula (4): Z_1 = U_1 / I_1.
[0054] Where Z_1 represents the fundamental impedance, U_1 represents the fundamental voltage amplitude, and I_1 represents the fundamental current amplitude.
[0055] Equation (5): C_conf = (1 / M) × Σ[(C_m - C_base) / (C_max - C_base)].
[0056] Where C_conf represents the confidence level of the capacitance feature, M represents the number of capacitor arrays, C_m represents the m-th capacitor detection value, C_base represents the baseline capacitance, and C_max represents the maximum touch capacitance.
[0057] Equation (6): F_conf = (F - F_min) / (F_max - F_min).
[0058] Where F_conf represents the confidence level of the pressure feature, F represents the pressure detection value, F_min represents the minimum touch force, and F_max represents the maximum touch force.
[0059] Equation (7): H_conf = w_C × C_conf + w_F × F_conf.
[0060] Where H_conf represents the fusion touch confidence level, w_C represents the capacitance weight, and w_F represents the pressure weight, and w_C + w_F = 1. Preferably, M = 9, F_min is 50g, and F_max is 500g; in a standard environment, w_C can be 0.6 and w_F can be 0.4; under strong light, w_C can be appropriately lowered and w_F can be appropriately increased.
[0061] Equation (8): ΔI = I_L - I_N.
[0062] Where ΔI represents the difference between the live wire current and the neutral wire current, I_L represents the live wire current, and I_N represents the neutral wire current.
[0063] Equation (9): w_i = exp(alpha × S_i) / Σ exp(alpha × S_j).
[0064] Where w_i represents the attention weight of the i-th branch, S_i represents the confidence of the i-th branch, S_j represents the confidence of the j-th branch participating in the fusion, and alpha represents the attention adjustment factor. Optionally, alpha is set to 0.8.
Claims
1. A detection system for distinguishing between human touch and electrical loads based on multimodal sensing and a dual-branch design, characterized in that, The system includes a multimodal sensing module, a data preprocessing module, a dual-branch neural network inference module, a fusion decision module, and an execution control module. The multimodal sensing module collects capacitance, pressure, and environmental features related to human touch, as well as current, voltage, and power features related to the load. The data preprocessing module performs synchronous sampling, filtering and noise reduction, standardization, and feature construction on the data from each channel. The dual-branch neural network inference module includes a human body branch and a load branch. The human body branch infers human touch features and outputs a first confidence level, while the load branch infers load electrical features and outputs a second confidence level and / or load type result. The fusion decision module is used to perform attention fusion based on the first confidence level and the second confidence level to obtain the discrimination results of human touch danger state, normal load operation state or ambiguous state; The execution control module is used to output a control signal to cut off the power supply circuit and / or sound an alarm when the judgment result is a dangerous state of human touch.
2. The multimodal sensing and dual-branch human touch and load electrical appliance differentiation detection system according to claim 1, characterized in that, The multimodal sensing module includes at least an array-type capacitive sensing unit, a pressure sensing unit, a current sensing unit, a voltage sensing unit, a power factor / power detection unit, an environmental parameter sensing unit, and a temperature sensing unit; wherein, the array-type capacitive sensing unit is disposed in the vicinity of the live wire terminal and / or the neutral wire terminal, the environmental parameter sensing unit is used to collect at least one parameter among temperature, humidity, and light intensity, and each sensing unit is triggered by a unified clock for synchronous sampling.
3. The system for distinguishing and detecting human touch and load electrical appliances based on multimodal sensing and dual-branch technology as described in claim 1, characterized in that, The data preprocessing module is configured to: synchronously sample the signals of each channel at a sampling rate of 1kHz and form data frames with timestamps; perform notch filtering and wavelet denoising on the capacitance signal; perform moving average filtering or median filtering on the pressure signal; and perform frequency domain analysis based on Hanning window on the current signal and voltage signal to extract the effective value of current, effective value of voltage, phase difference, power factor, harmonic ratio, impedance value, peak value of impact current, and impact duration.
4. The multimodal sensing and dual-branch human touch and load electrical appliance differentiation detection system according to claim 1, characterized in that, The human body branch includes a convolutional neural network layer and a recurrent neural network layer. The inputs of the human body branch include at least capacitance change features, pressure change features, touch duration features, human body contact impedance features, environmental correction features, and current difference features. The human body branch is used to output a first confidence level related to human body touch.
5. The multimodal sensing and dual-branch human touch and load electrical appliance differentiation detection system according to claim 1, characterized in that, The load branch includes a convolutional neural network layer and a recurrent neural network layer. The input of the load branch includes at least harmonic ratio characteristics, power factor characteristics, phase difference characteristics, impedance characteristics, impedance temperature coefficient characteristics, inrush current characteristics, and load power characteristics. The load branch is used to output identification results corresponding to at least one type of resistive load, inductive load, and capacitive load.
6. The multimodal sensing and dual-branch human touch and load electrical appliance differentiation detection system according to claim 1, characterized in that, The fusion decision module includes an attention fusion unit and a timing analysis unit. The attention fusion unit assigns dynamic weights based on the first confidence level of the human branch output and the second confidence level of the load branch output, and generates the final decision result by combining the current difference threshold, power threshold and duration threshold. When the confidence level of human touch is higher than the first preset threshold and the current difference is higher than the second preset threshold, a human touch danger status is output. When the load confidence level is higher than the third preset threshold and the load power is higher than the fourth preset threshold, the normal working status of the load is output; when both the first confidence level and the second confidence level are in the fuzzy range, timing analysis is started to complete the second decision.
7. The multimodal sensing and dual-branch human touch and load electrical appliance differentiation detection system according to claim 1, characterized in that, The execution control module includes a drive unit, a switching unit, and an alarm unit; the drive unit is used to respond to the danger status signal of the fusion decision module, control the switching unit to cut off the live wire circuit and / or the neutral wire circuit, and control the alarm unit to output an audible and visual alarm; when any key sensing unit fails, the system switches to a redundant decision path composed of the remaining available features.
8. A method for distinguishing and detecting human touch and electrical loads based on multimodal sensing and a dual-branch neural network, characterized in that, include: S1 synchronously collects capacitance, pressure, environmental and current difference data related to human touch, as well as current, voltage and power data related to the load; S2 performs time alignment, filtering and denoising, normalization, and feature construction on the collected multimodal data; S3, input the human touch features into the human branch neural network and the load electrical appliance features into the load branch neural network to obtain the human touch recognition results and the load electrical appliance recognition results respectively; S4, perform attention fusion and temporal collaborative decision on the human touch recognition result and the load electrical appliance recognition result to obtain the final distinction result; S5, when the determination result is a dangerous situation of human contact, outputs power-off and / or alarm control commands.
9. An electronic device, characterized in that, It includes a processor, a memory, and a program stored in the memory and capable of running on the processor, wherein the program, when executed by the processor, implements the method of claim 8.
10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method of claim 8.