A lighting control device based on a CNN-LSTM hybrid model

Abstract

The application discloses a lighting control device based on a CNN-LSTM hybrid model, comprising: a radar sensing module for real-time monitoring of activity data of personnel in a target area, a data processing module comprising a feature extraction unit, a feature fusion unit, a behavior classification unit, a behavior prediction unit and an evaluation unit, the feature extraction unit extracts core features according to the activity data, the feature fusion unit outputs a core feature vector after fusing the features extracted by the feature extraction unit, the behavior classification unit adopts a CNN-LSTM hybrid model to output a global time sequence feature vector and identify personnel behavior classification, the behavior prediction unit adopts a bidirectional LSTM model to output a prediction result and a prediction confidence based on the global time sequence feature vector and spatial historical information, and the evaluation unit is configured to output the prediction result when the prediction confidence is greater than or equal to a preset confidence threshold; and a lighting control module matches a lighting mode according to the personnel behavior classification and the prediction result and generates a lighting control instruction.

Description

Technical Field

[0001] This invention relates to the field of intelligent building lighting automation control technology based on radar data, specifically to a lighting control device based on a CNN-LSTM hybrid model. Background Technology

[0002] The current technological development in the field of intelligent building lighting control has formed four core links: "sensing, communication, algorithm, and integration." Among them, the sensing layer adopts the collaborative application of multiple types of sensors, with the mainstream solution being "traditional sensors as the mainstay and radar sensors as the supplement." Core sensing technologies include: basic sensors: illuminance sensors (detecting ambient light intensity to achieve "constant illuminance control"), human infrared sensors (detecting the presence of people and triggering "lights on when people are present, lights off when people leave"), and temperature and humidity sensors (linking with the air conditioning system to optimize overall energy consumption); advanced sensors: millimeter-wave radar (some mid-to-high-end solutions use the 24GHz / 77GHz frequency band to solve the problem of infrared sensors being "susceptible to temperature / obstruction interference" and improve the detection accuracy of personnel position and movement direction), and ToF time-of-flight sensors (used in a few commercial complexes to detect personnel density and adapt to large-space lighting adjustment).

[0003] The algorithm layer is upgrading from "passive response" to "basic intelligence." Mainstream algorithms still focus on "rule-based control," while some mid-to-high-end solutions introduce basic AI capabilities, including basic algorithms such as timed control (presetting on / off times according to weekdays / weekends), constant illuminance algorithms (adjusting artificial lighting brightness based on natural light intensity), and simple occupancy control (only determining "whether there are people" without behavioral classification); advanced algorithms include energy consumption statistics algorithms based on big data and basic scene matching algorithms (presetting "meeting / office / cleaning" modes, with manual or semi-automatic switching). A few solutions attempt to optimize dimming strategies using simple machine learning models (such as decision trees), but have not achieved "behavior prediction" and "dynamic scene self-optimization."

[0004] Based on the analysis of existing technologies, all of the above solutions have certain limitations: insufficient algorithm depth, only implementing linear logic of "real-time detection → control", lacking a closed loop of "behavior classification → behavior prediction → dynamic scene self-optimization", and unable to adapt to the diversity of personnel behavior (such as differentiated handling of "brief departure" and "leaving after get off work" in the office area); weak hardware-software collaboration, with millimeter-wave radar only used as a "high-precision sensor", failing to fully explore the value of its multi-dimensional data of "speed, direction, and position" through algorithms, and the sensor's capabilities not being maximized; poor scene adaptability, with control strategies relying on manual presets, unable to automatically adjust according to personnel habits (such as employees in a certain office area "often get up and move around at 3 pm") or environmental changes (such as a sudden drop in natural light on a cloudy day), resulting in low intelligence. Summary of the Invention

[0005] To address at least one of the aforementioned technical problems, embodiments of the present invention provide a lighting control device based on a CNN-LSTM hybrid model, comprising: a radar sensing module for real-time monitoring of human activity data within a target area, the activity data including position, direction of movement, and speed; and a data processing module including a feature extraction unit, a feature fusion unit, a behavior classification unit, a behavior prediction unit, and an evaluation unit. The feature extraction unit extracts motion features, temporal features, frequency domain features, and spatial features based on the activity data. The feature fusion unit sequentially standardizes, assigns feature weights to, and performs dimensionality reduction using principal component analysis on the features extracted by the feature extraction unit, outputting a core feature vector. The behavior classification unit employs a CNN-LSTM hybrid model. A CNN network extracts personnel grid distribution information and determines the relative position information of personnel and lighting fixtures based on the personnel grid information, generating temporal spatial feature vectors. An LSTM network is used to output global temporal feature vectors based on the temporal feature sequence and identify personnel behavior classification. The behavior prediction unit uses a bidirectional LSTM model to output prediction results and prediction confidence based on global temporal feature vectors and spatial history information. The prediction results include behavior classification probability, prediction trend, and spatial location. The evaluation unit is configured to output prediction results when the prediction confidence is greater than or equal to a preset confidence threshold. The lighting control module includes multiple lighting modes. The lighting control module matches the lighting mode according to the personnel behavior classification and prediction results and generates lighting control commands.

[0006] Preferably, the data processing module further includes a preprocessing unit, which filters the activity data based on the speed and removes activity data with a speed greater than a speed threshold, aligns the activity data according to a preset time window, and standardizes the activity data.

[0007] Preferably, the step of the behavior prediction unit outputting the prediction confidence score includes: calculating the feature consistency score of the radar data. ,in, For time continuity values, For time continuity weights, This is the spatial consistency value. For spatial consistency weights;

[0008] Based on the primary behavior with the highest probability and the secondary behavior with the second highest probability in the behavior classification, calculate the probability percentage of the primary behavior. ,in, The probability value of the main behavior. The probability value for this action;

[0009] Calculate the concentration of the probability distribution of behaviors. Where H is the information entropy of the 5-dimensional probability. For the i-th behavior classification, The probability of classifying the i-th behavior;

[0010] Information entropy normalization ,in Given that the probabilities of the five types of behaviors are equal, the concentration is... ;

[0011] Calculate the prediction confidence level based on preset weights. ,in, , and These are the confidence weights for the proportion of the main behavior probability, the concentration of the behavior probability distribution, and the feature consistency score, respectively.

[0012] Preferably, the CNN network of the CNN-LSTM hybrid model includes a first convolutional layer, a max pooling layer, a second convolutional layer, a global average pooling layer, a flattening layer, and a fully connected layer.

[0013] Preferably, the LSTM network of the CNN-LSTM hybrid model includes an input layer, a first forward hidden layer, a second forward hidden layer, a first fully connected layer, and a second fully connected layer.

[0014] Preferably, the training steps of the CNN-LSTM hybrid model include: collecting radar data and adding scene and behavior labels to the radar data; preprocessing, extracting features, and representing tensors in the labeled radar data; and partitioning and augmenting the radar data dataset. The model training convergence conditions include loss function convergence conditions, classification accuracy convergence conditions, and early stopping mechanism convergence conditions. The loss function conditions include a training set loss function convergence threshold and a validation set loss function convergence threshold. The classification accuracy convergence conditions include overall behavior classification accuracy, key behavior classification accuracy, and single-class behavior classification accuracy. The early stopping mechanism convergence conditions include a validation set traversal threshold, a training set and validation set classification accuracy threshold, and a training epoch threshold.

[0015] Preferably, the behavior prediction unit is configured to output prediction results according to a preset step size.

[0016] Preferably, the preset step size includes multiple prediction step sizes, which form an arithmetic sequence, and the forward LSTM fusion coefficients of the multiple prediction step sizes decrease sequentially, while the reverse LSTM fusion coefficients increase sequentially.

[0017] Preferably, the lighting control commands include on / off status, brightness, and color temperature.

[0018] Preferably, the radar sensing module adopts a multi-antenna array distribution.

[0019] The lighting control device based on the CNN-LSTM hybrid model in this invention has the following technical effects: By fusing multimodal features of perceived human behavior data, it overcomes the limitations of traditional single-feature (such as only movement speed, only position) perception. By fusing motion features (speed / trajectory), spatial features (grid coordinates / person density), frequency domain features (activity frequency), and temporal features (staying time / time distribution), combined with dynamic weight allocation and PCA dimensionality reduction, it achieves high-precision extraction of human behavior data in complex building environments (strong light, occlusion, equipment interference), improves anti-interference ability, and provides highly reliable data support for subsequent behavior classification and prediction.

[0020] The behavior classification algorithm based on a CNN-LSTM hybrid model addresses the problems of insufficient spatial feature extraction and low accuracy in distinguishing complex behaviors in existing technologies. By using CNN to extract spatial distribution patterns of people (such as being stationary at workstations or clustered in meeting rooms) and LSTM to capture time-series dynamics (such as entry / exit trajectories), it achieves accurate classification of five types of behaviors: "entry, departure, stationary, movement, and brief stay," with a classification accuracy of over 95%. The accuracy in distinguishing between "stationary vs. brief departure" is improved to 92%, avoiding the problems of traditional systems such as "mistakenly turning off lights while sitting still" and "misjudging departure when briefly standing up."

[0021] By employing a bidirectional LSTM-driven behavior prediction mechanism, the limitations of existing lighting systems—which only offer real-time response and lack proactive control—are overcome. Forward LSTM analyzes historical movement trends (current window time-series data), while backward LSTM combines historical scene patterns (current window time-series data and historical scene probability distribution data). This allows for the prediction of human behavior (such as "about to enter a certain area" or "about to leave the workstation") 0.5-2 seconds in advance. When the prediction confidence level is ≥85%, pre-adjustment of the lighting is triggered, avoiding the delay problem of "lights only turning on after people have arrived," thus improving user experience satisfaction and reducing ineffective lighting time, thereby enhancing energy efficiency.

[0022] The lighting control logic, optimized by dynamic scenarios, differs from traditional "fixed-mode dimming" (such as adjusting only by time or presence of people). Based on behavior classification and prediction results, it automatically adapts to scenario requirements (such as cool white light during the day and warm white light in the evening in office areas, and adjusting the lighting coverage in meeting rooms according to the number of people). It also supports online learning (fine-tuning parameters when scenarios change) without the need for manual reconfiguration, improving system adaptability and balancing energy saving (energy consumption reduced by 25%-30%) with comfort (lighting parameters match people's behavioral habits).

[0023] Through the collaborative adaptation design of millimeter-wave radar and software algorithms, it is not simply about adding millimeter-wave radar hardware. Instead, it optimizes hardware selection (24GHz band radar, angular resolution ≤5°) and data sampling rate (above 10Hz) according to algorithm requirements, ensuring the real-time performance of multimodal feature extraction (data processing latency ≤100ms). At the same time, it achieves seamless connection between "perception-algorithm-control" through ZigBee low-latency communication (latency ≤50ms), avoiding the problem of "accurate algorithm but lagging control" caused by hardware and software decoupling, and improving the overall response efficiency of the system. Attached Figure Description

[0024] To better understand the above and other objects, features, advantages, and functions of the present invention, reference can be made to the embodiments shown in the accompanying drawings. The same reference numerals in the drawings refer to the same parts. Those skilled in the art should understand that the drawings are intended to schematically illustrate preferred embodiments of the invention and do not limit the scope of the invention in any way; the parts in the drawings are not drawn to scale.

[0025] Figure 1 The diagram shown is a schematic of the behavior prediction process of a lighting control device based on a CNN-LSTM hybrid model according to an embodiment of this application.

[0026] Figure 2 The diagram shown is a schematic representation of the data preprocessing flow of the preprocessing unit in an embodiment of this application.

[0027] Figure 3 The diagram shown is a schematic diagram of the feature extraction process of the feature extraction unit in an embodiment of this application;

[0028] Figure 4 The diagram shown is a schematic representation of the feature fusion process of the feature fusion unit in an embodiment of this application.

[0029] Figure 5 The diagram shown is a schematic representation of the behavior classification and recognition process of the behavior classification unit in an embodiment of this application.

[0030] Figure 6 The diagram shows a behavior prediction process of the behavior prediction unit in an embodiment of this application. Detailed Implementation

[0031] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0032] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0033] To at least partially address one or more of the aforementioned problems and other potential issues, embodiments of this disclosure propose an embodiment of the present invention that provides a lighting control device based on a CNN-LSTM hybrid model, comprising: a radar sensing module, a data processing module, and a lighting control module.

[0034] like Figure 1 As shown, the radar sensing module is used to monitor the activity data of people within the target area in real time. This activity data includes location, direction of movement, and speed. In some embodiments, the radar sensing module employs a multi-antenna array distribution. This array distribution design improves detection accuracy and coverage.

[0035] The data processing module includes a feature extraction unit, a feature fusion unit, a behavior classification unit, a behavior prediction unit, and an evaluation unit. The feature extraction unit extracts motion features, temporal features, frequency domain features, and spatial features from the activity data. The feature fusion unit standardizes the features extracted by the feature extraction unit, assigns feature weights, and performs dimensionality reduction using principal component analysis to output a core feature vector. The behavior classification unit uses a CNN-LSTM hybrid model to extract personnel grid distribution information and determines the relative positions of personnel and lighting fixtures based on the grid information, generating a temporal spatial feature vector. An LSTM network is then used to output a global temporal feature vector based on the temporal feature sequence and to identify personnel behavior classifications. The behavior prediction unit uses a bidirectional LSTM model to output prediction results and prediction confidence based on the global temporal feature vector and spatial historical information. The prediction results include behavior classification probability, prediction trend, and spatial location. The evaluation unit is configured to output prediction results when the prediction confidence is greater than or equal to a preset confidence threshold.

[0036] The lighting control module includes multiple lighting modes. The lighting control module matches the lighting modes based on the classification and prediction results of human behavior and generates lighting control instructions.

[0037] As will be understood by those skilled in the art, the lighting control device also includes a communication module and a power management module. The communication module uses wireless communication technology (such as ZigBee, Wi-Fi or LoRa) to realize interconnection between device modules and supports remote interaction with building management system (BMS) or user terminal. The power management module and the lighting control module receive lighting control commands.

[0038] Specifically, the radar sensing module includes multiple millimeter-wave radars, which are designed with a multi-antenna array. The radar angular resolution is less than or equal to 5°, and the radar sampling rate is 10Hz. The multiple millimeter-wave radars of the radar sensing module are configured to synchronously collect activity data, including position (x / y grid coordinates), radial velocity, echo signal strength, and direction of movement.

[0039] In some embodiments, the data processing module further includes a preprocessing unit, which filters the activity data based on the speed in the activity data, removes activity data with a speed greater than a speed threshold, aligns the activity data according to a preset time window, and standardizes the activity data.

[0040] Specifically, as shown in Figure 2, the preprocessing unit receives raw activity data collected by the radar sensing module. It evaluates the velocity in the raw activity data; for example, a velocity threshold of 5 m / s is used. If the velocity in the activity data exceeds the velocity threshold, the activity data is deleted. This avoids interference from non-personnel movement data. In some embodiments, a preset time window is set to 10 seconds, and the multi-dimensional raw data is aligned according to this 10-second time window. In other embodiments, the duration of the preset time window can be adjusted according to actual needs. Finally, the activity data is standardized, converting the different dimensions of the activity data into a 0-1 range.

[0041] like Figure 3 As shown, the feature extraction unit of the data processing module extracts motion features based on the Doppler effect of millimeter-wave radar. It collects the radial velocity (unit: m / s), direction of movement (angle value with the radar installation point as the origin, range 0°-360°) and trajectory of personnel in real time. It filters out "non-person movement interference" (such as the low-speed irregular movement of curtains blown by the wind, and the high-frequency small-amplitude movement of equipment vibration), and retains the personnel movement data with "velocity 0.1-2m / s and continuous trajectory".

[0042] The feature extraction unit extracts spatial features and divides the building into 1m×1m grid cells (the accuracy can be dynamically adjusted according to the area size). The multi-antenna array of millimeter-wave radar (24GHz band, angular resolution ≤5°) locates the coordinates of people in the grid. At the same time, it records the "distance between people and lighting fixtures" (for subsequent local lighting adjustment) and the "number of people in the same grid" (for density judgment, such as needing to turn on the lights in the entire conference room when many people are gathered).

[0043] The feature extraction unit extracts frequency domain features by performing Fourier transform on the original radar echo signal to extract the frequency features of human activities (such as "still breathing" corresponding to low frequency signals of 0.2-0.5Hz, "walking" corresponding to mid frequency signals of 1-3Hz, and "rapid movement" corresponding to high frequency signals of 3-5Hz). Through frequency threshold filtering, "interference from inanimate objects" (such as the fixed frequency signals of elevator operation) are excluded.

[0044] The feature extraction unit extracts time features, using 10-second time windows to record the duration of people's stay in each grid cell (e.g., "a person stays in a workstation grid for more than 30 minutes = stationary work") and the distribution of activity time (e.g., "the office area is crowded from 9:00 to 12:00 on weekdays, and the number of people decreases from 12:00 to 13:00"), providing historical data support for subsequent behavior prediction.

[0045] like Figure 4 As shown, the feature fusion unit adopts a "weighted fusion + feature filtering" mechanism to avoid misjudgments caused by single feature bias. Specifically, firstly, the features of each dimension are standardized (e.g., data of different dimensions such as speed, position, and frequency are converted into the 0-1 range); then, the weights are dynamically allocated according to the building scenario (e.g., in the office area, the weight of "spatial features + time features" accounts for 60%, prioritizing the determination of whether there are people at the workstations; in the corridor, the weight of "motion features + frequency domain features" accounts for 70%, prioritizing the identification of personnel movement); finally, dimensionality reduction is achieved through PCA (principal component analysis), retaining the core features that contribute the most to "behavior recognition," reducing data redundancy, and improving the real-time performance of the algorithm (processing latency ≤100ms, meeting the immediacy requirements of lighting control).

[0046] The CNN-LSTM hybrid model for behavior classification extracts spatial feature correlations through the CNN network and captures dynamic changes in time series through the LSTM network. For example, the CNN network uses 3×3 convolutional kernels to identify the "distribution pattern of people in the grid" (e.g., "single stationary workstation" corresponds to "high activation value of a single grid," and "multi-person meeting" corresponds to "high activation value of continuous grids"); it compresses data dimensionality through pooling layers, retaining key spatial information such as the "relative relationship between people and lighting positions," providing a basis for subsequent "local lighting adjustment." The LSTM network uses gating units (input gate, forget gate, output gate) to memorize the "continuous movement trajectory of people" (e.g., the continuous path "from elevator entrance to workstation" corresponds to "entry behavior," and "from workstation to elevator entrance" corresponds to "departure behavior"); it solves the problem of "long-term dependence," such as identifying the behavior sequence "people stay in the meeting room for 10 minutes and then get up and leave," avoiding misjudging "briefly getting up to pick up documents" as "leaving."

[0047] Specifically, such as Figure 5 As shown, for the baseline input feature dimension of the CNN-LSTM hybrid model, after multimodal feature fusion and PCA dimensionality reduction, a standardized spatial feature matrix of 1×32×32 is output (corresponding to a 1m×1m grid of a building, with the 32×32 grid covering an area of ​​1024㎡, suitable for typical building spaces such as office areas / corridors / meeting rooms). Simultaneously, a temporal feature sequence with a time step of T=10 and a feature dimension of F=64 is generated (T=10 corresponds to a 10-second time window, and F=64 is the core feature dimension after fusing motion / space / frequency domain / time). For the input data format, the CNN input is a four-dimensional tensor [Batch, Channel, Height, Width] (Batch=16, batch size; Channel=1, single-channel spatial features); the LSTM input is a three-dimensional tensor [Batch, Time_Step, Feature]; ReLU is preferentially used as the activation function for all layers (to avoid gradient vanishing and improve computational speed), and the LSTM output layer uses Sigmoid (for multi-class classification tasks, outputting 0-1 probability values). For the design of hybrid models, the number of convolutional layers should not exceed 3 and the number of LSTM hidden layers should not exceed 2, adapting to edge computing devices for building lighting control (such as embedded gateways and low-computing-power MCUs), with no redundant layers.

[0048] In some embodiments, the CNN network of the CNN-LSTM hybrid model includes a first convolutional layer, a max pooling layer, a second convolutional layer, a global average pooling layer, a flattening layer, and a fully connected layer.

[0049] Specifically, the core function of the CNN network is to extract local correlations from a 32×32 rasterized spatial feature matrix, identify the distribution patterns of people in the building grid (such as grid activation features of people sitting still at a single workstation, people gathering in a meeting room, and people moving in a corridor), and preserve the relative positional relationship between people and lighting fixtures, providing a spatial basis for subsequent local lighting adjustment; the output is a temporal spatial feature sequence, which is directly connected to the LSTM network. The input data format of the CNN network is a four-dimensional tensor [16, 1, 32, 32] (Batch=16, Channel=1, Height=32, Width=32), and the overall output format is a three-dimensional tensor [16, 10, 64] (matching the time step T=10 and feature dimension F=64 of the LSTM input).

[0050] The core parameters of the first convolutional layer are set as follows: 3×3 convolutional kernel, number of kernels 16, stride 1, padding Same. This is used to extract basic local features of the grid space (such as the activation value of people in a single grid and the correlation features between two adjacent grids). ReLU activation introduces non-linearity to avoid linear superposition of features.

[0051] The core parameters of the max pooling layer are set as follows: 2×2 pooling kernel, stride 2, and padding Same. This is used to compress data dimensionality, reduce computation, retain key spatial features (such as core features of grid areas where people gather), and filter redundant details (such as weak interference at grid edges).

[0052] The core parameters of the second convolutional layer are set as follows: 3×3 convolutional kernel, number of kernels 32, stride 1, padding Same. This is used to extract high-order correlation features in the grid space (such as the spatial distribution of personnel movement trajectories in continuous grids and the features of grid clusters where multiple people gather), thereby enhancing the spatial feature representation of personnel behavior.

[0053] The core parameters of the global average pooling layer are set to: 1×1 pooling kernel and stride 1. This is used to perform global pooling on the entire spatial feature matrix, generate spatial feature vectors, eliminate local biases in grid positions, and preserve the global spatial distribution pattern.

[0054] The flattening layer, without parameter settings, flattens by channel and is used to convert a four-dimensional tensor into a two-dimensional feature vector, thus adapting the dimension for subsequent fully connected layers.

[0055] The core parameters of the fully connected layer are set as follows: 640 neurons and ReLU activation, which are used to improve the dimensionality of spatial features. The two-dimensional vector is reshaped into a three-dimensional temporal feature matrix by Reshape, which matches the input format of the LSTM network (Time_Step=10, Feature=64).

[0056] In some embodiments, the LSTM network of the CNN-LSTM hybrid model includes an input layer, a first forward hidden layer, a second forward hidden layer, a first fully connected layer, and a second fully connected layer.

[0057] Specifically, the core function of the LSTM network is to receive the temporal spatial feature sequence output by the CNN network, capture the dynamic changes in human behavior over time (such as the continuous movement trajectory from the elevator entrance to the workstation, the behavior sequence of being stationary at the workstation → briefly getting up → returning), solve the problem of long temporal dependencies through gating units, and ultimately achieve accurate classification of five types of building human behavior: entering, leaving, remaining stationary, moving, and briefly staying. It forms a dual fusion of "spatial + temporal" features with the CNN, improving the accuracy of behavior classification. The input of the LSTM network is a three-dimensional tensor [16, 10, 64] (CNN network output, Batch=16, Time_Step=10, Feature=64), and the output data is a two-dimensional tensor [16, 5] (Batch=16, 5 is the number of behavior classification categories, and the output is the 0-1 probability value of each category).

[0058] The input layer of the LSTM network has no parameter settings and is configured as a temporal feature map to receive the temporal spatial features output by the CNN, complete feature format matching, and provide the original temporal data for the LSTM hidden layer.

[0059] The core parameters of the first forward hidden layer are set as follows: number of hidden units 128, return sequence True, dropout=0.2, which are used to capture positive time series features of human behavior (such as changes in movement trajectory over the past 10 seconds, changes in grid dwell time), and return a complete sequence to provide time series data for the next LSTM layer; at the same time, dropout=0.2 is set to prevent overfitting (to adapt to the limited sample size in building scenarios).

[0060] The core parameters of the second forward hidden layer are set as follows: 64 hidden units, False return sequence, and dropout=0.2. This is used to further refine the forward temporal features and output a global temporal feature vector (discarding intermediate time steps and retaining the overall behavior pattern of the 10-second window). The number of hidden units is halved to reduce the amount of computation and adapt to edge computing.

[0061] The core parameters of the first fully connected layer are set as follows: 32 neurons and ReLU activation. This is used to perform dimensionality compression and nonlinear transformation on the global temporal features extracted by LSTM, enhance the expressive power of core behavioral features, and filter redundant temporal information.

[0062] The core parameters of the second fully connected layer are set as follows: 5 neurons and sigmoid activation, which are used to map the feature vector to a probability distribution (0-1) of 5 types of behavior, such as [0.95,0.02,0.01,0.01,0.01] corresponding to "static" behavior, providing a quantitative basis for subsequent behavior classification decisions.

[0063] The CNN-LSTM hybrid model achieves dimensionality transformation from "spatial feature matrix" to "temporal feature sequence" through fully connected reshape layers, eliminating the need for additional feature mapping layers, ensuring the continuity of feature transmission, and reducing data loss; the LSTM input layer directly connects to the CNN output, achieving seamless fusion of "spatial features + temporal features".

[0064] The output of the CNN-LSTM hybrid model is a 5-dimensional probability vector. The category corresponding to the maximum value is taken as the final behavior result. For example:

[0065] Probability vector [0.95, 0.02, 0.01, 0.01, 0.01] → Decision: Enter;

[0066] Probability vector [0.03, 0.96, 0.00, 0.00, 0.01] → Decision: Leave;

[0067] Probability vector [0.01, 0.02, 0.94, 0.02, 0.01] → Decision: stationary;

[0068] Probability vector [0.02, 0.01, 0.03, 0.93, 0.01] → Decision: Move.

[0069] Probability vector [0.01, 0.01, 0.05, 0.02, 0.91] → Decision: Short stay.

[0070] The connection between the CNN-LSTM hybrid model and the subsequent bidirectional LSTM model is achieved by using the 64-dimensional global temporal feature vector [16,64] of the second forward hidden layer as the input of the bidirectional LSTM model, which preserves the core temporal features of behavior classification, realizes the feature reuse of "classification-prediction", and improves the overall efficiency of the model.

[0071] like Figure 6As shown, the behavior prediction unit uses a bidirectional LSTM model to output prediction results and prediction confidence based on global temporal feature vectors and spatial historical information. By simultaneously predicting from historical data (forward LSTM) and looking back from future data (backward LSTM) using the bidirectional LSTM structure, it comprehensively judges personnel behavior trends. For example, forward LSTM analyzes "personnel moved from the elevator entrance to their workstation in the past 10 seconds, with a stable speed of 0.5 m / s," and initially predicts "they will continue to move towards their workstation." Backward LSTM combines "historical data of the workstation area (90% of personnel stayed at their workstations during this period)" to further confirm "personnel will remain stationary at their workstations," and finally outputs the prediction result. In some embodiments, the bidirectional LSTM model is configured to output prediction results at "0.5 seconds / step" (e.g., "personnel will enter workstation grid number 2 in the next 0.5 seconds and will remain stationary in the next 1 second"). Only when the prediction confidence is ≥85% is a command sent to the lighting control module (to avoid erroneous adjustments caused by low-confidence predictions), such as "predicting that personnel will enter the end of the corridor," and turning on the lights in that area 0.5 seconds in advance. In some embodiments, the behavior prediction unit is configured to output a prediction result based on a preset step size. In some embodiments, the preset step size includes multiple prediction step sizes, which are arranged in an arithmetic sequence, and the forward LSTM fusion coefficients of the multiple prediction step sizes decrease sequentially, while the reverse LSTM fusion coefficients increase sequentially.

[0072] Specifically, the input to the bidirectional LSTM model is a 64-dimensional global temporal feature vector (integrating motion / spatial / frequency / temporal features) output from the second feedforward hidden layer of the CNN-LSTM hybrid model; in the temporal dimension, the input is the temporal features within a 10-second time window (denoted as t). -9 -t0, a total of 10 time steps, where t0 is the current time), the prediction step size is 0.5 seconds / step, and the output is 0.5 / 1 / 1.5 / 2 seconds in the future (denoted as t). +0.5 -t +2 Personnel behavior is analyzed using a total of 4 prediction steps; 5 types of behavior (entering / leaving / stationary / moving / short stay) are represented by 5-dimensional one-hot encoding; the tensor is uniformly [Batch, Time_Step, Feature], where Batch=16 (adapting to the computing power / memory of building edge computing devices), Time_Step is the number of time steps, and Feature is the feature dimension; historical workstation data is pre-statistical building grid / workstation behavior pattern data, stored in a local feature library, and called in real time according to the location of personnel, serving as the exclusive feature input for the inverse LSTM.

[0073] Forward LSTM captures the dynamic trends of continuous human movement (such as movement trajectory, speed / direction changes) based solely on the real-time temporal features of the current detection window. Without the intervention of historical scene data, it is a preliminary behavior prediction driven purely by real-time data, providing the basic prediction results for inverse LSTM.

[0074] The input vector Xforward∈R of the forward LSTM [16,10,64] Batch=16, 10 time steps (t -9 -t0), 64-dimensional core features; each row corresponds to a 64-dimensional feature at a time step, reflecting the person's motion / spatial / frequency domain / temporal state at that moment (e.g., t). -1 The movement speed at any given time, the grid coordinates, the activity frequency, etc., constitute the global temporal feature vector. The output vector Yforward∈R [16,4,5] Batch=16, 4 prediction steps (t) +0.5 -t +2 A 5-dimensional probability vector of behaviors; each output element Yforward[i,j,k] represents the probability (0-1 interval) of occurrence of the i-th sample, the j-th prediction step, and the k-th behavior, such as [0.02,0.01,0.95,0.01,0.01] representing t +0.5 The probability of it being "still" at any moment is 95%.

[0075] The inverse LSTM uses real-time time-series features and historical behavior data of workstations / areas as dual inputs to perform scenario-based verification, correction and optimization of the preliminary prediction results of the forward LSTM, solving the misjudgment problem of pure real-time prediction and outputting behavior prediction results that are more in line with the actual building scenario.

[0076] The underlying tensor representation of the real-time input to the inverse LSTM is Xbackward_base∈R. [16,10,64] The input is exactly the same as that of the forward LSTM, which is t. -9 The 10-step × 64-dimensional real-time core features of -t0, i.e., the global temporal feature vector, ensure the consistency of the dual-branch basic data and provide a real-time benchmark for behavior verification. The historical feature vector H∈R for the workstation. [16,10,16] Batch=16, 10 time steps, 16-dimensional workstation / area historical features used to represent spatial historical information; structured features representing the historical behavioral patterns of the current personnel's grid / workstation, obtained from long-term building sensing data statistics, dynamically retrieved according to personnel location. The final input is concatenated as Xbackward = Concat(Xbackward_base,H)∈R. [16,10,80]The 64-dimensional real-time features and 16-dimensional historical features are concatenated along the feature dimension to serve as the final input to the inverse LSTM, achieving the fusion of "real-time data + historical patterns". The output vector of the inverse LSTM is Ybackward∈R. [16,4,5] Batch=16, 4 prediction steps, 5-dimensional behavior probability vector; it is a corrected prediction result after fusing historical patterns of the scene. The tensor dimension is completely consistent with the output of the forward LSTM, which facilitates subsequent dual-branch fusion and the probability distribution is more in line with the actual behavior patterns of the building.

[0077] The workstation historical feature vector H used to represent spatial historical information is a lightweight feature pre-statistic by grid / workstation, time period, and weekday / weekend, with no redundant dimensions. It directly serves the behavior correction of the inverse LSTM and specifically includes grid behavior base probability, time period / scene attribute features, and behavior transfer historical probability. The grid behavior base probability includes 5 dimensions, including the historical occurrence probability of 5 types of behavior of the grid in the current time period (e.g., 9:00-9:30, 14:00-14:30) (e.g., stationary 90%, moving 5%, short stay 5%, entering 0%, leaving 0%); the time period / scene attribute features include 3 dimensions, time period label (1-dimensional, morning / noon / evening / night), weekday / weekend label (1-dimensional), and the activity level of people in the grid during the time period (1-dimensional, 0-1 normalized, reflecting the frequency of people appearing during the time period); the behavior transfer history probability includes 8 dimensions, the 8 core behavior transfer rules of the grid (e.g., stationary → short stay, short stay → stationary, moving → entering, moving → leaving, entering → stationary, etc.), which are the core basis for the reverse LSTM to correct behavior.

[0078] The specific mechanism by which historical workstation data H affects the output of the inverse LSTM is through dynamic weight modulation achieved by the gating units (input gate / forget gate / output gate) of the inverse LSTM. This allows for scenario-based filtering and enhancement of real-time features, ultimately correcting the behavior probability distribution. The specific process is as follows:

[0079] a) Forgotten Gate: Weakening subtle real-time features that contradict historical patterns.

[0080] If historical data shows that a certain behavior (such as "stationary") occurs with a very high probability in a certain grid, the forget gate will reduce the weight of weak interference signals that contradict this behavior in the real-time features (such as the "movement" feature of slight limb movements of a person, invalid features caused by equipment vibration), reduce meaningless feature inputs, and avoid misjudgment.

[0081] For example, if a workstation has a 90% historical static probability, the subtle "movement" of a person briefly getting up in real time will be weakened by the forget gate, reducing the probability of it being misjudged as "leaving".

[0082] b) Input gate: Enhance real-time features that match historical transfer patterns.

[0083] If historical data shows that a certain grid exhibits a typical behavior transition pattern (such as "moving → brief pause → stationary" or "entering → stationary"), the input gate will increase the weight of signals in the real-time features that match this transition pattern (such as features where the person's movement speed slows down or the grid position approaches the workstation), thus reinforcing the correct behavior trend features.

[0084] Example: When a person moves from the elevator entrance to their workstation (real-time "movement" feature), and the historical pattern of the workstation is "movement → stationary", the input gate will strengthen the weight of this "movement" feature, guiding the model to predict "stationary" rather than other behaviors.

[0085] c) Output gate: redistribute behavior probabilities based on historical patterns.

[0086] The output gate of the inverse LSTM redistributes the probability of the prediction results of the hidden layer based on the behavior probability distribution of historical data, tilting the probability towards behaviors with high historical probability, and finally outputting a corrected 5-dimensional behavior probability vector.

[0087] Example: A forward LSTM predicts a 70% probability that an employee at a certain workstation will "leave," but the historical probability of "leaving" at that workstation is only 5%. The output gate will redistribute the probability, reducing the probability of "leaving" and increasing the probability of "staying briefly."

[0088] The core principle of fusing the forward and backward LSTM outputs of a bidirectional LSTM model is: the forward real-time trend is used as the basis, and the backward scene correction is used as the weights, with the fusion coefficients dynamically allocated according to the prediction step size. The nearest prediction step size (e.g., t) +0.5 The focus is on positive real-time data (short-term human behavior is dominated by real-time trajectories), and the long-term prediction step size (e.g., t) +2 It focuses on reverse historical data (long-term human behavior is dominated by scene patterns) and finally generates a unified prediction result after fusion, taking into account both real-time performance and scene fit.

[0089] For example, for 4 prediction step sizes (t) +0.5 / t +1 / t +1.5 / t +2 A fusion coefficient (α for forward LSTM fusion and β for reverse LSTM fusion) is set that varies linearly with the prediction step size, satisfying α+β=1, to suit the "short-term dynamics, long-term patterns" characteristic of human behavior. For the prediction step size t... +0.5 The forward LSTM fusion coefficient is 0.8, the reverse LSTM fusion coefficient is 0.2, and the prediction step size t +0.5 For short-term predictions, human behavior is primarily driven by real-time trajectories, with only minor adjustments made to historical data; the prediction step size is t. +1The forward LSTM fusion coefficient is 0.6, the reverse LSTM fusion coefficient is 0.4, and the prediction step size t +1 For medium-term forecasting, real-time trajectories and historical patterns jointly influence behavioral trends; the forecasting step size is t. +1.5 The forward LSTM fusion coefficient is 0.4, the reverse LSTM fusion coefficient is 0.6, and the prediction step size t +1.5 For medium- to long-term forecasting, historical patterns have a greater influence on behavioral trends than real-time trajectories; the forecasting step size is t. +2 The forward LSTM fusion coefficient is 0.2, the reverse LSTM fusion coefficient is 0.8, and the prediction step size t +2 For long-term prediction, human behavior closely follows historical patterns of the scenario, and real-time trajectories are only used as supplementary references.

[0090] In some embodiments, the step of the behavior prediction unit outputting the prediction confidence score includes: calculating the feature consistency score of the radar data. ,in, For time continuity values, For time continuity weights, This is the spatial consistency value. For spatial consistency weights;

[0091] Based on the primary behavior with the highest probability and the secondary behavior with the second highest probability in the behavior classification, calculate the probability percentage of the primary behavior. ,in, The probability value of the main behavior. The probability value for this action;

[0092] Calculate the concentration of the probability distribution of behaviors. Where H is the information entropy of the 5-dimensional probability. For the i-th behavior classification, The probability of classifying the i-th behavior;

[0093] Information entropy normalization ,in Given that the probabilities of the five types of behaviors are equal, the concentration is... ;

[0094] Calculate the prediction confidence level based on preset weights. ,in, , and These are the confidence weights for the proportion of the main behavior probability, the concentration of the behavior probability distribution, and the feature consistency score, respectively.

[0095] Specifically, the prediction confidence is a quantified value of the reliability of the fused prediction result at a single prediction step (value from 0 to 1, with the document requiring ≥85% to trigger a lighting control command). The core calculation logic is: the proportion of the main behavior probability + the concentration of the behavior probability distribution + the consistency of real-time features. All three calculations are normalized to 0 to 1, with no subjective thresholds, resulting in objective results.

[0096] Preliminary definition (single prediction step size): For the fusion result Yfusion[i,j,:] (a 5-dimensional behavioral probability vector for a certain sample and a certain prediction step size), three core parameters are defined:

[0097] Main behavior probability P max The maximum value in the 5-dimensional probability is the predicted probability of the "most likely behavior" (e.g., Pmax=0.94 for [0.02,0.01,0.94,0.02,0.01]).

[0098] The probability P of this action sub The probability value second only to Pmax in the 5-dimensional probability, reflecting the degree of dispersion of the probability distribution;

[0099] Feature consistency score S cons The temporal continuity and spatial consistency normalization score (0~1) of the current 10-second real-time multimodal features is calculated in real time by the multimodal feature extraction module. It is close to 1 when there is no effective interference, and decreases when there is noise / occlusion / feature jump.

[0100] Step 1: Calculate the feature consistency score S cons (Base score, confidence weight 30%), S cons The reliability of millimeter-wave radar sensing data is the basis for confidence score calculation, which is obtained by weighted summation of temporal continuity and spatial consistency. For example, if the weight of temporal continuity is 0.6 and the weight of spatial consistency is 0.4, then the feature consistency score is: S cons =0.6×S time +0.4×S space .

[0101] Time continuity S time : This is the normalized value of the inter-frame change rate of the features in the current 10 seconds. The more continuous the trajectory / velocity / frequency, the smaller the change rate. S time The closer to 1; in some embodiments, the trajectory, velocity, and frequency change rate are first normalized by minimum-maximum or Z-score, and then the weights are set as needed for fusion.

[0102] Spatial consistency S space S: The normalized value of the correlation between the characteristics of the grid where a person is located and the surrounding grids. The higher the correlation between location and behavior, the higher the correlation. spaceThe closer to 1. In some embodiments, the fit between the grid features and the scene is first measured by location-behavior matching to ensure that "the higher the matching degree, the higher the value"; neighborhood correlation weighting is introduced to make the values ​​of adjacent grids transition smoothly and eliminate spatial jumps; finally, the "center grid feature + neighborhood smooth feature + matching degree weight" is normalized and fused, and mapped to [0,1] to take into account both spatial continuity and behavioral rationality.

[0103] Step 2: Calculate the probability percentage of the main behavior C1 (core score, confidence weight 40%), reflecting the probability advantage of the "most likely behavior", P. max The larger, P sub The smaller the value, the closer C1 is to 1. The formula is:

[0104] .

[0105] Example 1, P max =0.94, P sub =0.02 → C1=(0.94-0.02) / (1-0.02)≈0.939 (the dominant behavior is significantly dominant).

[0106] Example 2, P max =0.5, P sub =0.45 → C1=(0.5-0.45) / (1-0.45)≈0.091 (The main behavior has no obvious advantage, and the probability distribution is scattered).

[0107] Step 3: Calculate the concentration C2 of the behavior probability distribution (auxiliary score, confidence weight 30%), which reflects the overall dispersion of the 5-dimensional behavior probabilities. The more concentrated the probabilities are in the main behavior, the closer C2 is to 1. It is calculated using the normalized inverse value of information entropy.

[0108] 1. Calculate the information entropy of a 5-dimensional probability. ;

[0109] 2. Information entropy normalization: ( ≈2.32 is the maximum entropy when the probabilities of the five types of behaviors are equal.

[0110] 3. Concentration calculation: C2 = 1 - H norm .

[0111] Example 1, P=[0.94,0.02,0.01,0.02,0.01]→H≈0.32→Hnorm≈0.138→C2≈0.862 (highly concentrated distribution);

[0112] Example 2, P=[0.2,0.2,0.2,0.2,0.2]→H≈2.32→Hnorm=1→C2=0 (completely dispersed, no clear dominant behavior).

[0113] Step 4: Weighted fusion to obtain the final confidence level C, and then combine C1, C2, and S... cons The final confidence level (value from 0 to 1) for a single prediction step is obtained by weighting and summing the results using a 4:3:3 weighting. The formula is as follows:

[0114] C = 0.4 × C1 + 0.3 × C2 + 0.3 × S cons .

[0115] In some embodiments, if the preset confidence threshold is 0.85, then the confidence determination rule is as follows:

[0116] If C≥0.85: The prediction result is reliable, and a behavior control instruction for the prediction step size is sent to the lighting control module (such as lighting up the area in advance or gradually dimming the lights).

[0117] If C < 0.85: the prediction result is unreliable. Return to the multimodal feature extraction module to re-extract features and iteratively optimize without triggering any lighting control operations to avoid erroneous adjustments.

[0118] In some embodiments, the training steps of the CNN-LSTM hybrid model include: collecting radar data and adding scene and behavior labels to the radar data; preprocessing, extracting features, and representing tensors in the labeled radar data; and partitioning and augmenting the radar data dataset. The model training convergence conditions include loss function convergence conditions, classification accuracy convergence conditions, and early stopping mechanism convergence conditions. The loss function conditions include a training set loss function convergence threshold and a validation set loss function convergence threshold. The classification accuracy convergence conditions include overall behavior classification accuracy, key behavior classification accuracy, and single-class behavior classification accuracy. The early stopping mechanism convergence conditions include a validation set traversal threshold, a training set and validation set classification accuracy threshold, and a training epoch threshold.

[0119] Specifically, the overall construction of the training set follows the entire process of "raw radar data acquisition → multi-dimensional data preprocessing → feature extraction and labeling → standardized data representation → dataset partitioning and enhancement". The collected samples cover four typical areas of smart buildings: office area, corridor, meeting room and lobby, with a total sample size of ≥100,000 and a single-class behavior sample size of ≥20,000, to avoid model bias caused by sample imbalance.

[0120] 1) Data acquisition: Raw radar data and scene tags are acquired synchronously.

[0121] The data acquisition equipment is a 24GHz millimeter-wave radar (multi-antenna array, angular resolution ≤5°, data sampling rate 10Hz) consistent with the actual deployment, synchronously acquiring raw data of position (x / y grid coordinates), radial velocity, echo signal strength, and direction of movement.

[0122] Scene and Behavior Labeling: Each radar data point is bound to two tags: a scene tag (office area / corridor / meeting room / lobby) and a behavior tag (enter / leave / stationary / moving / brief stop). The tags are simultaneously labeled by the labeling personnel through video surveillance to ensure that the labeling accuracy rate is ≥99%.

[0123] Data collection scenarios cover a wide range of scenarios, including different environmental interferences (equipment vibration, curtain swaying, strong light blocking), different numbers of people (single / multiple people), and different time periods (weekdays 9-6 pm / weekends / nighttimes), ensuring the generalization of the training set.

[0124] 2) Data Processing: Preprocessing rules consistent with the preprocessing unit of the data processing module ensure consistency between the training set data and the actual inference data. The processing steps are as follows:

[0125] Data cleaning: Filter outliers (non-human interference data with speeds >5m / s or <0.1m / s, invalid data with grid coordinates outside the area range), and remove empty frame data caused by signal loss;

[0126] Data synchronization: Align multi-dimensional raw data in 10-second time windows, generate one behavioral sample for each time window (matching the time window rules of actual inference), and pad short frame data of less than 10 seconds with zeros.

[0127] Data standardization: Convert data of different dimensions such as speed, direction of movement, and echo signal frequency into the 0-1 interval. The formula is: X′=(X-Xmin) / (Xmax-Xmin), where Xmin / Xmax are the minimum / maximum values ​​of each feature in the collected samples.

[0128] Interference filtering: According to the rules for multimodal feature extraction in the patent, non-human interference (such as the fixed frequency signal of elevator operation, the low-speed irregular movement of curtains blown by the wind) is filtered out, and only valid human behavior data is retained.

[0129] 3) Data Extraction: Multimodal core features are extracted according to the rules of the feature extraction unit. Based on the preprocessed radar data, four types of core features—motion, spatial, frequency domain, and temporal—are extracted as the basic feature set for model training. The feature dimensions extracted from a single sample total 128 dimensions. The specific extraction rules are as follows:

[0130] Feature type extraction dimensions and specific extraction rules: Motion features: 32-dimensional radial velocity (1-dimensional), direction of movement (1-dimensional), motion trajectory fitting parameters (10-dimensional), trajectory continuity (1-dimensional), velocity change rate (19-dimensional); retaining the effective velocity range of 0.1-2m / s. Spatial features: 32-dimensional 1m×1m grid coordinates (2-dimensional), relative position of people and lamps (2-dimensional), number of people in the grid (1-dimensional), spatial distribution clustering features (27-dimensional); dividing fixed grid ranges according to building areas. Frequency domain features: 32-dimensional frequency value after Fourier transform of echo signal (1-dimensional), frequency range label (1-dimensional), spectral features of the 0.2-5Hz effective frequency band (30-dimensional); filtering fixed frequency signals of non-personnel. Temporal features: 32-dimensional grid dwell time (1-dimensional), time period label (1-dimensional), historical dwell pattern matching degree (10-dimensional), behavior duration (1-dimensional), time series features (19-dimensional); statistics are performed with a 10-second window.

[0131] 4) Data Representation: Adapting to the tensor format of the CNN-LSTM hybrid model, the extracted multimodal features are converted into a standardized tensor format with CNN+LSTM dual inputs, matching the input requirements of the model's hierarchical structure, achieving separate input and fusion training of "spatial features + time series features". Specific representation rules are as follows:

[0132] CNN Spatial Feature Tensor: The 32-dimensional spatial features are reshaped into an 8×4 two-dimensional matrix, and finally represented as a four-dimensional tensor [N,1,8,4], where N is the number of samples, 1 is a single channel, and 8×4 is the two-dimensional grid distribution of spatial features (matching the 1m×1m grid division logic of buildings); it provides spatial feature input to the CNN network to extract the distribution pattern of people in the grid.

[0133] LSTM Time Series Feature Tensor: The 96 dimensions of motion, frequency domain, and time features are divided into 10 time steps (matching a 10-second time window, 1 time step per second), and finally represented as a three-dimensional tensor [N, 10, 96], which is rounded to [N, 10, 10] (insufficient dimensions are padded with zeros), where N is the number of samples, 10 is the time step size, and 10 is the feature dimension of each time step; it serves as the time series feature input for the LSTM network to capture the temporal dynamic changes of human behavior.

[0134] Label representation: Convert the 5 types of behavior labels into one-hot encoding format, such as "still" as [0,0,1,0,0] and "away" as [0,1,0,0,0], to adapt to the probability distribution format of the model classification output.

[0135] 5) Dataset post-processing includes dataset partitioning and data augmentation, which includes time dimension augmentation, feature dimension augmentation, and spatial dimension augmentation.

[0136] Dataset partitioning: The dataset is divided into training, validation, and test sets in a 7:2:1 ratio, with 70,000 records in the training set, 20,000 records in the validation set, and 10,000 records in the test set. This ensures that the scene and behavior distributions of the three datasets are consistent and avoids partitioning bias.

[0137] Data augmentation targets "brief pause" behaviors (easily confused with stationary / moving behaviors) with relatively small sample sizes, providing targeted enhancements to improve the model's recognition accuracy for this behavior.

[0138] Time dimension enhancement: The features of the 10-second time window are shifted by time (±1 second) to generate new samples.

[0139] Feature Dimension Enhancement: Gaussian noise perturbation (noise variance 0.01) is applied to multimodal features to simulate signal noise in real building environments.

[0140] Spatial Dimension Enhancement: Make a small offset (±0.5 grids) to the grid coordinates to simulate the small error in radar positioning.

[0141] Final training set: The augmented training set has a sample size of ≥80,000, with a sample size of ≥25,000 for "short stay" behavior, to ensure sample balance.

[0142] The model training convergence condition settings are tailored to the training characteristics of the CNN-LSTM hybrid model (hybrid networks are prone to gradient vanishing / overfitting, and a balance needs to be struck between classification accuracy and generalization). A triple convergence condition is adopted, consisting of loss function convergence, classification accuracy achievement, and an early stopping mechanism. Additionally, upper limits are set for training hyperparameters to adapt to the computing power requirements of edge computing devices. Specific settings are as follows:

[0143] 1) Basic training hyperparameter settings: First, define the basic hyperparameters for model training to provide a benchmark for convergence conditions. The hyperparameters are adapted to the training requirements of lightweight models and have no redundant calculations.

[0144] Optimizer: The Adam optimizer is selected, the initial learning rate is set to 1e-3, the learning rate decay strategy is step decay (decaying to 0.5 of the original value every 10 epochs), and the weight decay coefficient is 1e-4 (to prevent overfitting).

[0145] Batch Size: Set to 32 to accommodate the video memory / memory requirements of edge computing devices.

[0146] Training epochs: The maximum number of training epochs is set to 100 to avoid overfitting caused by overtraining.

[0147] Loss function: Cross-entropy loss is selected to suit multi-class classification tasks. The calculation formula is as follows: ,in One-hot encoding for the real label, This represents the class probabilities predicted by the model.

[0148] 2) The convergence conditions include the loss function convergence condition, the classification accuracy convergence condition, and the early stopping mechanism convergence condition. When all conditions are met simultaneously, the model training stops, ensuring that the model achieves both the classification accuracy requirement and good generalization ability, without overfitting / underfitting problems.

[0149] Condition 1: Loss function convergence condition. The loss function value on the training set decreases by less than 0.001 for 10 consecutive epochs, and the loss function value on the validation set shows no significant increase (increase of less than 0.005) for 10 consecutive epochs; the final loss function value on the training set is ≤0.15, and the final loss function value on the validation set is ≤0.20. Its purpose is to determine the model's fit and avoid underfitting (excessively high loss value) or overfitting (low loss on the training set, sudden increase in loss on the validation set).

[0150] Condition 2: Classification accuracy convergence condition, including overall behavior classification accuracy, key behavior classification accuracy, and single-class behavior classification accuracy.

[0151] The model achieves the preset classification accuracy on the validation set and remains stable for 5 consecutive epochs (accuracy fluctuation < 0.5%); the overall behavior classification accuracy is ≥ 95% (accuracy = number of correctly classified samples / total number of validation samples); the distinction accuracy between the key behavior "stationary vs. brief stay" is ≥ 92% (distinction accuracy = number of samples correctly distinguished between the two behaviors / total number of samples between the two behaviors); the classification accuracy for a single behavior (entering / leaving / moving) is ≥ 96%; its function is to directly determine whether the model meets the classification accuracy requirements of the business scenario and is a core convergence indicator.

[0152] Condition 3: The early stopping mechanism convergence condition is used as a fallback to prevent overfitting. The early stopping mechanism convergence condition includes a validation set traversal threshold, a classification accuracy threshold between the training and validation sets, and a training epoch threshold. When any of the following conditions are met, training is immediately stopped, and the model parameters with the highest validation set accuracy are loaded to avoid overtraining.

[0153] Validation set traversal threshold: The classification accuracy of the validation set decreases for 15 consecutive epochs, and the decrease is ≥2%;

[0154] Classification accuracy thresholds for training and validation sets: Classification accuracy of the training set ≥ 99%, but classification accuracy of the validation set < 93% (significant overfitting occurs);

[0155] Training epoch threshold: Training will stop once the maximum number of training epochs reaches 100, even if the first two conditions are not met (to avoid meaningless training).

[0156] In some embodiments, lighting control commands include on / off status, brightness, and color temperature.

[0157] Specifically, the lighting modes include an entry mode, an exit mode, and a stationary mode. Entry mode: If it is predicted that "personnel will enter a certain area," the brightness of that area is pre-lit at 30% (to avoid glare), and then gradually adjusted to the target brightness (e.g., 500 lux for an office area) after the personnel arrive. Exit mode: If it is predicted that "personnel will leave a certain area," the brightness is gradually reduced during the personnel's movement (e.g., from 500 lux to 100 lux), and then turned off after a 5-second delay after the personnel leave (to avoid being unlit when returning). Stationary mode: If it is predicted that "personnel will remain stationary in a certain area for an extended period," the brightness is kept stable, while the color temperature is finely adjusted based on time characteristics (e.g., 3 PM) (from cool white light 6500K to warm white light 5000K) to improve comfort.

[0158] For example, if the behavior is classified as movement and the predicted trend is entering, then the corresponding lights will be adjusted to execute the entering mode based on the spatial location in the prediction results. If the behavior is classified as entering and the predicted trend is stationary, then the corresponding lights will be adjusted to execute the stationary mode based on the spatial location in the prediction results. If the behavior is classified as brief stay and the predicted trend is leaving, then the corresponding lights will be adjusted to execute the leaving mode based on the spatial location in the prediction results. Those skilled in the art will understand that the matching of behavior classification and prediction results with lighting modes can be set according to the specific needs of the application scenario to adapt to the lighting requirements of different scenarios.

[0159] This patent focuses on spaces with "frequent human activity and high requirements for intelligent and energy-efficient lighting," specifically covering the following products and fields: smart building fields such as office buildings, commercial complexes, government office buildings, hotels and other large buildings; smart public space fields such as libraries, museums, hospitals, subway stations and other public areas; and smart manufacturing and industrial parks.

[0160] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand this document.

Claims

1. A lighting control device based on a CNN-LSTM hybrid model, characterized in that, include: The radar sensing module is used to monitor the activity data of people in the target area in real time. The activity data includes location, direction of movement and speed. The data processing module includes a feature extraction unit, a feature fusion unit, a behavior classification unit, a behavior prediction unit, and an evaluation unit. The feature extraction unit extracts motion features, temporal features, frequency domain features, and spatial features based on activity data. The feature fusion unit standardizes the features extracted by the feature extraction unit, assigns feature weights, and performs dimensionality reduction using principal component analysis to output core feature vectors. The behavior classification unit uses a CNN-LSTM hybrid model to extract personnel grid distribution information and determines the relative position information of personnel and lighting fixtures based on the personnel grid information, generating a temporal spatial feature vector. An LSTM network is used to output a global temporal feature vector based on the temporal feature sequence and identify personnel behavior classification. The behavior prediction unit uses a bidirectional LSTM model to output prediction results and prediction confidence based on the global temporal feature vector and spatial historical information. The prediction results include behavior classification probability, prediction trend, and spatial location. The evaluation unit is configured to output prediction results when the prediction confidence is greater than or equal to a preset confidence threshold. The data processing module also includes a preprocessing unit, which filters the activity data based on the speed and removes activity data with a speed greater than the speed threshold, aligns the activity data according to a preset time window, and standardizes the activity data. The steps for the behavior prediction unit to output prediction confidence include: calculating the feature consistency score of the radar data. ,in, For time continuity values, As a weight for time continuity, This is the spatial consistency value. Spatial consistency weight; Based on the primary behavior with the highest probability and the secondary behavior with the second highest probability in the behavior classification, calculate the probability percentage of the primary behavior. ,in, The probability value of the main behavior. The probability value for this action; Calculate the concentration of the probability distribution of behaviors. Where H is the information entropy of the 5-dimensional probability. For the i-th behavior classification, The probability of classifying the i-th behavior; Information entropy normalization ,in Given that the probabilities of the five types of behaviors are equal, the concentration is... ; Calculate the prediction confidence level based on preset weights. ,in, , and These are the confidence weights for the proportion of the main behavior probability, the concentration of the behavior probability distribution, and the feature consistency score, respectively. The lighting control module includes multiple lighting modes. The lighting control module matches the lighting modes based on the classification and prediction results of human behavior and generates lighting control instructions.

2. The apparatus according to claim 1, characterized in that, The CNN network of the CNN-LSTM hybrid model includes a first convolutional layer, a max pooling layer, a second convolutional layer, a global average pooling layer, a flattening layer, and a fully connected layer.

3. The apparatus according to claim 2, characterized in that, The LSTM network of the CNN-LSTM hybrid model consists of an input layer, a first forward hidden layer, a second forward hidden layer, a first fully connected layer, and a second fully connected layer.

4. The apparatus according to claim 3, characterized in that, The training steps for the CNN-LSTM hybrid model include: Collect radar data, add scene and behavior labels to the radar data, preprocess the labeled radar data, extract features and tensor represent it, and perform dataset partitioning and data augmentation on the radar data; The model training convergence conditions include loss function convergence conditions, classification accuracy convergence conditions, and early stopping mechanism convergence conditions. The loss function conditions include the training set loss function convergence threshold and the validation set loss function convergence threshold. The classification accuracy convergence conditions include overall behavior classification accuracy, key behavior classification accuracy, and single-class behavior classification accuracy. The early stopping mechanism convergence conditions include the validation set traversal threshold, the training set and validation set classification accuracy threshold, and the training rounds threshold.

5. The apparatus according to claim 1, characterized in that, The behavior prediction unit is configured to output prediction results based on a preset step size.

6. The apparatus according to claim 5, characterized in that, The preset step size includes multiple prediction step sizes, which form an arithmetic sequence. The forward LSTM fusion coefficients of the multiple prediction step sizes decrease sequentially, while the reverse LSTM fusion coefficients increase sequentially.

7. The apparatus according to claim 1, characterized in that, Lighting control commands include on / off status, brightness, and color temperature.

8. The apparatus according to claim 1, characterized in that, The radar sensing module uses a multi-antenna array distribution.

Images