Multi-mode adaptive smart home illumination adjustment method and system

Abstract

The invention relates to the technical field of smart home, in particular to a multi-mode adaptive smart home illumination adjustment method and system, and the method comprises the steps: collecting indoor and outdoor environment data through an environment sensor, and collecting indoor figure image data through a camera; training and establishing an illumination intensity demand prediction model based on a long and short term memory (LSTM) network model; training and establishing a personnel activity behavior recognition model based on a convolutional neural network (CNN) model; constructing a home illumination adjustment algorithm, and continuously adjusting the corresponding illumination behavior according to the predicted illumination intensity demand and the identified current personnel activity behavior; and judging the illumination adjustment satisfaction degree of the home illumination adjustment algorithm, and carrying out iterative optimization on the home illumination adjustment algorithm. According to the invention, intelligence, self-adaption and high efficiency of illumination adjustment are realized, and the user experience in the aspect of illumination smart home is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart home, and specifically to a smart home lighting adjustment method and system with multi-mode adaptation. Background Art

[0002] Traditional home lighting control mainly senses the ambient light intensity through photosensitive elements and combines with user-set thresholds to achieve the on-off control of light sources. However, this control method lacks environmental adaptability and personalized adjustment capabilities and is difficult to meet the lighting needs of users in different scenarios.

[0003] The existing home lighting adjustment technologies still have the following deficiencies: poor environmental adaptability, the existing methods mainly rely on preset control rules and thresholds, lack the adaptive ability to dynamic changing environments, and are difficult to cope with the influence of environmental factors such as sunlight and weather; low personalization level, the lighting needs of users often change with activity scenarios and personal preferences, the existing methods lack in-depth understanding of user behaviors and preferences, and cannot provide personalized lighting experiences; insufficient interactivity, the current control methods mainly rely on sensing signals such as photosensitive elements, lack active interaction and feedback with users, are difficult to timely obtain the satisfaction degree of users with lighting effects, and cannot achieve closed-loop optimization; limited energy-saving effect, the energy-saving operation of the lighting system requires minimizing energy consumption while meeting user needs, which requires intelligent balancing between effects and efficiency rather than simple threshold control.

[0004] To address the above problems, there is an urgent need for a home lighting adjustment method with multi-mode perception and intelligent optimization. This method should be able to make full use of environmental sensing data and user behavior information, construct prediction and optimization models through machine learning algorithms, and achieve adaptive and personalized lighting control. At the same time, a user feedback mechanism needs to be introduced to continuously evaluate the lighting effect and dynamically optimize the adjustment strategy to achieve energy-saving operation while meeting the user experience.

[0005] In view of this, the present invention proposes a smart home lighting adjustment system and method with multi-mode adaptation. Summary of the Invention

[0006] To achieve the above object, the present invention provides a smart home lighting adjustment method and system with multi-mode adaptation, and the specific technical solutions are as follows:

[0007] The smart home lighting adjustment method with multi-mode adaptation includes:

[0008] Real-time collect indoor and outdoor environmental data through environmental sensors, and real-time collect indoor human image data through cameras;

[0009] Based on the historical environmental data collected by environmental sensors, train and establish a lighting intensity demand prediction model based on the long short-term memory network (LSTM) model;

[0010] Based on the indoor human image data collected by the camera, train and establish a human activity behavior recognition model based on the convolutional neural network (CNN) model;

[0011] Construct a home lighting adjustment algorithm, and continuously adjust the corresponding lighting behaviors according to the predicted lighting intensity requirements and the recognized current human activity behaviors. The lighting behaviors include lighting intensity, lighting color temperature, and lighting switch status;

[0012] Based on the corresponding lighting behaviors adjusted by the home lighting adjustment algorithm, judge the lighting adjustment satisfaction of the home lighting adjustment algorithm according to the activity frequency of the person in the lighting area and the frequency of manually adjusting the lighting behavior, and perform iterative optimization on the home lighting adjustment algorithm.

[0013] Preferably, install light intensity sensors and temperature and humidity sensors indoors and outdoors to collect environmental parameter data in real time;

[0014] Arrange M sensor nodes indoors, numbered (r 1 , r 2 ,..., r M ), arrange N sensor nodes outdoors, numbered (s 1 , s 2 ,..., s N ). Each sensor node integrates a light intensity sensor and a temperature and humidity sensor;

[0015] Let the illuminance collected by the i-th indoor sensor node r i at time t be E i (t), the temperature be T i (t), and the humidity be H i (t); the illuminance collected by the j-th outdoor sensor node s j at time t be E j ′(t), the temperature be T j ′(t), and the humidity be H j ′(t); arrange K cameras in the indoor area, numbered (c 1 , c 2 ,..., c K ); let the image frame collected by the k-th camera c k at time t be I k (t);

[0016] The environmental sensors and cameras work together to collect indoor and outdoor environmental data and human image data simultaneously.

[0017] Preferably, the input and output of the LSTM model are determined. The input of the LSTM model is historical environmental data, including indoor and outdoor illuminance, temperature, and humidity, as well as the illuminance setting value at the corresponding moment; the output of the LSTM model is the predicted value of the illuminance demand at the future moment.

[0018] Let the illuminance setting value at time t be y (LSTM) (t), and the indoor sensor node r i collects the illuminance, temperature, and humidity as E i (t), T i (t), and H i (t), and the outdoor sensor node s j collects the illuminance, temperature, and humidity as E j ′(t), T j ′(t), and H j ′(t). Then the environmental parameter vector at time t is:

[0019] X (LSTM) (t) = [y (LSTM) (t), E i (t), T i (t), H i (t), E j ′(t), T j ′(t), H j ′(t)]

[0020] The environmental parameter vectors of ζ historical moments are formed into an input sequence: X (LSTM) = [X (LSTM) (t - ζ), …, X (LSTM) (t - 1), X (LSTM) (t)]; the illuminance demands of the next ζ′ moments are formed into an output sequence: Y (LSTM) = [y (LSTM) (t + 1), …, y (LSTM) (t + ζ′)]; then the LSTM model can be expressed as a mapping relationship: Y (LSTM) = F(X (LSTM) ); where, is the backtracking order; ζ′ is the prediction order;

[0021] Design the network structure of the LSTM model. The number of input layer nodes of the LSTM model is N in = ζ × (3M + 3N + 1), the number of output layer nodes is N o = ζ′; the hidden layer adopts a double-layer LSTM structure, and the number of memory blocks in each layer is N h1 and N h2 , and the dimension of the storage unit in the memory block is d; a fully connected structure is adopted between each layer;

[0022] The goal of training the LSTM model is to minimize the mean squared error between the predicted sequence of light intensity requirements and the actual value sequence.

[0023] Preferably, to train the LSTM model, historical data is collected from environmental sensors and light controllers, aligned by time and resampled at fixed time intervals to obtain a complete sequence of environmental data and light intensity setting values, and the collected environmental data is preprocessed.

[0024] From the preprocessed historical data, input-output sequences (X (LSTM) , Y (LSTM) ) are extracted in the form of a sliding time window as a training sample; the step size of the sliding window is the prediction order ζ′, that is, the start time of the next sample is the end time of the previous sample.

[0025] Randomly shuffle the order of the training samples and divide them into a training set, a validation set, and a test set according to a certain proportion; the training set is used for learning the parameters of the LSTM model, the validation set is used for tuning parameters and early stopping, and the test set is used for evaluating the performance of the LSTM model.

[0026] Test the performance of the trained LSTM model on the test set to evaluate the generalization ability of the LSTM model; if the performance does not meet the requirements, improve the model by increasing the dimension of the hidden layer, extending the backtracking order, and introducing a regularization term.

[0027] Based on the trained LSTM model, predict the future light intensity requirements in real time; at time t, input the environmental parameter vector of the past ζ moments into the model, and the predicted sequence of light intensity requirements for the next ζ′ moments can be obtained.

[0028] Preferably, the input of the CNN model is the sequence of human image frames collected by the camera, and the output is the category of human activity behavior corresponding to the sequence of image frames; assuming there are C predefined activity behavior categories, the model is represented as a mapping relationship: y (CNN) = F(X (CNN) ); where the input is a sequence of human image frames of length T, and the size of each frame of image x T is H×W×D, where H, W, and D represent the height, width, and number of channels of the image respectively; the output is the probability distribution of each category, satisfying where C is the total number of human activity behavior categories;

[0029] Design the network structure of the CNN model, adopt the architecture of CNN structure plus temporal pooling, use the CNN structure to extract the features of each frame of image, then perform temporal pooling on the feature sequence, and finally use the fully connected layer to output the classification result.

[0030] The fully connected layer is removed from the CNN structure, and only the convolutional layer and pooling layer are retained for feature extraction. Suppose the size of the feature map after the last pooling layer of the CNN is H'×W'×d'. Then the feature map is flattened into a d-dimensional feature vector, where d = H′×W′×D', and H′, W′, and D' represent the height, width, and number of channels of the feature map after the last pooling operation of the CNN model, respectively.

[0031] The temporal pooling part performs a pooling operation on the d-dimensional feature vectors of T time steps extracted by the CNN structure to obtain a d-dimensional feature representation.

[0032] After obtaining the pooled feature representation z = [z 1 ,…,z d ), it is mapped to a C-dimensional output space through a fully connected layer, and then transformed into a probability distribution using the softmax function: y (CNN) = softmax(ψz + b); where ψ and b are the weight matrix and bias vector of the fully connected layer, respectively.

[0033] The goal of training the CNN model is to minimize the cross-entropy loss between the predicted probability distribution and the true label, and at the same time add an L2 regularization term to prevent overfitting.

[0034] Preferably, from the video stream collected by the camera, image frames are extracted at fixed time intervals, and consecutive T frames of images are used as a sample. At the same time, each sample is manually labeled to indicate the category of the human activity behavior to which the sample belongs, and the image sample is preprocessed.

[0035] The sample set is divided into a training set, a validation set, and a test set according to a certain proportion. The model is trained on the training set, the parameters are tuned and the optimal model is selected on the validation set, and the model performance is evaluated on the test set.

[0036] The SGD optimization algorithm using the mini-batch method is used to train the model. In each epoch, each batch is input in turn and the model parameters are updated. After all batches are trained, the next epoch is entered. During the training process, the model performance is evaluated on the validation set every preset number of epochs. If the performance does not improve for consecutive Q epochs, the training is terminated early.

[0037] The trained CNN model is tested on the test set to evaluate the generalization performance of the CNN model on new samples.

[0038] The trained human activity behavior recognition model is deployed to analyze the human image data collected by the camera in real time. When a person appears in the picture, consecutive T frames of images are extracted and input into the model to obtain the category of human activity behavior in the current time period.

[0039] Preferably, according to M sensor nodes arranged indoors, each sensor node corresponds to a lighting area at the same time, and there are M independently adjustable lighting areas in the smart home environment, numbered m = 1, …, M; the lighting behavior of the m-th lighting area includes the lighting intensity I m ∈{I 1 ,…,I M}, the lighting color temperature T m ∈{T 1 ,…,T M}, and the lighting switch state S m ∈{0, 1};

[0040] Suppose that at time t, the average lighting intensity demand predicted by the LSTM model at the future τ' moment is The current human activity behavior category recognized by the CNN model is A t , A t ∈{A 1 ,…,A C};

[0041] Adjust the lighting intensity of the lighting area according to the predicted lighting intensity demand. According to value, adjust the lighting intensity I m of the m-th lighting area proportionally: where α m ∈[0, 1], α m is the intensity distribution coefficient of the m-th lighting area, satisfying

[0042] Adjust the lighting color temperature of the lighting area according to the human activity behavior category, and pre-define the color temperature mapping table A 1 ,…,A C →T 1 ,…,T M , map the human activity behavior category A t to the recommended lighting color temperature value T m :

[0043] Adjust the lighting switch state of the lighting area according to the human activity area. Suppose the space corresponding to the m-th lighting area is R m , and the set of lighting areas where human activities are detected at time t is Then the lighting switch control rule is: when , then S m = 1, indicating that the lighting switch is in the on state; when , then S m = 0, indicating that the lighting switch is in the off state.

[0044] Preferably, according to M sensor nodes arranged indoors, each sensor node corresponds to a lighting area at the same time. It is set that there are M lighting areas in the smart home environment, numbered i = 1, …, M, and each lighting area corresponds to a type of personnel activity behavior category A i , A i ∈{A 1 ,…,A C};

[0045] For area i, within the current τ time, define n i,τ to represent the number of times of personnel activities detected in lighting area i within the current τ time; f i,τ represents the number of times the user manually adjusts the lighting parameters of lighting area i within the current τ time; respectively represent the average light intensity, light color temperature, and light switch state of lighting area i determined by the home lighting adjustment algorithm within the current τ time;

[0046] Introduce the lighting adjustment satisfaction degree r i,τ ∈[0, 1] to represent the degree of satisfaction of the user with the lighting adjustment effect of lighting area i within the current τ time, and construct the lighting adjustment satisfaction calculation formula:

[0047]

[0048] where r i,τ ∈[0, 1], α is a scaling factor, α > 0, and exp[·] represents the exponential function with e as the base;

[0049] According to the statistically obtained n i,τ and f i,τ , calculate the lighting adjustment satisfaction degree r i,τ of each area; based on the value of the lighting adjustment satisfaction degree r i,τ , adaptively adjust the relevant parameters of the lighting adjustment algorithm.

[0050] A multi-mode adaptive smart home lighting adjustment system, which is used to implement the multi-mode adaptive smart home lighting adjustment method, including: a data acquisition module, a light intensity prediction module, a behavior type recognition module, a lighting adjustment module, and an iterative optimization module;

[0051] The data acquisition module collects indoor and outdoor environment data in real time through environmental sensors and collects indoor human image data in real time through cameras;

[0052] The light intensity prediction module trains and establishes a light intensity demand prediction model based on the long short-term memory network LSTM model based on the historical environment data collected by the environmental sensors;

[0053] The behavior type recognition module trains and establishes a personnel activity behavior recognition model based on the indoor human image data collected by the camera and based on the convolutional neural network (CNN) model;

[0054] The light intensity adjustment module constructs a home lighting adjustment algorithm, and continuously adjusts the corresponding lighting behaviors according to the predicted light intensity requirements and the recognized current personnel activity behaviors. The lighting behaviors include light intensity, light color temperature, and lighting switch status;

[0055] The iterative optimization module discriminates the lighting adjustment satisfaction degree of the home lighting adjustment algorithm based on the corresponding lighting behaviors adjusted by the home lighting adjustment algorithm, and the activity frequency of the personnel in the lighting area and the manual adjustment lighting behavior frequency, and iteratively optimizes the home lighting adjustment algorithm.

[0056] An electronic device includes: a processor and a memory. Among them, a computer program that can be called by the processor is stored in the memory; the processor executes the multi-mode adaptation-based smart home lighting adjustment method by calling the computer program stored in the memory.

[0057] A computer-readable storage medium stores instructions, and when the instructions run on a computer, the computer is made to execute the multi-mode adaptation-based smart home lighting adjustment method.

[0058] Advantages of the present invention:

[0059] The present invention collects indoor and outdoor environmental data and human image data in real time through environmental sensors and cameras, provides comprehensive and dynamic environmental and user behavior information for intelligent lighting adjustment, establishes a multi-modal perception mechanism, and improves the adaptability of the system to the environment and user needs.

[0060] The present invention uses the LSTM model to learn historical environmental data, constructs a light intensity requirement prediction model, can predict the user's light intensity requirements in advance according to the dynamic changes of environmental factors, realizes forward-looking intelligent adjustment, and improves the active adaptation ability of the lighting system.

[0061] The present invention analyzes the human image data collected by the camera using the CNN model, realizes real-time recognition of personnel activity behaviors, captures the changes in lighting requirements of users in different scenarios, provides a basis for personalized lighting adjustment, and improves the user experience of the lighting system.

[0062] The present invention designs a home lighting adjustment algorithm with multi-factor collaboration, comprehensively considers the predicted lighting requirements and the recognized personnel activity behaviors, dynamically adjusts the light intensity, color temperature, and regional lighting switches, realizes intelligent and fine-grained lighting optimization, meets user needs, and improves energy efficiency at the same time.

[0063] The present invention introduces a satisfaction evaluation mechanism based on user activities and feedback. By analyzing the frequency of personnel activities and the manual adjustment frequency, it determines the satisfaction level of the current lighting adjustment effect, and iteratively optimizes the adjustment algorithm based on this to form a closed-loop feedback, continuously improving the performance of the lighting system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of a multi-mode adaptable smart home lighting adjustment method provided by the present invention;

[0065] Figure 2 It is a structural diagram of a multi-mode adaptable smart home lighting adjustment system provided by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0066] To better understand the present invention, more detailed descriptions of various aspects of the present invention will be made with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present invention, and do not limit the scope of the present invention in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0067] In the drawings, for the sake of clarity, the sizes, dimensions, and shapes of the elements have been slightly adjusted. The drawings are only for illustration and are not drawn to an exact scale. As used herein, the terms "substantially", "about", and similar terms are used as terms of approximation, not as terms of degree, and are intended to account for the inherent deviations in measured or calculated values that would be recognized by a person of ordinary skill in the art. Additionally, in the present invention, the order of description of the steps of the processes does not necessarily represent the order in which these processes occur in actual operation, unless otherwise clearly specified or derivable from the context.

[0068] It should also be understood that expressions such as "comprises", "comprising", "has", "including", and / or "including having" in this specification are open-ended rather than closed-ended expressions, which mean the presence of the stated features, elements, and / or components, but do not exclude the presence of one or more other features, elements, components, and / or combinations thereof. In addition, when an expression such as "at least one of..." appears after a list of listed features, it modifies the entire list of features, rather than just individual elements in the list. Further, when describing embodiments of the present invention, the use of "may" means "one or more embodiments of the present invention". And the term "exemplary" is intended to refer to an example or illustration.

[0069] Unless otherwise defined, all terms (including technical and scientific terms) used herein shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It should also be understood that, unless explicitly stated otherwise in this invention, words defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art and shall not be interpreted in an idealized or overly formal sense.

[0070] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0071] Example 1

[0072] Referring to Figure 1 , the first embodiment of the present invention provides a multi-mode adaptive intelligent home lighting adjustment method.

[0073] S1: Collect indoor and outdoor environmental data in real time through environmental sensors, and collect indoor human image data in real time through cameras.

[0074] Install light intensity sensors and temperature and humidity sensors indoors and outdoors to collect environmental data in real time.

[0075] Exemplarily, the requirements for sensor selection are as follows: (1) Light intensity sensor: Use a digital illuminance sensor, such as BH1750; the measurement range is 0 - 65535 lx, the resolution is 1 lx, the spectral response is close to the human eye, and the communication method is I2C; (2) Temperature and humidity sensor: Use a digital temperature and humidity sensor, such as DHT22; the temperature measurement range is -40 - 80 °C, the humidity measurement range is 0% - 100% RH, the temperature resolution is 0.1 °C, the humidity resolution is 0.1% RH, and the communication method is single bus.

[0076] Arrange M sensor nodes indoors, numbered (r 1 , r 2 ,..., r M ), and deploy them in the main rooms such as the living room, bedroom, study, kitchen, and bathroom respectively; arrange N sensor nodes outdoors, numbered (s 1 , s 2 ,..., s N ), and deploy them on the balcony, outside the window, etc. respectively; each sensor node integrates a light intensity sensor and a temperature and humidity sensor, and the sampling frequency is set to f s , usually taking 1 / 60 Hz - 1 / 300 Hz.

[0077] Let the illuminance collected at the i-th indoor sensor node r i at time t be E i (t), and the temperature be Ti (t), with humidity H i (t); at the j-th outdoor sensor node s j The collected illuminance is E j ′(t), with temperature T j ′(t), with humidity H j ′(t); The sensor uploads the data to the intelligent gateway through a wireless communication module (such as WiFi, ZigBee).

[0078] Arrange K cameras in the indoor area, numbered (c 1 , c 2 ,..., c K ); To cover the entire room and avoid dead angles, fisheye cameras can be used, with a horizontal field of view angle ≥ 180° and a vertical field of view angle ≥ 90°; The cameras are installed on the ceiling, looking down on the entire room; The image acquisition requirements are as follows: (1) Use high-definition cameras, with a resolution ≥ 1080P (1920×1080) and a frame rate ≥ 15fps; (2) Have infrared night vision function and can still collect clear images in low illuminance environments; (3) Have pan-tilt control function and can remotely adjust the camera angle; (4) Support H.264 and H.265 video coding to compress and transmit video data; Let the image frame collected by the k-th camera c k at time t be I k (t), with a resolution of W k ×H k ; To save transmission and storage overhead, the image frames are transmitted in the form of a video stream; Considering the moving speed of people, the image frame rate is set to f c , generally taking 5 - 30fps.

[0079] The environmental sensors and cameras work together to collect indoor and outdoor environmental data and human image data in real time at a frequency of min(1 / f s , 1 / f c ), comprehensively perceive the dynamic changes of the indoor environment, and provide a data basis for the intelligent dimming function.

[0080] S2: Based on the historical environmental data collected by the environmental sensors, train and establish a prediction model for the illuminance demand based on the long short-term memory network LSTM model.

[0081] Determine the input and output of the LSTM model. The input of the LSTM model is historical environmental data, including indoor and outdoor illuminance, temperature, and humidity, as well as the illuminance setting value at the corresponding time; The output of the LSTM model is the predicted value of the illuminance demand at the future time.

[0082] Let the illuminance setting value at time t be y (LSTM) (t), and the indoor sensor node r iThe collected illuminance, temperature and humidity are E i (t), T i (t) and H i (t), respectively. The environmental parameter vector at time t for the outdoor sensor node s j The collected illuminance, temperature and humidity are E j ′(t), T j ′(t) and H j ′(t), respectively. Then the environmental parameter vector at time t is:

[0083] X (LSTM) (t) = [y (LSTM) (t), E i (t), T i (t), H i (t), E j ′(t), T j ′(t), H j ′(t)]

[0084] The environmental parameter vectors at ζ historical moments are formed into an input sequence: X (LSTM) = [X (LSTM) (t - ζ), …, X (LSTM) (t - 1), X (LSTM) (t)]; The illuminance requirements at the next ζ' moments are formed into an output sequence: Y (LSTM) = [y (LSTM) (t + 1), …, y (LSTM) (t + ζ′)]; And the LSTM model is expressed as a mapping relationship: Y (LSTM) = F(X (LSTM) ); where F(·) represents the mapping relationship function, which is a non-linear prediction model constructed using a long short-term memory network (LSTM) to extract features from the input environmental parameter sequence and predict the output illuminance value; τ is the backtracking order, that is, how long historical environmental data in the past is considered; ζ′ is called the prediction order, that is, how long the illuminance requirement in the future is predicted; The backtracking order generally takes 24 - 48, which can cover the day-night change cycle; The prediction order is determined according to the actual scenario requirements, usually taking 1 - 4, that is, predicting the illuminance requirement in the next 1 - 4 time units.

[0085] Design the network structure of the LSTM model. The number of input layer nodes of the LSTM model is N in = ζ × (3M + 3N + 1), and the number of output layer nodes is N o = ζ′; The hidden layer adopts a double-layer LSTM structure, and the number of memory blocks in each layer is N h1 and N h2, the dimension of the storage unit in the memory block is d; a fully connected structure is adopted between layers; the LSTM memory block internally includes four parts: an input gate, a forget gate, an output gate, and a storage unit.

[0086] The goal of training the LSTM model is to minimize the mean square error between the predicted sequence of light intensity requirements and the true value sequence. The Adam optimization algorithm can be used for iterative optimization, and an early stopping mechanism is used to prevent overfitting.

[0087] To train the LSTM model, historical data is collected from environmental sensors and lighting controllers, aligned by time, and resampled at a fixed time interval (such as 1 minute) to obtain a complete sequence of environmental data and light intensity setting values, and the collected environmental data is preprocessed.

[0088] This includes outlier removal, missing value imputation, data normalization, etc.; outliers can be determined by statistical methods (such as the 3σ principle) or physical constraints; missing values can be filled by methods such as linear interpolation; normalization uses min-max normalization.

[0089] From the preprocessed historical data, input-output sequences (X (LSTM) , Y (LSTM) ) are extracted in the form of a sliding time window as a training sample; the step size of the sliding window is the prediction order ζ′, that is, the start time of the next sample is the end time of the previous sample.

[0090] Randomly shuffle the order of the training samples and divide them into a training set, a validation set, and a test set according to a certain proportion; the training set is used for learning the parameters of the LSTM model, the validation set is used for parameter tuning and early stopping, and the test set is used for evaluating the model performance; the proportion of sample data division can be set to 8:1:1.

[0091] Initialize the parameters of the LSTM model and train the model using the mini-batch method; the samples within each batch are calculated in parallel, and the batches are iterated sequentially; the training samples are divided into several batches according to the batch size (such as 32 or 64), and each batch is input sequentially within each epoch and the model parameters are updated. After all batches are trained, enter the next epoch; evaluate the model performance on the validation set every certain number of epochs. If the model performance does not improve for several consecutive epochs, terminate the training early; the performance metrics of the LSTM model can be the mean absolute error (MAE) or the root mean square error (RMSE).

[0092] Test the performance of the trained model on the test set to evaluate its generalization ability; if the performance does not meet the requirements, the model can be improved by methods such as increasing the hidden layer dimension, extending the backtracking order, and introducing regularization terms.

[0093] Based on the trained LSTM model, the future light intensity demand is predicted in real time; at time t, by inputting the environmental parameter vectors of the past ζ moments into the model, the predicted sequence of the light intensity demand for the next ζ′ moments can be obtained.

[0094] S3: Based on the indoor human image data collected by the camera, train and establish a human activity behavior recognition model based on the convolutional neural network CNN model.

[0095] The input of the CNN model is the sequence of human image frames collected by the camera, and the output is the category of human activity behavior corresponding to the image frame sequence; assuming there are a total of C predefined human activity behavior categories, the model can be expressed as a mapping relationship: y (CNN) = F(X (CNN) ); where the input is an image frame sequence of length T, and the size of each frame image x T is H×W×D, where H, W, and D represent the image height, width, and number of channels respectively; the output is the probability distribution of each category, satisfying where C is the total number of human activity behavior categories.

[0096] Design the network structure of the CNN model, adopt the architecture of CNN structure plus temporal pooling, use the CNN structure to extract the features of each frame image, then perform temporal pooling on the feature sequence, and finally use the fully connected layer to output the classification result.

[0097] For the CNN structure part, use the VGG16 pre-trained model, remove the fully connected layer, and only retain the convolutional layer and pooling layer for feature extraction; assuming the size of the feature map after the last pooling layer of the CNN is H'×W′×D', then flatten the feature map into a d-dimensional feature vector, where d = H′×W′×D', and H′, W′, and D′ represent the height, width, and number of channels of the feature map after the last pooling operation of the CNN model respectively.

[0098] For the temporal pooling part, perform pooling operations on the d-dimensional feature vectors of T time steps extracted by the CNN structure to obtain a d-dimensional feature representation; the optional pooling operations include max pooling, average pooling, and attention pooling, etc.; max pooling extracts the maximum value of each feature dimension over the entire time series, capturing significant temporal features.

[0099] After obtaining the pooled feature representation z = [z 1 , …, z d , map it to the C-dimensional output space through the fully connected layer, and then convert it into a probability distribution using the softmax function: where ψ and b are the weight matrix and bias vector of the fully connected layer respectively.

[0100] The goal of training the CNN model is to minimize the cross - entropy loss between the predicted probability distribution and the true labels, while adding an L2 regularization term to prevent overfitting.

[0101] From the video stream captured by the camera, extract image frames at fixed time intervals (such as 1 second), and use consecutive T frames of images as a sample; at the same time, manually annotate each sample to indicate the category of the human activity behavior to which the sample belongs, and preprocess the image samples, including size normalization, data augmentation, etc.; size normalization scales the image to a fixed size (such as 224×224) to ensure the consistency of the CNN input; data augmentation generates more samples through operations such as random cropping, flipping, and rotation to improve the robustness of the model.

[0102] Divide the sample set into a training set, a validation set, and a test set according to a certain ratio, and the ratio can be set as 8:1:1; perform model training on the training set, tune parameters and select the optimal model on the validation set, and evaluate the model performance on the test set.

[0103] Use the SGD optimization algorithm with the mini - batch method to train the model, and set the batch size to 32 or 64; input each batch in turn within each epoch and update the model parameters, and enter the next epoch after all batches are trained; during the training process, evaluate the model performance on the validation set every certain number of epochs. If the performance does not improve for consecutive Q epochs, terminate the training in advance; the performance evaluation metrics can be accuracy, F1 - score, area under the ROC curve, etc.

[0104] Test the trained CNN model on the test set to evaluate the generalization performance of the CNN model on new samples; if the performance is not ideal, it can be improved and optimized by adjusting the network structure, introducing more training data, refining the definition of human activity behavior categories, etc.

[0105] Deploy the trained human activity behavior recognition model to analyze the human image data captured by the camera in real - time; when a person appears in the picture, extract consecutive T frames of images and input them into the model to obtain the human activity behavior category during that time period; according to the recognized activity behavior, combined with the current environmental state and user preferences, trigger the corresponding automatic dimming strategy. For example, when it is detected that the user is reading or studying, the illumination brightness and color temperature in the reading area can be appropriately increased; when the user leaves the room, the lighting fixtures can be automatically turned off to save energy.

[0106] S4: Construct a home lighting adjustment algorithm, and continuously adjust the corresponding lighting behaviors according to the predicted lighting intensity requirements and the recognized current human activity behaviors. The lighting behaviors include lighting intensity, lighting color temperature, and lighting switch status.

[0107] According to M sensor nodes arranged indoors, each sensor node corresponds to a lighting area at the same time. There are M independently adjustable lighting areas in the smart home environment, numbered m = 1, …, M; the lighting behavior of the m-th lighting area includes the lighting intensity I m ∈{I 1 ,…,I M}, the lighting color temperature T m ∈{T 1 ,…,T M}, and the lighting switch state S m ∈{0, 1}, where I m represents the lighting intensity value of the m-th lighting area, and T m represents the lighting color temperature value of the m-th lighting area;

[0108] Suppose that at time t, the average lighting intensity demand predicted by the LSTM model at the future τ′ moment is The current human activity behavior category recognized by the CNN model is A t , A t ∈{A 1 ,…,A C};

[0109] Adjust the lighting intensity of the lighting area according to the predicted lighting intensity demand. According to the value of , adjust the lighting intensity I m of the m-th lighting area m proportionally: where α m ∈[0, 1], and α m is the intensity distribution coefficient of the m-th lighting area, satisfying

[0110] Adjust the lighting color temperature of the lighting area according to the human activity behavior category. Pre-define the color temperature mapping table A 1 ,…,A C →T 1 ,…,T M , map the human activity behavior category A t to the recommended lighting color temperature value T m : represents the relationship mapping between behavior classification and lighting color temperature, which can be a look-up table or a rule function;

[0111] Adjust the lighting area switch according to the human activity area. Let the space corresponding to the l-th lighting area be R m . At time t, the set of lighting areas where human activities are detected is Then the lighting switch control rule is: when , then S m= 1 indicates that the light switch is in the on state; when then S m = 0, indicating that the light switch is in the off state.

[0112] S5: Based on the corresponding lighting behavior adjusted by the home lighting adjustment algorithm, and according to the activity frequency of the person in the lighting area and the manual adjustment lighting behavior frequency, judge the lighting adjustment satisfaction of the home lighting adjustment algorithm, and perform iterative optimization on the home lighting adjustment algorithm.

[0113] According to the M sensor nodes arranged indoors, each sensor node corresponds to a lighting area at the same time. It is set that there are M lighting areas in the smart home environment, numbered i = 1, …, M, and each lighting area corresponds to a type of personnel activity behavior A i , A i ∈{A 1 , …, A C}.

[0114] For area i, within the current τ time, define n i,τ to represent the number of times of detecting personnel activities in lighting area i within the current τ time; f i,τ represents the number of times the user manually adjusts the lighting parameters of lighting area i within the current τ time; respectively represent the average light intensity, color temperature and switch state of lighting area i decided by the lighting adjustment algorithm within the current τ time.

[0115] Introduce the lighting adjustment satisfaction r i,τ to represent the degree of satisfaction of the user with the lighting adjustment effect of lighting area i within the current τ time, and construct the lighting adjustment satisfaction calculation formula:

[0116]

[0117] where r i,τ ∈[0, 1], α is a scaling factor, α > 0, and exp[·] represents the exponential function with e as the base.

[0118] Considering that the higher the personnel activity frequency n i,τ , it indicates that the lighting adjustment demand in this area is stronger; if n i,τ is very high but the manual adjustment frequency f i,τ is very low, it means that the lighting adjustment algorithm better meets the user's needs and the satisfaction is relatively high.

[0119] The characteristic of the lighting adjustment satisfaction calculation formula is that when n i,τ is relatively large and f i,τ is relatively small, the lighting adjustment satisfaction r i,t is close to 1; when n i,τ is relatively small and f i,τWhen it is relatively large, the satisfaction degree r of light intensity adjustment i,τ is close to 0; when n i,τ and f i,τ are comparable, the satisfaction degree r of light intensity adjustment i,τ is close to 0.5.

[0120] According to the statistically counted n i,τ and f i,τ , calculate the satisfaction degree r of light intensity adjustment for each area i,τ ; based on the value of the satisfaction degree r of light intensity adjustment i,τ , adaptively adjust the relevant parameters of the light intensity adjustment algorithm.

[0121] Exemplarily, the setting standard of the intensity distribution coefficient α i : according to the relative size of r i,τ , appropriately increase the value of α in the area with high satisfaction degree and decrease the value of α in the area with low satisfaction degree i , but still need to ensure i The setting standard of the color temperature mapping table: for the area with low satisfaction degree, try to randomly select from the nearby color temperature values to explore more popular color temperature settings. The modified light intensity adjustment algorithm will be used in the next light intensity adjustment decision and continue to receive satisfaction feedback to form a closed-loop optimization; through the dynamic coupling of decision parameters and satisfaction degree, the light intensity adjustment algorithm can be continuously and adaptively optimized to better fit the user's preferences.

[0122] Example 2

[0123] Example 2

[0124] Referring to Figure 2 , the second embodiment of the present invention provides a smart home light intensity adjustment system with multi-mode adaptation.

[0125] The system includes: a data acquisition module, a light intensity prediction module, a behavior type recognition module, a light intensity adjustment module, and an iterative optimization module.

[0126] The data acquisition module collects indoor and outdoor environmental data in real time through environmental sensors and collects indoor human image data in real time through a camera.

[0127] The light intensity prediction module trains and establishes a light intensity demand prediction model based on the long short-term memory network LSTM model based on the historical environmental data collected by the environmental sensors.

[0128] The behavior type recognition module trains and establishes a personnel activity behavior recognition model based on the indoor human image data collected by the camera based on the convolutional neural network CNN model.

[0129] The lighting adjustment module constructs a home lighting adjustment algorithm, and continuously adjusts the corresponding lighting behaviors according to the predicted lighting intensity requirements and the recognized current personnel activity behaviors. The lighting behaviors include lighting intensity, lighting color temperature, and lighting switch status.

[0130] The iterative optimization module determines the lighting adjustment satisfaction degree of the home lighting adjustment algorithm based on the corresponding lighting behaviors adjusted by the home lighting adjustment algorithm, and the activity frequency of the personnel in the lighting area and the manual adjustment lighting behavior frequency, and iteratively optimizes the home lighting adjustment algorithm.

[0131] Embodiment 3

[0132] The present invention also provides an electronic device. The electronic device may include one or more processors and one or more memories. Among them, computer-readable code is stored in the memory, and when the computer-readable code is run by one or more processors, it can execute the multi-mode adaptation smart home lighting adjustment method as described above.

[0133] The method or system according to the embodiment of the present invention can also be implemented by means of the architecture of the electronic device of the present invention.

[0134] The electronic device may include a bus, one or more CPUs, a read-only memory (ROM), a random access memory (RAM), a communication port connected to the network, input / output components, a hard disk, etc.

[0135] The storage device in the electronic device, such as ROM or hard disk, can store the multi-mode adaptation smart home lighting adjustment method provided by the present invention.

[0136] The multi-mode adaptation smart home lighting adjustment method includes: real-time collecting indoor and outdoor environment data through an environmental sensor, and real-time collecting indoor human image data through a camera; training and establishing a lighting intensity requirement prediction model based on a long short-term memory network (LSTM) model based on the historical environment data collected by the environmental sensor; training and establishing a personnel activity behavior recognition model based on a convolutional neural network (CNN) model based on the indoor human image data collected by the camera; constructing a home lighting adjustment algorithm, and continuously adjusting the corresponding lighting behaviors according to the predicted lighting intensity requirements and the recognized current personnel activity behaviors. The lighting behaviors include lighting intensity, lighting color temperature, and lighting switch status; determining the lighting adjustment satisfaction degree of the home lighting adjustment algorithm based on the corresponding lighting behaviors adjusted by the home lighting adjustment algorithm, and the activity frequency of the personnel in the lighting area and the manual adjustment lighting behavior frequency, and iteratively optimizing the home lighting adjustment algorithm.

[0137] Further, the electronic device may further include a user interface. Of course, the architecture of the present invention is only exemplary. When implementing different devices, one or more components of the electronic device disclosed in the present invention may be omitted according to actual needs.

[0138] Embodiment 4

[0139] The present invention also discloses a computer-readable storage medium.

[0140] Computer-readable instructions are stored on the computer-readable storage medium.

[0141] When the computer-readable instructions are run by a processor, the multi-mode adaptive smart home lighting adjustment method according to the embodiments of the present invention described with reference to the above drawings can be executed.

[0142] The storage medium includes but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. Additionally, according to the embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs.

[0143] For example, the present invention provides a non-transitory machine-readable storage medium storing machine-readable instructions that can be run by a processor to execute instructions corresponding to the method steps provided by the present invention, such as: collecting indoor and outdoor environmental data in real time through an environmental sensor, and collecting indoor human image data in real time through a camera; training and establishing a lighting intensity demand prediction model based on a long short-term memory network (LSTM) model based on historical environmental data collected by the environmental sensor; training and establishing a personnel activity behavior recognition model based on a convolutional neural network (CNN) model based on the indoor human image data collected by the camera; constructing a home lighting adjustment algorithm, and continuously adjusting the corresponding lighting behavior according to the predicted lighting intensity demand and the recognized current personnel activity behavior, where the lighting behavior includes lighting intensity, lighting color temperature, and lighting switch state; based on the corresponding lighting behavior adjusted by the home lighting adjustment algorithm, judging the lighting adjustment satisfaction of the home lighting adjustment algorithm according to the activity frequency of the personnel in the lighting area and the frequency of manually adjusting the lighting behavior, and iteratively optimizing the home lighting adjustment algorithm.

[0144] When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present invention are executed. The method, apparatus, and device of the present invention can be implemented in many ways. For example, the method, apparatus, and device of the present invention can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware.

[0145] The above order of steps for the method is for illustration purposes only. The steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated.

[0146] In addition, in some embodiments, the present invention can also be implemented as a program recorded in a recording medium, and these programs include machine-readable instructions for implementing the method according to the present invention. Therefore, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0147] Furthermore, parts of the above technical solutions provided in the embodiments of the present invention that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0148] The specific embodiments described above further elaborate on the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi-mode adaptive smart home lighting adjustment method, characterized in that: include: Collect indoor and outdoor environmental data through environmental sensors, and collect indoor character image data through cameras; Based on the historical environmental data collected by environmental sensors, a light intensity demand prediction model based on the long short-term memory network LSTM model is trained and established; Based on the indoor person image data collected by the camera, a convolutional neural network (CNN) model is trained and a person activity behavior recognition model is established; Construct a home lighting adjustment algorithm to continuously adjust the corresponding lighting behavior according to the predicted lighting intensity demand and the identified current personnel activity behavior, the lighting behavior including lighting intensity, lighting color temperature and lighting switch status; Based on the corresponding lighting behavior adjusted by the home lighting adjustment algorithm, the lighting adjustment satisfaction of the home lighting adjustment algorithm is judged according to the activity frequency of people in the lighting area and the frequency of manual lighting adjustment behavior, and the home lighting adjustment algorithm is iteratively optimized.

2. The multi-mode adaptive smart home lighting adjustment method according to claim 1, characterized in that: Install light intensity sensors and temperature and humidity sensors indoors and outdoors to collect environmental data; M sensor nodes are arranged indoors and numbered as (r1, r2, ..., r M ), N sensor nodes are arranged outdoors and numbered as (s1, s2, ..., s N ), each sensor node integrates a light intensity sensor and a temperature and humidity sensor; Assume that at time t, at the i-th indoor sensor node r i The collected illuminance is E i (t), temperature is T i (t), humidity is H i (t); at the jth outdoor sensor node s j The collected illuminance is E j ′(t), temperature is T j ′(t), humidity is H j ′(t); K cameras are arranged in the indoor area, numbered as (c1,c2,...,c K ); Let the kth camera c at time t k The image frame collected is I k (t); Environmental sensors and cameras work together to collect indoor and outdoor environment data and character image data.

3. The multi-mode adaptive smart home lighting adjustment method according to claim 2, characterized in that: Determine the input and output of the LSTM model. The input of the LSTM model is historical environmental data, including indoor and outdoor light intensity, temperature and humidity, as well as the light intensity setting value at the corresponding moment; the output of the LSTM model is the predicted value of light intensity demand at the future moment; Assume that the light intensity at time t is set to y (LSTM) (t), indoor sensor node r i The collected illumination, temperature and humidity are E i (t), T i (t) and H i (t), outdoor sensor node s j The collected illumination, temperature and humidity are E j ′(t), T j ′(t) and H j ′(t), then the environmental parameter vector at time t is: X (LSTM) (t)=[y (LSTM) (t),E i (t),T i (t),H i (t),E j ′(t),T j ′(t),H j ′(t)] The environmental parameter vectors of ζ historical moments constitute the input sequence: X (LSTM) =[X (LSTM) (t-ζ),…,X (LSTM) (t-1),X (LSTM) (t)]; the light intensity requirements for the next ζ′ moments form an output sequence: Y (LSTM) =[y (LSTM) (t+1),…,y (LSTM) (t+ζ′)]; And the LSTM model is represented as a mapping relationship: Y (LSTM) =F(X (LSTM) ), where ζ is the backtracking order; ζ′ is the prediction order; Design the network structure of the LSTM model. The number of input layer nodes of the LSTM model is N. in =ζ×3M+3N+1), the number of output layer nodes is N o =ζ′; the hidden layer adopts a double-layer LSTM structure, and the number of memory blocks in each layer is N h1 and N h2 , the storage unit dimension in the memory block is d; a fully connected structure is used between each layer; The goal of LSTM model training is to minimize the mean square error between the light intensity demand prediction sequence and the true value sequence.

4. The multi-mode adaptive smart home lighting adjustment method according to claim 3 is characterized in that: Train the LSTM model, collect historical data from environmental sensors and light controllers, align them in time and resample them at fixed time intervals to obtain a complete sequence of environmental data and light intensity setting values, and preprocess the collected environmental data; From the preprocessed historical data, extract the input and output sequences (X (LSTM) ,Y (LSTM) ) as a training sample; the step size of the sliding window is the prediction order ζ′, that is, the starting time of the next sample is the ending time of the previous sample; The order of training samples is randomly disrupted and divided into training set, validation set and test set in proportion; the training set is used for LSTM model parameter learning, the validation set is used for tuning and early stopping, and the test set is used to evaluate the performance of the LSTM model; Test the performance of the trained LSTM model on the test set to evaluate the generalization ability of the LSTM model; If the performance does not meet the requirements, improve the model by increasing the hidden layer dimension, extending the backtracking order, and introducing regularization terms; Based on the trained LSTM model, the future light intensity demand is predicted in real time. At time t, the environmental parameter vectors of the past ζ moments are input into the model to obtain the light intensity demand prediction sequence for the future ζ′ moments.

5. The multi-mode adaptive smart home lighting adjustment method according to claim 4 is characterized in that: The input of the CNN model is the sequence of human image frames collected by the camera, and the output is the category of human activity behavior corresponding to the image frame sequence; assuming that there are C predefined categories of human activity behavior, the model is represented by a mapping relationship: y (CNN) =F(X (CNN) ); where input is a character image frame sequence of length T, each frame image x T The size of is H×W×D, where H, W, and D represent the image height, width, and number of channels, respectively; output is the probability distribution of each category, satisfying Where C is the total number of categories of personnel activity behavior; Design the network structure of the CNN model, using the CNN structure plus temporal pooling architecture, use the CNN structure to extract the features of each frame, then perform temporal pooling on the feature sequence, and finally use the fully connected layer to output the classification results; The fully connected layer is removed from the CNN structure, and only the convolutional layer and the pooling layer are retained for feature extraction; assuming that the size of the feature map after the last layer of pooling in CNN is H'×W'×D', the feature map is flattened into a d-dimensional feature vector, where d = H'×W'×D', H', W' and D' respectively represent the height, width and number of channels of the feature map after the last layer of pooling in the CNN model; The temporal pooling part performs a pooling operation on the d-dimensional feature vectors of T time steps extracted by the CNN structure to obtain a d-dimensional feature representation; After pooling, the feature representation z=[z1,…,z d ], it is mapped to the C-dimensional output space through the fully connected layer, and then converted into a probability distribution using the softmax function: y (CNN) =softmax(ψz+b); where ψ and b are the weight matrix and bias vector of the fully connected layer respectively; The goal of CNN model training is to minimize the cross entropy loss between the predicted probability distribution and the true label, while adding an L2 regularization term to prevent overfitting.

6. The multi-mode adaptive smart home lighting adjustment method according to claim 5, characterized in that: Extract image frames from the video stream collected by the camera at fixed time intervals, and take T consecutive frames as a sample; at the same time, manually label each sample to indicate the category of human activity behavior to which the sample belongs, and pre-process the image sample; Divide the sample set into training set, validation set and test set in proportion; train the model on the training set, adjust parameters and select the optimal model on the validation set, and evaluate the model performance on the test set; The model is trained using the SGD optimization algorithm with the minimum batch method. Each batch is input in turn in each cycle and the model parameters are updated. After all batches are trained, the next cycle begins. During the training process, the model performance is evaluated on the validation set every preset number of cycles. If the performance does not improve after Q consecutive cycles, the training is terminated. Test the trained CNN model on the test set to evaluate the generalization performance of the CNN model on new samples; Deploy the trained human activity behavior recognition model to analyze the human image data collected by the camera in real time; When a person is detected in the picture, T consecutive frames of images are extracted and input into the model to obtain the category of the person's activity behavior in the current time period.

7. The multi-mode adaptive smart home lighting adjustment method according to claim 6, characterized in that: According to the M sensor nodes arranged indoors, each sensor node corresponds to a lighting area at the same time. There are M independently adjusted lighting areas in the smart home environment, numbered as m=1,…,M; the lighting behavior of the mth lighting area includes the lighting intensity I m ∈{I1,…,I M }、Light color temperature T m ∈{T1,…,T M } and light switch status S m ∈{0,1}; Assume that at time t, the average light intensity requirement at the future time τ′ predicted by the LSTM model is The current activity category identified by the CNN model is A t , A t ∈{A1,…,A C }; Adjust the light intensity of the illuminated area according to the predicted light intensity demand. The value of proportionally adjusts the illumination intensity of the mth illumination area where α m ∈[0,1],α m is the intensity distribution coefficient of the mth illumination area, satisfying Adjust the lighting color temperature of the lighting area according to the activity category of the personnel, and predefine the color temperature mapping table Classify the personnel activities into A t Mapping to recommended light color temperature value Adjust the lighting switch state of the lighting area according to the personnel activity area, and set the space corresponding to the mth lighting area as R m , the set of illumination areas where human activity is detected at time t is Then the light switch control rule is: When S m =1, indicating that the light switch is on; when When S m =0, indicating that the light switch is in the off state.

8. The multi-mode adaptive smart home lighting adjustment method according to claim 7, characterized in that: According to the M sensor nodes arranged indoors, each sensor node corresponds to a lighting area at the same time. It is assumed that there are M lighting areas in the smart home environment, numbered as i=1,…,M, and each lighting area corresponds to a type of human activity behavior A i , A i ∈{A1,…,A C }; For region i, within the current τ time, define n i,τ represents the number of times human activities are detected in the illumination area i within the current τ time; f i,τ Indicates the number of manual adjustments made by the user to the illumination parameters of illumination area i at the current time τ; They represent the average light intensity, light color temperature and light switch state of the lighting area i decided by the home lighting adjustment algorithm within the current τ time; Introducing lighting adjustment satisfaction i,τ It represents the user's satisfaction with the lighting adjustment effect of lighting area i within the current τ time, and constructs the lighting adjustment satisfaction calculation formula: Among them, r i,τ ∈[0,1], α is the scaling factor, α>0, exp[·] represents the exponential function with e as the base; According to the statistics of n i,τ and f i,τ , calculate the lighting adjustment satisfaction r of each area i,τ ; Based on the lighting adjustment satisfaction r i,τ , and adaptively adjust the relevant parameters of the lighting adjustment algorithm.

9. A multi-mode adaptive smart home lighting adjustment system, which is used to implement the multi-mode adaptive smart home lighting adjustment method according to any one of claims 1 to 8, characterized in that: include: Data acquisition module, light intensity prediction module, behavior type recognition module, light adjustment module and iterative optimization module; The data acquisition module collects indoor and outdoor environmental data through environmental sensors and collects indoor character image data through cameras; The light intensity prediction module trains and establishes a light intensity demand prediction model based on a long short-term memory network LSTM model based on historical environmental data collected by environmental sensors; The behavior type recognition module is based on the indoor person image data collected by the camera and is based on the convolutional neural network (CNN) model training and establishes a person activity behavior recognition model; The lighting adjustment module constructs a home lighting adjustment algorithm, and continuously adjusts the corresponding lighting behavior according to the predicted lighting intensity demand and the identified current personnel activity behavior. The lighting behavior includes lighting intensity, lighting color temperature and lighting switch state; The iterative optimization module determines the lighting adjustment satisfaction of the home lighting adjustment algorithm based on the corresponding lighting behavior adjusted by the home lighting adjustment algorithm, according to the activity frequency of personnel in the lighting area and the frequency of manual lighting adjustment behavior, and iteratively optimizes the home lighting adjustment algorithm.

Images