Sleep management optimization system and method based on image recognition and environmental perception

Abstract

The application discloses a sleep management optimization system and method based on image recognition and environment perception, comprising an image recognition module: collecting image data, obtaining first sleep state data from the image data through an optimized bionic neural network; an environment perception module: collecting sleep environment data, obtaining time dimension features and space dimension features from the sleep environment data through a space-time environment sleep quality correlation model, and obtaining second sleep state data through a small neural network adaptive weight adjustment; a sleep management module: generating a continuously optimized sleep management strategy based on the first sleep state data and the second sleep state data through an improved reinforcement learning strategy, which can provide personalized suggestions for sleepers according to different sleep states and environment conditions, and meet diversified needs of different users.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sleep management, and particularly relates to a sleep management optimization system and method based on image recognition and environment perception. BACKGROUND

[0002] In modern life, sleep quality is crucial to people's physical and mental health, work efficiency and quality of life. However, there are many limitations in current sleep management technology. However, from the perspective of sleep state monitoring, traditional sleep monitoring methods such as polysomnography (PSG) are accurate, but the equipment is complex and needs to be performed in a hospital environment, which causes great inconvenience to the daily life of sleepers and makes it difficult to achieve long-term, real-time home monitoring. Although some wearable devices can monitor sleep data to some extent, they mainly focus on motion, heart rate and other aspects, and have limited access to key information such as sleepers' facial expressions and body postures, which cannot fully reflect the sleep state.

[0003] In existing sleep management systems, it is difficult to accurately determine the specific impact of environmental factors on sleep quality, and it is difficult to develop personalized and precise sleep management strategies. When sleepers experience sleep interruption, it is difficult to quickly determine whether it is caused by environmental noise, temperature changes or other factors, so it is difficult to take targeted measures to improve sleep in a timely manner. Therefore, a sleep management optimization system and method based on image recognition and environment perception are proposed to solve the above problems. SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art, in order to achieve the above-mentioned purpose, the present application proposes the following technical scheme:

[0005] The sleep management optimization system and method based on image recognition and environment perception comprise:

[0006] The image recognition module: acquires image data, and obtains first sleep state data from the image data through an optimized bionic neural network;

[0007] The environment perception module: acquires sleep environment data, obtains time dimension features and space dimension features from the sleep environment data through a space-time environment sleep quality correlation model, and obtains second sleep state data through self-adaptive adjustment of weights by a small neural network;

[0008] The sleep management module: generates a continuously optimized sleep management strategy based on the first sleep state data and the second sleep state data through an improved reinforcement learning strategy;

[0009] The hidden layer of the optimization bionic neural network comprises a feature extraction sublayer and a feature fusion sublayer, the feature extraction sublayer randomly initializes the convolution kernel of the original convolutional neural network according to biological neurons, and adjusts the convolution kernel weight according to reinforcement learning, and the feature fusion sublayer fuses features through an attention mechanism.

[0010] The improved reinforcement learning strategy comprises an optimization policy network, and the optimization policy network introduces a biological neuron connection weight adjustment mode to initialize policy network parameters.

[0011] The image data comprises facial expression image data and body posture image data.

[0012] The image data is converted into a matrix form after gray scale processing and normalization operation and is input into the input layer of the optimization bionic neural network.

[0013] The process of obtaining the first sleep state data through the optimization bionic neural network comprises the following steps:

[0014] The optimization bionic neural network comprises an input layer, a hidden layer and an output layer.

[0015] The input layer of the optimization bionic neural network receives image data converted into a matrix form after gray scale processing and normalization operation.

[0016] The hidden layer outputs a feature map through the feature extraction sublayer and obtains a fused feature map through the feature fusion sublayer.

[0017] The output layer converts the fused feature map into a vector form through a full connection layer, and then obtains the first sleep state data by using a Softmax function to calculate the probability distribution of different sleep states of the fused feature map.

[0018] The hidden layer is specifically implemented in the following manner:

[0019] The convolution operation of the convolutional neural network is taken as the basic operation of the feature extraction sublayer, and the convolution kernel of the convolutional neural network is randomly initialized according to biological neurons.

[0020] After the random initialization, the convolution kernel is trained.

[0021] During the training of the convolution kernel, the convolution kernel weight is adjusted according to reinforcement learning, the convolution kernel is taken as an agent of reinforcement learning, the agent interacts with a sleep environment, learns an optimal strategy according to a feedback reward signal, and after the adjustment, performs a convolution operation on input image data to output a feature map.

[0022] The feature fusion sub-layer obtains the importance dimension of each feature map through an attention mechanism, obtains the attention weight of each feature map in sleep state recognition through the importance dimension, and realizes feature map fusion through weighted summation based on the attention weight to obtain a fused feature map.

[0023] The importance dimension is obtained by constructing an association model of facial expression features and light sleep state.

[0024] The association model of facial expression features and light sleep state uses a logistic regression algorithm in machine learning, takes a facial expression feature vector as input, and takes a light sleep state as output label, trains the model using labeled facial image data, and outputs the importance dimension corresponding to different feature maps.

[0025] The sleep environment data includes temperature data, light data, noise data, and air quality data.

[0026] The space-time environment sleep quality association model includes a part for capturing the trend of sleep environment data over time and a part for capturing the spatial association between sensor nodes of sleep environment data.

[0027] The part for capturing the trend of sleep environment data over time uses a long short-term memory network to capture the trend of sleep environment data over time, generates a hidden state reflecting the time characteristics of the environment parameters at the current time point, and obtains a time feature dimension.

[0028] The part for capturing the spatial association between sensor nodes of sleep environment data uses a graph convolution network to model the sensor network, treats different sensor nodes as graph vertices, and treats physical connections or correlations between nodes as edges. A degree matrix is obtained by adding a self-loop to the adjacency matrix before graph convolution operation, and a spatial feature dimension is obtained through graph convolution operation.

[0029] The number of neurons in the input layer, hidden layer and output layer of the small neural network is obtained based on the time feature dimension and the spatial feature dimension.

[0030] The small neural network adaptively adjusts the importance weight of the time feature dimension and the spatial feature dimension, and then outputs the second sleep state data through the space-time environment sleep quality association model.

[0031] The improved reinforcement learning strategy construction process is:

[0032] The improved reinforcement learning strategy includes an optimization strategy network, a value network, and a reward function.

[0033] The optimization strategy network applies a biological neuron connection weight adjustment mode to the optimization strategy network parameter initialization, and outputs a probability distribution of taking different sleep intervention actions after initialization.

[0034] The value network obtains a sleep state based on the first sleep state data and the second sleep state data, and evaluates an expected value of sleep quality improvement after taking all possible actions in the current sleep state based on the sleep state;

[0035] The reward function components include sleep efficiency improvement, wake-up frequency reduction, deep sleep time increase and user satisfaction with the intervention strategy.

[0036] The process of obtaining the continuously optimized sleep management strategy is:

[0037] The process of collecting sleep states, intervention actions and training the strategy update process through the reward function based on the improved reinforcement learning strategy, and the importance sampling technique is used to reduce the sampling error to obtain the target function of the strategy update during the training process;

[0038] The optimal sleep management strategy is generated by iteratively optimizing the strategy network parameters through the target function.

[0039] The sleep management optimization method based on image recognition and environmental perception, the method steps include:

[0040] S1: Collect image data, and obtain first sleep state data from the image data through an optimized bionic neural network;

[0041] S2: Collect sleep environment data, obtain time dimension features and space dimension features from the sleep environment data through a space-time environment sleep quality correlation model, and obtain second sleep state data by adaptively adjusting the weights through a small neural network;

[0042] S3: Based on the first sleep state data and the second sleep state data, a continuously optimized sleep management strategy is generated through an improved reinforcement learning strategy.

[0043] The present application has the following advantages:

[0044] In the present application, first, the collected image data is processed through an optimized bionic neural network to obtain first sleep state data, including the facial expression, body posture, and turning frequency of the sleeper, etc. Compared with traditional monitoring methods, the monitoring indicators are more comprehensive, and the physiological and psychological state of the sleeper is reflected in depth. At the same time, the environmental perception module collects and analyzes environmental data such as temperature, light, noise, and air quality through a space-time environment sleep quality correlation model to obtain second sleep state data, accurately capturing the influence of the environment on sleep quality from the time and space dimensions. This multi-dimensional sleep state monitoring method provides a more comprehensive and accurate data basis for subsequent sleep management.

[0045] Secondly, by optimizing the biomimetic neural network to dynamically adjust the weights of the convolution kernels based on reinforcement learning, it is made to better fit the sleep state recognition task. At the same time, the small neural network in the spatiotemporal environment sleep quality correlation model can dynamically adjust the weights of time and space dimension features based on historical data and real-time feedback, thereby improving the accuracy of sleep quality assessment.

[0046] Finally, the data from the first and second sleep states are fused, and a continuously optimized sleep intervention strategy is generated through an improved reinforcement learning strategy. The optimization strategy network in the improved reinforcement learning strategy is given initial values ​​to the corresponding parameters by adjusting the connection weights of biological neurons. This allows the network to be sensitive to sleep environment characteristics in the initial stage, fully considering the individual differences of sleepers and the dynamic changes of the sleep environment. Personalized suggestions are provided to sleepers based on different sleep states and environmental conditions to meet the diverse needs of different users. Attached Figure Description

[0047] Figure 1 This is a system block diagram of the sleep management optimization system and method based on image recognition and environmental perception proposed in this invention.

[0048] Figure 2 This diagram illustrates the steps of the sleep management optimization system and method based on image recognition and environmental perception proposed in this invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Example 1

[0051] like Figure 1 As shown, the sleep management optimization system based on image recognition and environmental perception proposed in this invention includes:

[0052] Image recognition module: Collects image data and obtains the first sleep state data from the image data through an optimized bionic neural network;

[0053] An infrared night vision camera is deployed in the sleep environment, installed vertically downwards above the head of the bed, to capture images at a frequency of 2-3 frames per second. This captures facial expression (relaxed, tense, sad) and body posture (supine, lateral, prone) data of the sleeper, and converts the color images to grayscale images using the following formula:

[0054] G = 0.299g1 + 0.587g2 + 0.114g3

[0055] wherein g1, g2, g3 are red, green, blue components of the color image respectively;

[0056] The range of the pixel value of the gray image is uniformly adjusted to [0, 1], and then the image data in the form of a matrix is constructed;

[0057] Specifically, although the color image contains rich color information, in the sleep state recognition task, part of the color information is not helpful for judging the sleep state, but will increase the data dimension and processing complexity. After the image data grayscale processing and normalization operation, each pixel point of the image corresponds to an element in the matrix;

[0058] The matrix form image data is input into the optimized bionic neural network;

[0059] The construction process of the optimized bionic neural network is:

[0060] The optimized bionic neural network includes an input layer, a hidden layer, and an output layer;

[0061] The hidden layer is composed of a feature extraction sublayer and a feature fusion sublayer;

[0062] The input layer receives the preprocessed image data matrix, and the input layer arranges and organizes the input image data matrix through preliminary integration of the data, and orderly enters the subsequent hidden layer;

[0063] The construction process of the feature extraction sublayer is:

[0064] The convolution operation of the convolutional neural network (CNN) is used as the basic operation of the feature extraction sublayer, and the convolution kernel of the convolutional neural network is randomly initialized according to the biological neuron;

[0065] After random initialization, the convolution kernel training is performed, and in the convolution kernel training process, the convolution kernel weight is adjusted according to the reinforcement learning;

[0066] In reinforcement learning, the agent interacts with the sleep environment, and learns the optimal strategy according to the feedback reward signal. In the feature extraction sublayer, the convolution kernel is the agent, and the extracted features are the environmental feedback for sleep state recognition. When the features extracted by the convolution kernel can accurately indicate the sleep state (such as the rapid rotation feature of the eyes during rapid eye movement sleep), a positive reward is given, and the weight is enhanced; otherwise, a negative reward is given, and the weight is weakened;

[0067] The process of adjusting the convolution kernel weight according to the reinforcement learning is:

[0068] A reward function R is defined, which takes a larger positive value when the feature extracted by the convolution kernel accurately identifies the sleep state (e.g., is identified as light sleep and is actually light sleep), and takes a negative value when the identification is wrong.

[0069] The formula for updating the convolution kernel weight by adjusting the weight step size to be a is as follows:

[0070] k +1 = k + aR

[0071] For example, in a certain calculation, the eye feature extracted by the convolution kernel accurately indicates rapid eye movement sleep, the reward function R = 0.5, the step size a = 0.1, and the original weight k = 0.3. Then the updated weight k +1 = 0.3 + 0.1 x 0.5 = 0.35, and the updated weight increases, making the convolution kernel more likely to extract such key features subsequently.

[0072] After adjusting the convolution kernel weight, the convolution kernel performs convolution operation on the input image data, and outputs a feature map Y, which is represented by the formula as follows:

[0073]

[0074] where q and p are the size of the convolution kernel, X is the coordinate of the output feature map, and k is the convolution kernel.

[0075] Specifically, the optimized bionic neural network is based on the convolution operation of the convolutional neural network, combined with the characteristics of biological neurons, so that the convolution kernel fits the working mode of the biological nervous system to improve the targeting of feature extraction. At the same time, reinforcement learning is introduced to dynamically adjust the convolution kernel weight, changing the traditional CNN fixed or simple rule weight adjustment method. Through the reward mechanism closely combined with the sleep state recognition task, the convolution kernel is constantly self-optimized during the training process.

[0076] The feature fusion sub-layer construction process is as follows:

[0077] The feature maps obtained by different feature extraction sub-layers contain different aspects of image information, such as facial expression features and body posture features. The feature fusion sub-layer will optimally fuse these feature maps to obtain a more comprehensive image representation.

[0078] Through an attention mechanism, the corresponding attention weight is obtained based on the importance dimension of each feature map in sleep state recognition. The importance dimension is obtained by analyzing a large number of labeled sleep image data to construct a facial expression feature and light sleep state association model. Let the facial expression feature vector be F = (f1, f2,.., f n ), where f nrepresent the n-th facial feature parameter, such as representing the eye muscle relaxation degree, representing the facial micro-expression change frequency, at the same time, the association model of the facial expression feature and the light sleep state is constructed to obtain the form:

[0079] S light-sleep = w1f1+ w2f2+..+ w n f n

[0080] wherein S light-sleep is the output (i.e. the importance dimension) of the association model of the facial expression feature and the light sleep state, w n is the weight coefficient of the n-th machine learning algorithm, through the logistic regression in the machine learning algorithm, taking the facial expression feature vector as the input and taking the light sleep state (yes or no) as the output label, a large number of labeled facial image data (divided into training set and validation set) is used to train the model, after the training is completed, the model will output the importance dimension S light-sleep corresponding to different feature maps;

[0081] Suppose the number of feature maps obtained by the feature extraction sub-layer is N, for two feature maps Y i (i∈N) and Y j (j∈N), the attention weight formula is represented as:

[0082]

[0083] wherein a represents the attention weight, s i ∈S light-sleep is the importance dimension of the feature map Y i , and s j ∈S light-sleep is the importance dimension of the feature map Y j ;

[0084] After determining the attention weight a of each feature map, the weighted sum formula is used to realize the feature map fusion:

[0085]

[0086] wherein Q is the fused feature map, i.e. the output of the feature fusion sub-layer;

[0087] Specifically, for the output feature map of the feature extraction sub-layer, the attention mechanism is introduced, the feature map importance is determined through the deep analysis of a large number of sleep image data, and the association model of the facial expression feature and the sleep state is constructed to determine the feature map importance dimension, combined with the characteristics of the sleep image, different from the general feature fusion method, this specific model construction for the sleep field can better tap the potential value of different feature maps in the sleep state recognition, and provide more effective feature representation for the sleep state judgment;

[0088] The feature fusion sub-layer outputs the fused feature map as the output of the output layer. After feature extraction and fusion of the hidden layer, a fully connected layer is set in the output layer to convert the fused feature map into a vector form. Then, the Softmax function is used to calculate the probability distribution of the fused feature map in different sleep states (light sleep, deep sleep, and rapid eye movement sleep), which is expressed by the formula:

[0089]

[0090] where Q i is the input value (fused feature map) of the nth output layer, M is the number of output layer neurons, and P is the probability representation of different sleep states.

[0091] Specifically, by starting from matrix-form image data and processing through each layer of the network, the probability information of the sleeper in different sleep states and the different facial expression and body posture feature information of the sleeper corresponding to different sleep states are finally obtained, which constitute the first sleep state data [J].

[0092] The environmental perception module collects sleep environment data, obtains time dimension features and space dimension features from the sleep environment data through a space-time environment sleep quality correlation model, and obtains the second sleep state data by adaptively adjusting the weights through a small neural network.

[0093] The sleep environment data in the sleep state is collected, temperature data, light data, noise data, and air quality data are collected through different types of sensors, and the influence of the sleep environment on the sleep quality is preliminarily understood through these data. Then, data normalization preprocessing is performed, which is used as the input of a space-time environment sleep quality correlation model, and the input sleep environment data is denoted as E.

[0094] The construction process of the space-time environment sleep quality correlation model is as follows:

[0095] Capture the time-varying trend of sleep environment data;

[0096] The long short-term memory network is used to capture the time-varying trend of sleep environment data, and the hidden state H reflecting the time characteristics of the environmental parameters at the current time point is generated. The hidden state is constantly updated through the input gate, the forgetting gate, and the output gate, and the core formula is expressed as:

[0097] H t = o t tanh(c t )

[0098] where H tis the hidden state of the input data at time t, which integrates the current input environmental data E and the previously memorized information, is a feature representation output by the LSTM network at time t, t is the activation value of the output gate, which controls the proportion of memory cell information output, c t is the memory cell that stores long-term dependency information, and tanh is the hyperbolic tangent function.

[0099] Specifically, the hidden state H t The values of each dimension will change over time and with changes in environmental parameters. For example, if the sleep environment temperature continues to decrease, the values of the temperature-related dimensions will change accordingly. By observing the changes in the hidden state H t at different times, we can reflect the downward trend of sleep environment data over time. The hidden state H t is the time dimension feature.

[0100] Capture the spatial correlation between sensor nodes in the sleep environment data.

[0101] The sensor data at different positions in the sleep environment have spatial correlations. We use a graph convolutional network (GCN) to model the sensor network, treating different sensor nodes as graph vertices and the physical connections or correlations between nodes as edges. Let the adjacency matrix of the graph be A and the node feature matrix be X (E) (the initial stage node feature matrix obtained based on the fused environmental data E).

[0102] Before performing graph convolution operations, we add self-loops to the adjacency matrix A , where I is the identity matrix. The purpose of adding self-loops is to allow each node to consider its own feature information and enhance the model's ability to capture the characteristics of the node itself.

[0103] Get the degree matrix D , where D represents the connection degree of all nodes in the sleep environment data.

[0104] The node feature matrix Z after graph convolution operation based on the degree matrix D is represented as:

[0105]

[0106] where W is the weight matrix in the graph convolution operation, represents the normalization processing of the adjacency matrix, balancing the connection weights between nodes, so that the output node feature matrix Z can more reasonably consider the correlation strength between different nodes, and σ is the activation function.

[0107] The spatial correlation between sensor nodes is captured by the graph convolution operation, which comprehensively considers the characteristics of each node itself and the connection relationship with adjacent nodes in the sleep environment data, and extracts the node feature matrix Z reflecting the spatial correlation by transforming the node feature matrix X (E) The node feature matrix Z, i.e., the spatial dimension feature, is used to obtain the distribution pattern of temperature, humidity and other parameters in the sleep environment, and to judge the correlation between the environmental conditions in different regions.

[0108] Further, adaptive fusion weights are assigned to the time dimension feature H and the spatial dimension feature Z, respectively, the importance weight of the time dimension feature is W (H) , the importance weight of the spatial dimension feature is W (Z) , a small neural network is used to learn the historical environment data and sleep quality data, and the values of W (H) and W (Z) are dynamically adjusted;

[0109] Specifically, the small neural network includes an input layer, a hidden layer and an output layer, the input layer neurons are the lengths of the time dimension feature H and the spatial dimension feature Z, the hidden layer uses the ReLU function, and the number of neurons is wherein, u1 is the number of input layer neurons, u2 is the number of output layer neurons, γ is an adjustment constant, and the number of output layer neurons u2 is two, corresponding to the importance weights W (H) and W (Z) ;

[0110] When the small neural network finds that the recent sleep quality fluctuation is more related to the change of environmental parameters in time (such as continuous change of temperature at night), the importance weight of the time dimension feature W (H) is increased, and the importance weight of the spatial dimension feature W (Z) is correspondingly reduced, if the environmental difference in space (such as noise difference at different positions in the room) has greater influence on sleep quality, the importance weight of the spatial dimension feature W (Z) is increased;

[0111] The final spatio-temporal environment-sleep quality correlation model is represented as:

[0112] [E] = W (H) H + W (Z) Z

[0113] wherein, W (Z) and W (H) represent the importance weights adjusted in real time by the small neural network, and [E] represents the second sleep state data output by the spatio-temporal environment-sleep quality correlation model through sleep environment data;

[0114] The sleep environment data is used to train the spatio-temporal environment-sleep quality correlation model, a mean square error loss function is used in the training process, and the importance weight in the model is continuously adjusted in real time through an optimization algorithm (here, a small neural network), so that the model can evaluate the sleep quality in real time according to the spatio-temporal (time dimension and space dimension) correlation environment data, and obtain second sleep state data;

[0115] Specifically, by correlating the environmental characteristics of the time dimension and the space dimension, the characteristics of the time and space dimensions are deeply mined, the long short-term memory network and the graph convolution network are used to extract the optimized features of the time dimension and the space dimension respectively, and then the features are organically fused, and the small neural network is introduced to dynamically adjust the importance weight of the time and space dimension features. Unlike the traditional fixed weight or simple heuristic weight adjustment method, the final correlation model can automatically optimize the weight in real time according to the actual sleep environment data and sleep quality feedback, adapt to different sleep scenarios and individual differences, and provide more accurate basis for sleep quality evaluation and intervention.

[0116] Sleep management module: generating a continuously optimized sleep management strategy based on the first sleep state data and the second sleep state data through an improved reinforcement learning strategy;

[0117] The first sleep state data and the second sleep state data are fused to obtain comprehensive sleep state data U = {[E], [J]};

[0118] The improved reinforcement learning strategy construction process is:

[0119] The improved reinforcement learning strategy includes an optimization strategy network, a value network and a reward function;

[0120] The construction process of the optimization strategy network and the value network is:

[0121] The different sleep features in the fused comprehensive sleep state data U are used as the sleep state of the improved reinforcement learning strategy

[0122] The optimization strategy network is defined as Wherein, θ is the parameter of the optimization strategy network, and a is the intervention action for different sleep states;

[0123] The biological neuron connection weight adjustment mode is applied to the optimization strategy network parameter θ initialization, and the optimization strategy network Finally, the probability distribution of taking different sleep intervention actions a is outputted;

[0124] Specifically, in the traditional policy network, the parameters θ are usually initialized randomly, which does not take into account the specific information related to sleep, resulting in low learning efficiency of the network in the initial stage. In order to optimize the policy network, the biological neuron connection weight adjustment mode is introduced to initialize the parameters θ. The implementation process of the connection weight adjustment mode is as follows:

[0125] There is a specific connection weight distribution rule between biological neurons. For example, when an individual feels a decrease in external temperature during sleep, the temperature receptor neurons in the skin will be activated. These neurons are connected to other neurons that transmit temperature information through synapses, and the connection weight will be adjusted according to the intensity and frequency of the stimulus. The biological neuron connection weight adjustment process between neuron 1 and neuron 2 is represented as follows:

[0126]

[0127] wherein, represents the change in connection weight between neuron β1 and neuron β2, η is a learning rate parameter that controls the magnitude of weight change, f(s temp is a function that reflects the intensity and characteristics of the temperature change coefficient s temp on the influence of weight adjustment, β1 and β2 are the activation values of neuron 1 and neuron 2, respectively, representing the activity level of the neurons when receiving stimuli;

[0128] According to the above biological neuron connection weight adjustment mode, the initial values of the corresponding parameters are assigned, so that the network can be sensitive to the sleep environment characteristics such as temperature (or other light data, noise data, air quality data) in the initial stage. For other related sleep environment characteristics, the parameters are initialized according to the temperature adjustment mode of the neuron connection weight change;

[0129] The value network is defined as is the parameter of the value network, and the value network estimates the value of the current sleep state , i.e. evaluates the expected value of improving sleep quality after taking all possible actions in the current sleep state;

[0130] The reward function is defined as The components of the reward function include the amount of sleep efficiency improvement the amount of reduction in wake-up times the amount of increase in deep sleep time ΔT deep and the user's satisfaction with the intervention strategy ΔS.Considering these components comprehensively, the reward obtained after taking a certain sleep intervention action is measured, and the optimization policy network is guided to learn a better strategy according to the reward, which is expressed by the formula:

[0131]

[0132] wherein, is the execution cost of the intervention strategy, w1 is the weight of the sleep efficiency improvement amount , w2 is the weight of the wake-up frequency reduction amount , w3 is the weight of the deep sleep time increase amount ΔT deep , w4 is the weight of the user's satisfaction ΔS with the intervention strategy, and w5 is the weight of the execution cost of the intervention strategy.

[0133] The sleep state U and the intervention action a at multiple time steps are collected, and the reward function is used to train the update process of the strategy. In the training process, the importance sampling technique is used to reduce the sampling error, and a target function for policy update is set:

[0134]

[0135] wherein, denotes the expectation operator, min(·) is the minimum value function, clip is the clipping function, r (θ) denotes the ratio between the new and old strategies, is the advantage estimate value, and θ is the optimization policy network parameter.

[0136] Specifically, the expectation operator represents the expectation of the quantity at the time step, and the expectation of the quantity is used to comprehensively consider the situation at different time steps, so as to avoid being excessively affected by individual abnormal time step data and obtain a more representative result for policy update. The minimum value function min(·) aims to prevent the step length from being too large during policy update, and the clipping function clip is used to limit the change range of the action probability between the current policy and the old policy, so that the ratio r (θ) between the new and old strategies is limited in (1-∈, 1+∈), which prevents the policy update from being too aggressive and causing the model to be unstable. ∈ is the clipping parameter, which limits the change range of r (θ) , and the advantage estimate value is used to obtain the advantage degree of the current intervention action a relative to the average action, and to evaluate the additional reward brought by the action in the current state compared with other actions.

[0137] The policy network parameter θ is iteratively optimized (i.e., continuously adjusted) through the target function L for policy update, so that the long-term cumulative reward of the optimization policy network parameter is maximized, thereby generating an optimal sleep management strategy.

[0138] According to the optimal strategy generated by the improved reinforcement learning strategy, a relevant sleep management decision is made.

[0139] For example:

[0140] According to the first sleep state data (based on image data) and the sleep state, it is found that the sleeper has whole body muscle tension, posture stiffness and the like before sleep, and the optimal strategy suggests that a progressive muscle relaxation exercise is performed 20 minutes before sleep;

[0141] According to the second sleep state data (based on sleep environment data), the optimal strategy suggests that the best sleep environment is temperature 24℃, humidity 50%, light intensity 5lux, and noise control below 30dB.

[0142] Embodiment two

[0143] As shown in Figure 2 The sleep management optimization method based on image recognition and environment perception provided by the present application has the following method steps:

[0144] S1: Collect image data, and obtain first sleep state data from the image data through an optimized bionic neural network;

[0145] S2: Collect sleep environment data, obtain time dimension features and space dimension features from the sleep environment data through a space-time environment sleep quality correlation model, and obtain second sleep state data through adaptive adjustment of weights by a small neural network;

[0146] S3: Based on the first sleep state data and the second sleep state data, a continuously optimized sleep management strategy is generated through an improved reinforcement learning strategy.

[0147] In the application, several formulas involved are calculated by taking their values after dimensionless, and the establishment of the formula is obtained by a formula of the nearest real situation through software simulation of a large amount of data, and part of the coefficients or weights in the formula are set by the person skilled in the art according to the actual situation, so here is not more elaborated.

[0148] The above embodiments can be realized by software, hardware, firmware or any combination thereof, in whole or in part. When realized by software, the above embodiments can be realized in the form of a computer program product in whole or in part. Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution.

[0149] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. A sleep management optimization system based on image recognition and environmental perception, characterized in that, Comprise: An image recognition module: collect image data, and obtain first sleep state data from the image data through an optimized bionic neural network; The image data comprises facial expression image data and body posture image data; An environment perception module: collect sleep environment data, and obtain time dimension features and space dimension features from the sleep environment data through a space-time environment sleep quality correlation model, and obtain second sleep state data through adaptive adjustment of weights by a small neural network; The space-time environment sleep quality correlation model comprises a part for capturing the change trend of sleep environment data over time and a part for capturing the spatial correlation between sensor nodes of sleep environment data; The part for capturing the change trend of sleep environment data over time uses a long short-term memory network to capture the change trend of sleep environment data over time, generates a hidden state reflecting the time characteristics of the environment parameters at the current time point, and obtains the time feature dimension; The part for capturing the spatial correlation between sensor nodes of sleep environment data uses a graph convolution network to model the sensor network, regards different sensor nodes as vertices of a graph, regards physical connections or correlations between nodes as edges, adds self-loops to the adjacency matrix to obtain a degree matrix before graph convolution operation, and obtains the space feature dimension through graph convolution operation; A sleep management module: generate a continuously optimized sleep management strategy based on the first sleep state data and the second sleep state data through an improved reinforcement learning strategy; The improved reinforcement learning strategy construction process is: The improved reinforcement learning strategy comprises an optimization policy network, a value network, and a reward function; The optimization policy network applies a biological neuron connection weight adjustment mode to the optimization policy network parameter initialization, and outputs the probability distribution of taking different sleep intervention actions after initialization is completed; The value network obtains the sleep state based on the first sleep state data and the second sleep state data, and evaluates the expected value of improving the sleep quality after taking all possible actions in the current sleep state based on the sleep state; The reward function components include sleep efficiency improvement, wake-up frequency reduction, deep sleep time increase, and user satisfaction with the intervention strategy; The hidden layer of the optimized bionic neural network comprises a feature extraction sublayer and a feature fusion sublayer, the feature extraction sublayer randomly initializes the convolution kernel of the original convolutional neural network according to biological neurons, adjusts the convolution kernel weight according to reinforcement learning, and the feature fusion sublayer fuses features through an attention mechanism; The improved reinforcement learning strategy comprises an optimization policy network, and the optimization policy network introduces a biological neuron connection weight adjustment mode to initialize the policy network parameters.

2. The image recognition and environment perception based sleep management optimization system of claim 1, wherein, The process of obtaining first sleep state data through an optimized bionic neural network is: The optimized bionic neural network comprises an input layer, a hidden layer, and an output layer; The image data is converted into a matrix form after grayscale processing and normalization operation, and is input into the input layer of the optimized bionic neural network; The hidden layer outputs a feature map through a feature extraction sublayer, and obtains a fused feature map through a feature fusion sublayer; The output layer converts the fused feature map into a vector form through a full connection layer, and then obtains the first sleep state data by using a Softmax function to calculate the probability distribution of the sleep state of the fused feature map in different sleep states.

3. The image recognition and environment perception based sleep management optimization system of claim 2, wherein, The hidden layer is specifically implemented as follows: The convolution operation of the convolutional neural network is taken as the basic operation of the feature extraction sublayer, and the convolution kernel of the convolutional neural network is randomly initialized according to biological neurons; After random initialization, the convolution kernel is trained; During the convolution kernel training process, the convolution kernel weight is adjusted according to reinforcement learning, the convolution kernel is taken as an agent of reinforcement learning, the agent interacts with the sleep environment, and the optimal strategy is learned according to the feedback reward signal, and after the adjustment is completed, the convolution operation is performed on the input image data to output the feature map; The feature fusion sublayer obtains the importance dimension of each feature map through an attention mechanism, obtains the attention weight of each feature map in sleep state recognition through the importance dimension, and realizes feature map fusion based on the attention weight to obtain a fused feature map.

4. The image recognition and environment perception based sleep management optimization system of claim 3, wherein, The importance dimension is obtained by constructing an association model of facial expression features and light sleep states. The association model of facial expression features and light sleep states uses a logistic regression algorithm in machine learning, takes a facial expression feature vector as input, and takes a light sleep state as output label, uses labeled facial image data to train the model, and outputs the importance dimension corresponding to different feature maps.

5. The image recognition and environment perception based sleep management optimization system of claim 1, wherein, The sleep environment data includes temperature data, light data, noise data, and air quality data.

6. The image recognition and environment perception based sleep management optimization system of claim 1, wherein, The number of neurons in the input layer, hidden layer and output layer of the small neural network is obtained based on the time feature dimension and the space feature dimension; The second sleep state data output by the spatio-temporal environment sleep quality association model after the small neural network adaptively adjusts the importance weight of the time feature dimension and the space feature dimension.

7. The image recognition and environment perception based sleep management optimization system of claim 1, wherein, The process of obtaining the continuously optimized sleep management strategy is as follows: Based on the improved reinforcement learning strategy, the sleep state and intervention action of multiple time steps are collected, and the update process of the strategy is trained through a reward function, and the importance sampling technique is used to reduce the sampling error to obtain the target function of the strategy update during the training process; The optimal sleep management strategy is generated by iteratively optimizing the strategy network parameters through the target function.

8. The method of sleep management optimization based on image recognition and environmental perception according to any one of claims 1-7, characterized in that, The method steps include: S1: Collecting image data, obtaining first sleep state data from the image data through an optimized bionic neural network; S2: Collecting sleep environment data, obtaining time dimension features and space dimension features from the sleep environment data through a spatio-temporal environment sleep quality association model, and obtaining second sleep state data through a small neural network adaptively adjusting the weight; S3: Generating a continuously optimized sleep management strategy based on the first sleep state data and the second sleep state data through an improved reinforcement learning strategy.

Images