A method for human sleep posture recognition for a pressure sensing array

Abstract

The application provides a human sleep posture recognition method for a pressure sensing array, comprising the following steps: S1, acquiring a real-time pressure data of a user lying on the pressure sensing array every preset time interval; S2, carrying out denoising, extreme value removal and fusion average preprocessing on each real-time pressure data to obtain corresponding preprocessed real-time pressure data; S3, carrying out picture processing on each preprocessed real-time pressure data to obtain corresponding real-time grayscale images; and S4, inputting each real-time grayscale image into a sleep posture recognition model of a residual convolutional neural network which is pre-trained to obtain a sleep posture recognition result representing a sleep posture category of the user. The application can accurately recognize the sleep posture of the user when lying in bed, has a high recognition speed, and can be well used for bed sleep posture detection.

Description

Technical Field

[0001] This invention relates to the technical field of signal processing and pattern recognition, and more specifically, to a method for recognizing human sleeping postures using a pressure sensing array. Background Technology

[0002] In recent years, artificial intelligence technology has developed rapidly. Against this backdrop, health monitoring has also entered people's lives. With the rapid economic development and the continuous improvement of living standards, people are no longer satisfied with basic needs when purchasing home products. Instead, they are pursuing more intelligent technological products. These products often have new features such as safety and interactivity. With the continuous integration of sensing technology and Internet technology, more and more researchers are beginning to use various sensors to monitor human health.

[0003] Sleep posture plays a crucial role in assessing sleep quality and preventing pressure ulcers. Research on sleep posture can greatly help patients adjust their sleep posture to prevent pressure ulcers. It has received much attention from researchers in recent years. Currently, the developed identification methods can be mainly divided into three categories: (1) visual feature extraction based on images and videos collected by cameras; (2) vital sign analysis based on wearable devices; and (3) pressure information extraction and identification based on pressure sensor arrays.

[0004] However, the aforementioned image and video-based recognition methods require basic camera devices, and the recognition results are affected by the brightness of the field of view, and there are privacy issues. Recognition methods based on wearable devices are prone to causing discomfort to users while sleeping, and the accuracy of vital sign assessment in analyzing sleep posture is low. Recognition methods based on pressure sensor arrays usually analyze sleep quality by collecting pressure changes caused by the user's breathing, body movement, and heart rhythm signals. However, analysis based solely on changes in pressure values ​​cannot accurately identify the user's sleep posture while lying in bed. Summary of the Invention

[0005] The problem to be solved by this invention is to provide a method for recognizing human sleeping postures using a pressure sensor array, which is unaffected by the brightness of the field of vision, avoids privacy leaks, ensures user comfort during sleep, and can accurately identify the user's sleeping posture when lying in bed.

[0006] To address the aforementioned problems, this invention provides a method for recognizing human sleeping postures using a pressure sensor array. The method involves pre-laying a pressure sensor array on a bed board and then placing a mattress on top of the array for a user to lie on and sleep. The method includes the following steps:

[0007] Step S1: At preset time intervals, acquire a segment of real-time pressure data of the user lying on the pressure sensor array;

[0008] Step S2: For each segment of the real-time pressure data, perform noise reduction, extremum removal, and fusion averaging preprocessing on the real-time pressure data to obtain the corresponding preprocessed real-time pressure data.

[0009] Step S3: For each of the preprocessed real-time pressure data, perform image processing on the preprocessed real-time pressure data to obtain a corresponding real-time grayscale image.

[0010] Step S4: For each of the real-time grayscale images, the real-time grayscale image is input into the pre-trained residual convolutional neural network sleeping posture recognition model to obtain the sleeping posture recognition result that represents the user's sleeping posture category.

[0011] Preferably, the pressure sensor array has 168 pressure sensor points. In step S1, the sensor data of the user lying on the pressure sensor array is obtained through each of the pressure sensor points, and the sensor data is integrated into the real-time pressure data.

[0012] Preferably, the pressure sensing array is equipped with a Bluetooth communication module. The input end of the Bluetooth communication module is connected to each of the pressure sensing points, and the output end of the Bluetooth communication module is connected to an external host computer. In step S1, the host computer obtains the sensing data output by each of the pressure sensing points through the Bluetooth communication module and integrates it into the real-time pressure data.

[0013] Preferably, in step S4, the sleep posture category represented by the sleep posture recognition result includes supine, left lateral, right lateral, prone, left curled-up, and right curled-up.

[0014] Preferably, a model training process is included before performing step S1, which includes the following steps:

[0015] Step A1: Acquire historical pressure data of different experimental subjects lying on the pressure sensor array in different sleeping positions;

[0016] Step A2: For each of the historical pressure data, perform noise reduction, extremum removal and fusion averaging preprocessing on the historical pressure data to obtain the corresponding preprocessed historical pressure data;

[0017] Step A3: For each of the preprocessed historical pressure data, perform image processing on the preprocessed historical pressure data to obtain the corresponding historical grayscale image;

[0018] Step A4: Using each of the historical grayscale images as input and the sleep posture category corresponding to each of the labeled historical grayscale images as output, the sleep posture recognition model is trained.

[0019] Preferably, in step A1, for each sleep posture of each experimenter, the historical pressure data of the experimenter lying on the pressure sensor array in the sleep posture is continuously collected within one minute at a sampling frequency of 3Hz.

[0020] Preferably, step A2 includes:

[0021] Step A21: Delete the historical pressure data with duplicate pressure values ​​in each of the historical pressure data, and use the remaining historical pressure data as the first preprocessed data.

[0022] Step A22: For each piece of the first preprocessed data, determine whether the pressure value of the first preprocessed data is greater than a preset threshold.

[0023] If so, delete the first preprocessed data and proceed to step A23;

[0024] If not, retain the first preprocessed data and proceed to step A23;

[0025] Step A23: The retained first preprocessed data are used as second preprocessed data. The second preprocessed data with pressure value fluctuations are filtered to obtain the corresponding third preprocessed data. The third preprocessed data are used as the preprocessed historical pressure data.

[0026] Preferably, the residual convolutional neural network used in step A4 contains multiple residual blocks arranged in layers between the input layer and the output layer, so as to extract the corresponding image features from the historical grayscale image layer by layer.

[0027] The present invention has the following beneficial effects:

[0028] (1) This invention abandons the traditional analysis and recognition method based on camera-acquired images, and collects and analyzes pressure data through a pressure sensor array. It is applicable at any time, is not affected by the brightness of the field of view, and does not leak privacy.

[0029] (2) In this invention, the user cannot wear additional devices, and a mattress is laid on the pressure sensor array, which will not cause discomfort to the user and can ensure the user's comfort during sleep.

[0030] (3) In this invention, noise reduction, extremum removal and fusion averaging preprocessing methods are used to remove data interference in advance, improve the effectiveness of real-time pressure data, and introduce a pre-trained sleep posture recognition model based on residual convolutional neural network to perform image feature recognition on real-time grayscale images, which can accurately identify the user's sleep posture when lying in bed. Attached Figure Description

[0031] Figure 1 This is a flowchart of the steps of the present invention;

[0032] Figure 2 This is a flowchart of the real-time pressure data processing of the present invention;

[0033] Figure 3 This is a flowchart illustrating the steps of the model training process of the present invention;

[0034] Figure 4 This is a flowchart illustrating step A2 of the present invention.

[0035] Figure 5 This is a schematic diagram of the network structure of the residual convolutional neural network of the present invention;

[0036] Figure 6 This is a schematic diagram of the residual block of the present invention;

[0037] Figure 7 These are grayscale images corresponding to the various sleep posture categories of this invention. Detailed Implementation

[0038] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0039] In a preferred embodiment of the present invention, based on the above-mentioned problems existing in the prior art, a method for human sleeping posture recognition using a pressure sensor array is provided. A pressure sensor array is pre-laid on a bed board, and a mattress is laid on the pressure sensor array for the user to lie on and sleep. The human sleeping posture recognition method is as follows: Figure 1 , 2 As shown, it includes the following steps:

[0040] Step S1: At preset time intervals, acquire a segment of real-time pressure data of the user lying on the pressure sensor array;

[0041] Step S2: For each segment of real-time pressure data, perform noise reduction, extremum removal, and fusion averaging preprocessing on the real-time pressure data to obtain the corresponding preprocessed real-time pressure data.

[0042] Step S3: For each preprocessed real-time pressure data, perform image processing on the preprocessed real-time pressure data to obtain a corresponding real-time grayscale image.

[0043] Step S4: For each real-time grayscale image, input the real-time grayscale image into the pre-trained residual convolutional neural network sleeping posture recognition model to obtain the sleeping posture recognition result that represents the user's sleeping posture category.

[0044] Specifically, in this embodiment, during the real-time recognition stage, for the new real-time pressure data input to the pressure sensor array, each frame of new data is first processed through a data preprocessing script and a grayscale graphics script to obtain a real-time grayscale image, and then the real-time grayscale image is input into the sleeping posture recognition model for recognition.

[0045] Preferably, a human-computer interaction interface can be implemented by writing QT scripts. Experimenters sleep on the pressure sensor array in different sleeping positions, and the PC will display the sleep pressure map and the current sleeping position category at one-minute intervals.

[0046] In a preferred embodiment of the present invention, the pressure sensor array is provided with 168 pressure sensor points. In step S1, the sensor data of the user lying on the pressure sensor array is obtained through each pressure sensor point, and the sensor data is integrated into real-time pressure data.

[0047] In a preferred embodiment of the present invention, a Bluetooth communication module is provided on the pressure sensing array. The input end of the Bluetooth communication module is connected to each pressure sensing point, and the output end of the Bluetooth communication module is connected to an external host computer. In step S1, the host computer obtains the sensing data output by each pressure sensing point from the Bluetooth communication module and integrates it into real-time pressure data.

[0048] In a preferred embodiment of the present invention, in step S4, the sleep posture categories represented by the sleep posture recognition results include supine, left lateral, right lateral, prone, left curled-up, and right curled-up.

[0049] In a preferred embodiment of the present invention, a model training process is further included before performing step S1, the model training process as follows: Figure 3 As shown, it includes the following steps:

[0050] Step A1: Acquire historical pressure data of different experimental subjects lying on the pressure sensor array in different sleeping positions;

[0051] Step A2: For each historical pressure data point, perform noise reduction, extremum removal, and fusion averaging preprocessing to obtain the corresponding preprocessed historical pressure data.

[0052] Step A3: For each preprocessed historical stress data, perform image processing on the preprocessed historical stress data to obtain the corresponding historical grayscale image;

[0053] Step A4: Using each historical grayscale image as input and the sleep posture category corresponding to each labeled historical grayscale image as output, a sleep posture recognition model is trained.

[0054] Specifically, in this embodiment, all historical grayscale images can be divided into training set images and test set images. During the training phase, the training set images are input into the residual convolutional neural network for training, and the sleeping posture recognition model is tested using the test set images. The residual convolutional neural network is iterated continuously to optimize and adjust parameters, improve the recognition rate, and finally the sleeping posture recognition model with the highest recognition rate is saved for the real-time recognition phase.

[0055] Preferably, the ratio of the total number of historical grayscale images in the training set to the total number of historical grayscale images in the test set is 4:1.

[0056] Preferably, in specific operations, a script can be used to annotate image features, with the annotation information being the different sleeping postures of different experimenters.

[0057] In a preferred embodiment of the present invention, in step A1, for each sleep posture of each experimenter, historical pressure data of the experimenter lying on the pressure sensor array in a sleep posture is continuously collected within one minute at a sampling frequency of three hertz.

[0058] In a preferred embodiment of the present invention, such as Figure 4 As shown, step A2 includes:

[0059] Step A21: Delete the historical pressure data with duplicate pressure values ​​in each historical pressure data, and use the remaining historical pressure data as the first preprocessing data.

[0060] Step A22: For each piece of first preprocessed data, determine whether the pressure value of the first preprocessed data is greater than a preset threshold.

[0061] If so, delete the first preprocessed data and proceed to step A23;

[0062] If not, retain the first preprocessed data and proceed to step A23;

[0063] Step A23: The retained first preprocessed data are used as second preprocessed data. The second preprocessed data with pressure value fluctuations are filtered to obtain the corresponding third preprocessed data. The third preprocessed data are used as preprocessed historical pressure data.

[0064] Specifically, in this embodiment, considering that there will be many identical data frames due to the fast sampling frequency during the data acquisition process, the main task is to eliminate duplicate and abnormal frames in the data.

[0065] Preferably, considering that the acquired data may contain pressure values ​​that deviate from normal values, a threshold α is set, and the comparison formula between the pressure value and the threshold is as follows:

[0066]

[0067] in,

[0068] a i Indicates the pressure value;

[0069] α represents the preset threshold.

[0070] In a preferred embodiment of the present invention, the residual convolutional neural network used in step A4 contains multiple residual blocks arranged in layers between the input layer and the output layer, so as to extract the corresponding image features from the historical grayscale image layer by layer.

[0071] Specifically, in this embodiment, the sleeping posture recognition model is constructed based on a residual convolutional neural network. It extracts image features through a multi-layered arrangement of residual blocks, and finally uses a softmax layer to perform sleeping posture classification and recognition to obtain accurate sleeping posture recognition results.

[0072] Preferably, residual blocks can effectively prevent gradient vanishing and gradient explosion caused by the increase of network layers, while also accelerating the convergence speed.

[0073] Preferably, since convolutional neural networks can automatically extract image features, unlike machine learning methods which require manual feature extraction, this method uses a two-dimensional convolutional neural network. The convolutional layer consists of several convolutional units. By sliding a single kernel on the image, convolution operations are performed. Different kernels are selected to perform different operations on the image. These operations typically include edge detection, blurring, and sharpening. This transforms the input historical grayscale image or real-time grayscale image into a feature map, i.e., an image with image features. The value of a pixel in the feature map is the dot product of the kernel weight and the pixels in a portion of the input image. The size of the input image is the same as the size of the filter.

[0074] Preferably, the residual block is divided into two parts: direct mapping and residual mapping, as shown in the following formula:

[0075] x i+1 =x i +F(x i W i )

[0076] in,

[0077] x i Indicates direct mapping;

[0078] F(xi W i ) represents residual mapping.

[0079] Preferably, the sleeping posture recognition model in this method uses multiple residual blocks to extract features, and the network structure of the residual convolutional neural network is as follows: Figure 5 As shown, a schematic diagram of the residual block is as follows: Figure 6 As shown, after flattening the extracted image features, they enter a fully connected layer. The data after the fully connected layer is then classified using softmax. The softmax function formula is as follows:

[0080]

[0081] in,

[0082] x represents the input data of the softmax layer, i.e., the output of the fully connected layer;

[0083] K represents the number of sleep posture categories.

[0084] Preferably, the softmax result is the probability of each sleep posture category, and the sum of their probabilities is 1.

[0085] Preferably, real-time grayscale images or historical grayscale images, such as Figure 7 As shown, these represent the corresponding sleep posture categories, including supine, left lateral, right lateral, prone, left curled-up, and right curled-up.

[0086] While the disclosure is as stated above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of this disclosure, and all such changes and modifications will fall within the protection scope of this invention.

Claims

1. A method for human sleep position recognition for a pressure sensing array, characterized in that, A pressure sensor array is pre-laid on a bed board, and a mattress is laid on the pressure sensor array for a user to lie on and sleep. The human sleeping posture recognition method includes the following steps: Step S1: At preset time intervals, acquire real-time pressure data of the user lying on the pressure sensor array; Step S2: For each of the real-time pressure data, perform noise reduction, extremum removal and fusion averaging preprocessing on the real-time pressure data to obtain the corresponding preprocessed real-time pressure data; Step S3: For each of the preprocessed real-time pressure data, perform image processing on the preprocessed real-time pressure data to obtain a corresponding real-time grayscale image. Step S4: For each of the real-time grayscale images, input the real-time grayscale image into the pre-trained residual convolutional neural network sleeping posture recognition model to obtain the sleeping posture recognition result that represents the user's sleeping posture category. Before performing step S1, a model training process is also included, which includes the following steps: Step A1: Acquire historical pressure data of different experimental subjects lying on the pressure sensor array in different sleeping positions; Step A2: For each of the historical pressure data, perform noise reduction, extremum removal and fusion averaging preprocessing on the historical pressure data to obtain the corresponding preprocessed historical pressure data; Step A3: For each of the preprocessed historical pressure data, perform image processing on the preprocessed historical pressure data to obtain the corresponding historical grayscale image; Step A4: Using each of the historical grayscale images as input and the sleep posture category corresponding to each of the labeled historical grayscale images as output, train the sleep posture recognition model. Step A2 includes: Step A21: Delete the historical pressure data with duplicate pressure values ​​in each of the historical pressure data, and use the remaining historical pressure data as the first preprocessed data. Step A22: For each piece of the first preprocessed data, determine whether the pressure value of the first preprocessed data is greater than a preset threshold. If so, delete the first preprocessed data and proceed to step A23; If not, retain the first preprocessed data and proceed to step A23; Step A23: The retained first preprocessed data are used as second preprocessed data. The second preprocessed data with pressure value fluctuations are filtered to obtain the corresponding third preprocessed data. The third preprocessed data are used as the preprocessed historical pressure data. The residual convolutional neural network used in step A4 contains multiple residual blocks arranged in layers between the input and output layers to extract corresponding image features from the historical grayscale image layer by layer.

2. The human sleep position recognition method according to claim 1, characterized in that, The pressure sensor array has 168 pressure sensor points. In step S1, the sensor data of the user lying on the pressure sensor array is obtained through each of the pressure sensor points, and the sensor data is integrated into the real-time pressure data.

3. The human sleep position recognition method according to claim 2, characterized in that, The pressure sensing array is equipped with a Bluetooth communication module. The input end of the Bluetooth communication module is connected to each of the pressure sensing points, and the output end of the Bluetooth communication module is connected to an external host computer. In step S1, the host computer obtains the sensing data output by each of the pressure sensing points through the Bluetooth communication module and integrates it into the real-time pressure data.

4. The human sleeping posture recognition method according to claim 1, characterized in that, In step S4, the sleep posture categories represented by the sleep posture recognition results include supine, left lateral, right lateral, prone, left curled-up, and right curled-up.

5. The human sleeping posture recognition method according to claim 1, characterized in that, In step A1, for each sleep posture of each experimenter, the historical pressure data of the experimenter lying on the pressure sensor array in the sleep posture is continuously collected within one minute at a sampling frequency of 3Hz.

Images