A sleeping posture recognition system

Abstract

The present invention provides a sleeping posture recognition system, including a pressure pad, a signal acquisition system, and a host computer. A flexible piezoresistive sensing unit is provided on the pressure pad. The flexible piezoresistive sensing unit is connected to the signal acquisition system. The signal acquisition system is connected to the host computer through multiple serial ports and sends the collected pressure signals to the host computer. A sleeping posture recognition model is integrated in the host computer, and the current sleeping posture is presented in real time by using the sleeping posture recognition model. The pressure pad of the present invention adopts a three-layer structure design and has the characteristics of wide sensing range, high sensitivity, fast response speed, good durability, etc. The present invention proposes a voltage source cross-interchange strategy to timely release all parasitic charges, which can well solve the cross-coupling phenomenon caused by high-density large-scale sensors, ensure high measurement accuracy between rows and columns and good stability under high-density layout, and greatly improve the accuracy of the analog quantity measured by the sensor.

Description

Technical Field

[0001] The present invention belongs to the technical field of human health detection, and in particular relates to a sleeping posture recognition system. Background Art

[0002] Pressure ulcers refer to skin and subcutaneous tissue damage caused by long-term pressure on the skin. If not treated in time or the potential health condition is not conducive to its healing, it can also endanger life, especially for the disabled elderly who are bedridden for a long time. The most common sleep monitoring systems in the commercial field are smart wearable devices, which have the characteristics of being convenient to wear and can be monitored anytime and anywhere. However, such wearable devices cannot fully and accurately reflect the whole-body sleeping posture, especially actions such as turning over. In addition, as a typical monitoring device with a sense of restraint, patients may be affected by the foreign body sensation generated by wearing and affect normal sleep. The vision-based sleeping posture recognition method can achieve a relatively high accuracy of sleeping posture recognition and provide data support for preventing pressure ulcers. However, there are some limitations, such as being easily blocked and sensitive to light. In addition, due to privacy issues, vision-based solutions are often unacceptable in the field of patient behavior monitoring. Limited by the sensing density, the accuracy of sleeping posture recognition cannot be guaranteed. For example, only general supine, side-lying and other postures can be detected, and actions such as turning over cannot be accurately recognized. The current pressure-based sleeping posture recognition devices or systems in the commercial and academic fields at home and abroad have the following deficiencies: (1) The limited sensing density or the number of sensing points leads to low performance in sleep posture recognition. The sensing density of the most advanced systems is generally lower than 1 point / cm 2 . The low sensing density will hinder the large-scale integration of sensing points and further result in the lack of fine acquisition of body pressure data; (2) The severe cross-coupling crosstalk phenomenon caused by high-density and large-scale sensor arrays. The complex cross-coupling crosstalk problem between rows and columns caused by high-density sensor arrays needs to be taken into account. This phenomenon will cause signal interference between different axes, resulting in the inability to accurately obtain the measured value of a single axis and the distortion of the sensor measurement result; when the sensor is affected by external interference, the cross-coupling will exacerbate the fluctuation of the system, making it difficult for the system to maintain a stable state. As a result, there is no longer a mapping relationship between the pressure at the sensor point and the obtained analog quantity. Summary of the Invention

[0003] In view of this, the present invention aims to overcome the above deficiencies in the prior art and proposes a sleeping posture recognition system.

[0004] To achieve the above object, the technical solution of the present invention is realized as follows:

[0005] A sleeping posture recognition system, including a pressure pad, a signal acquisition system, and a host computer. The pressure pad is provided with a flexible piezoresistive sensing unit, the flexible piezoresistive sensing unit is connected to the signal acquisition system, and the signal acquisition system is connected to the host computer through multiple serial ports, sending the collected pressure signals to the host computer. The host computer integrates a sleeping posture recognition model and uses the sleeping posture recognition model to present the current sleeping posture in real time.

[0006] Further, the pressure pad includes 6 small pressure pads of 200 cm² each. Each small pressure pad includes a top electrode layer, a bottom electrode layer, and an intermediate fiber matrix dielectric layer. The top electrode layer includes a top polyimide film, and copper electrodes are horizontally spaced below the top polyimide film. The bottom electrode layer includes a bottom polyimide film, and copper electrodes are vertically spaced above the bottom polyimide film. The intermediate fiber matrix dielectric layer includes a conductive fabric. The top electrode layer and the bottom electrode layer are cross-assembled to clamp the conductive fabric to form a small pressure pad. The intersection positions of the top electrode layer and the bottom electrode layer constitute 1 flexible piezoresistive sensing unit.

[0007] Further, the number of flexible piezoresistive sensing unit points on each small pressure pad is 3600, totaling 21600 points, constructing a pressure pad with a high-density large-scale sensing unit layout.

[0008] Further, the signal acquisition system includes 6 circuit boards to realize the acquisition of pressure signals from different pressure pads. Each circuit board is provided with a signal measurement circuit. The signal measurement circuit includes an MCU processor, multiple decoders, an analog switch, a voltage follower circuit, and a median filter. The MCU processor controls the analog switch through the multiple decoders to connect different rows and columns of the small pressure pad to determine the position of the sensing unit points to be collected. The generated signal passes through a voltage follower circuit and is converted into a digital signal through an ADC. The acquired original sensing unit data is uploaded to the host computer through USB after median filtering.

[0009] Further, the sleeping posture recognition model includes: an input module, the input module is connected to convolutional kernels with sizes of 1×1, 3×3, and 5×5 and a max pooling layer; the outputs of the 3 convolutional kernels are connected to a fusion module, the output of the max pooling layer is connected to the input of the fusion module through a 3×3 convolutional kernel, the output of the fusion module is connected to a 5×5 convolutional kernel, the output of this convolutional kernel is connected to a max pooling layer, the output of the max pooling layer is then connected to a 3×3 convolutional kernel, the output of this convolutional kernel is then connected to a max pooling layer, the output of the max pooling layer is connected to a global average pooling layer, and the global average pooling layer is then connected to a batch normalization layer; the batch normalization layer is then connected to a global average pooling layer and finally connected to an output layer.

[0010] Further, the implementation process of the sleep posture recognition model is as follows:

[0011] Use the input module to input an image, input the image data into three convolution kernels of different sizes, and extract sleep posture information of different scales;

[0012] Input the image data into the max pooling layer, and use the max pooling layer to extract features with spatial invariance at the same time;

[0013] Use the fusion module to calculate the outputs of three convolution kernels of different sizes and the max pooling layer in the depth dimension, and extract temporal features;

[0014] Use a global average pooling to map the multi-dimensional features to a one-dimensional vector;

[0015] Use the output layer to obtain the distribution of different sleep postures in terms of probability.

[0016] Further, a voltage source cross-interchange circuit is provided between each small pressure pad and the circuit board. The voltage source cross-interchange circuit includes a flexible flat cable. The copper electrodes of the top electrode layer and the bottom electrode layer of the small pressure pad are respectively connected to the circuit board through the flexible flat cable and form an electrical connection with a multi-channel analog switch.

[0017] Compared with the prior art, the sleep posture recognition system of the present invention has the following advantages:

[0018] (1) The present invention realizes the research and development of a high-spatial-density flexible piezoresistive sensing unit and the layout strategy of a large-scale sensing unit, conducts sensitivity analysis of the overall sleep posture recognition accuracy, and determines the layout density of the sensing unit on the basis of ensuring an acceptable sleep posture recognition accuracy.

[0019] (2) The present invention proposes a solution strategy for cross-coupling crosstalk caused by a high-density and large-scale sensing unit array. The complex cross-coupling crosstalk between rows and columns caused by a high-density sensing unit array cannot be solved from the sensor body structure, and this phenomenon causes the pressure at the sensing unit point to no longer have a mapping relationship with the obtained analog quantity. The present invention ensures the time-division acquisition of signals of different sensor units by designing a voltage source cross-interchange circuit.

[0020] (3) Realize the construction of a high-precision unconstrained sleep posture recognition system based on body pressure analysis data and deep learning. The obtained body pressure information is uploaded to the host system in real time based on a multi-port and real-time communication method. The host system developed using the Python language includes a real-time display platform for sleep posture pressure images, a deep learning engine, and a remote medical system. The pattern recognition algorithm based on deep learning technology will accurately recognize the current sleep posture of the patient, and the rehabilitation physician can timely guide and adjust the patient's in-bed state through the remote medical system, creating conditions for home rehabilitation. Description of the Drawings

[0021] The accompanying drawings, which form a part of the present invention, are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 Schematic diagram of the material used for the small pressure pad of the present invention;

[0023] Figure 2 Schematic diagram of the flexible piezoresistive sensing unit of the present invention;

[0024] Figure 3 Overall schematic diagram of the pressure pad for collecting sleeping posture information of the present invention;

[0025] Figure 4 Schematic diagram of the signal measurement circuit structure of the present invention;

[0026] Figure 5 Schematic diagram of the pressure pad signal acquisition hardware system of the present invention;

[0027] Figure 6 Schematic diagram of the pressure information real-time display system of the present invention;

[0028] Figure 7 Schematic diagram of the sleeping posture recognition model of the present invention;

[0029] Figure 8 Overall schematic diagram of the voltage source cross-interchange strategy of the present invention;

[0030] Figure 9 Schematic diagram of the supine posture pressure recognition of the present invention;

[0031] Figure 10 Schematic diagram of the prone posture pressure recognition of the present invention;

[0032] Figure 11 Schematic diagram of the left lateral lying posture pressure recognition of the present invention;

[0033] Figure 12 Schematic diagram of the right lateral lying posture pressure recognition of the present invention. Detailed implementation manners

[0034] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0035] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0036] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0037] The present invention will be described in detail below with reference to the drawings and in combination with embodiments.

[0038] The present invention provides a sleeping posture recognition system, including a pressure pad, a signal acquisition system, and a host computer. The pressure pad is provided with a flexible piezoresistive sensing unit, the flexible piezoresistive sensing unit is connected to the signal acquisition system, and the signal acquisition system is connected to the host computer through a plurality of serial ports, and sends the collected pressure signal to the host computer. A sleeping posture recognition model is integrated in the host computer, and the current sleeping posture is presented in real time by using the sleeping posture recognition model.

[0039] In the present invention, a complete pressure pad is composed of 6 small pressure pads of 400 cm², as Figure 1 shown. Each small pressure pad includes a top electrode layer, a bottom electrode layer, and an intermediate fiber matrix dielectric layer. The top electrode layer includes a top polyimide film, and copper electrodes are horizontally spaced below the top polyimide film. The bottom electrode layer includes a bottom polyimide film, and copper electrodes are vertically spaced above the bottom polyimide film. The intermediate fiber matrix dielectric layer includes a conductive fabric. The top electrode layer and the bottom electrode layer are cross-assembled to clamp the conductive fabric to form a small pressure pad, and the cross-point positions of the top electrode layer and the bottom electrode layer constitute 1 flexible piezoresistive sensing unit. Among them, Figure 1(a) shows the structure of each small pressure pad. Figure 1 (b) describes the morphology of the sensing unit in the non-compressed and compressed states. Specifically, when the fabric is compressed, the distance between the upper and lower electrode layers becomes smaller, further reducing the resistance between the upper and lower electrode layers.

[0040] Each small pressure pad of the present invention has up to 3,600 sensing unit points. The total number of sensing unit points on the pressure pad is up to 21,600, with a density of 9 points per 1 cm², achieving a sleep posture recognition accuracy rate of 97.29%. Figure 2 The schematic diagram of the sensing unit is shown, where Figure 2 (a) are the scanning electron microscope images of the non-woven fabric impregnated with MXene dispersion liquid at low magnification and high magnification. Figure 2 (b) is the sensitivity curve of the sensing unit to external mechanical pressure. The red line represents the sensitivity curve of the sensing unit to external mechanical pressure, and the blue line represents the sensitivity of the sensor at different stages. Among them, S1, S2, and S3 represent the sensitivity of the sensor at different stages. Figure 2 (c) is the response and recovery time of the sensing unit through a 28 kPa loading / unloading test. The blue line is the response curve of the sensing unit through a 28 kPa loading / unloading test, and the red line represents the response and recovery time of the sensing unit. Figure 2 (d) is the corresponding behavior of the sensing unit at different compression ratios. Figure 2 (e) is the cyclic behavior of the sensing unit through a test of applying more than 10,000 consecutive dynamic loading / unloading cycles at 28 kPa. The left red line represents the partial enlarged view of the sensing unit during the 1700 - 1708 cycles of the cyclic test, and the right red line represents the partial enlarged view of the sensing unit during the 8300 - 8308 cycles of the cyclic test. The prepared pressure pad for realizing sleep posture recognition is as Figure 3 shown. Figure 3 (a) is the overall schematic diagram and partial schematic diagram of the pressure pad; Figure 3 (b) is the signal response image of the pressure pad actually obtained when the hand is actually pressed on the pressure pad.

[0041] In the present invention, the signal acquisition system includes 6 circuit boards to achieve the acquisition of pressure signals from different pressure pads. Each circuit board is provided with a signal measurement circuit. The signal measurement circuit includes an MCU processor, a multi-decoder, an analog switch, a voltage follower circuit, and a median filter. The MCU processor controls the analog switch through the multi-decoder to connect different rows and columns of the small pressure pad to determine the position of the sensing unit points to be collected. The generated signal passes through a voltage follower circuit, and an ADC converts the analog signal into a digital signal. The obtained original sensing unit data is filtered by a median filter and then uploaded to the host computer through USB. The structure of the signal measurement circuit is as Figure 4As shown, after the system is initialized, the MCU (STM32F103VET6) controls the analog switch through multiple decoders to connect the copper electrodes of different rows and columns of the top electrode layer and the bottom electrode layer to determine the position of the sensing unit points to be collected. The generated signal passes through a voltage follower circuit, and an ADC converts the analog signal into a digital signal. The acquired original sensor data is median-filtered and then uploaded to the software system developed in the Pycharm editor on the host computer through USB.

[0042] The pressure pad signal acquisition hardware system of the present invention is as Figure 5 shown. The hardware system is modularly designed into 6 independent circuit boards to achieve the acquisition of pressure signals of different pressure pads and upload them to the host computer through serial communication. Figure 6 It shows the host computer software developed in the Python language, which presents the current sleeping posture in real time by receiving the pressure pad signal.

[0043] The sleeping posture recognition model is integrated in the host computer software of the present invention, and the implementation process is as Figure 7 shown. The input image data is first input into convolutional kernels of different sizes at the same time. The convolutional kernel sizes are respectively set to 1×1, 3×3, and 5×5 to fully extract the sleeping posture information of different scales. The 3×3 uses a max pooling layer to simultaneously extract features with spatial invariance. The outputs of these layers are calculated in the depth dimension, which is used to extract temporal features. The next layers are two convolutional layers, each with 64 and 128 filters respectively. And there is a max pooling layer behind each convolutional layer. Subsequently, a global average pooling (GAP) is used to map the multi-dimensional feature map to a one-dimensional vector. This step replaces the fully connected layer, thus greatly reducing the number of weights. There is a global average pooling (GAP) followed by a batch normalization layer (BN). The last output layer (a dense layer with a Softmax function classifier) obtains the probability distribution of different sleeping postures.

[0044] In a high-density and large-scale sensor array, due to the small spacing between sensor elements, the electromagnetic fields of each element are likely to overlap with each other, and the electromagnetic fields of each sensitive unit interact with each other, resulting in unnecessary coupling of signals during transmission. To solve this problem, the present invention proposes a voltage source cross-interchange strategy. A voltage source cross-interchange circuit is provided between each small pressure pad and the circuit board. The voltage source cross-interchange circuit includes a flexible cable. The copper electrodes of the top electrode layer and the bottom electrode layer of the small pressure pad are respectively connected to the circuit board through the flexible cable and form an electrical connection with the multi-channel analog switch. As Figure 8As shown, arbitrarily select a sensing unit point P. When measuring the resistance at the measurement point P, the single-pole double-throw switch S1 is closed. During the first measurement, S0 is connected to GND and S2 is connected to VCC. Taking the voltage output at this time as Vout = Vout1, then ;

[0045] where VCC is the power supply voltage (3.3V is adopted), R ref is the reference resistance, R point is the resistance at point P, and R othor is the equivalent resistance generated by cross-coupling. During the second measurement, S0 is connected to VCC and S2 is connected to GND. Taking the voltage output at this time as V out = V out2 , then ;

[0046] By cross-exchanging the voltage source and the results of the two measurements before and after, an accurate resistance of point P can be obtained, which is expressed as ;

[0047] Through measurement, it is found that the resistance R point at point P is only related to V out1 and V out2 , avoiding the influence of R other generated by other points. Therefore, the value of the obtained sensing unit directly reflects the resistance of point P.

[0048] To test the performance of the proposed system in recognizing sleeping postures, a set of experiments has been carried out. A total of 16 subjects (10 males and 6 females aged between 21 and 53 years old) participated in this experiment. Each subject signed an informed consent form before the experiment. Each patient was required to perform four sleeping postures: supine, prone, left lateral, and right lateral. Figures 9 - 12 Show the image data collected in four sleeping postures.

[0049] During the experiment, this study provided specific posture instructions to the subjects. To ensure that the performance of the system can be applied to different body types and sleeping postures, this study collected a large amount of sleeping posture data and did not limit the posture differences of the subjects. For example, when performing the supine state, the limb angles of the subjects are different, as shown in Figure 9 and Figure 10 , the angles of the waist and arms are different. In addition to different limb angles, variants of the same posture were also tried. For example, when performing the prone sleeping posture, Figure 11 shows symmetric support of the elbows on both sides, while Figure 12It shows that the position of the left elbow is above, while the position of the right elbow is below. 30 groups of data were collected for each sleeping position of each subject pair in this study, so the database contains a total of 1920 groups of sleeping position data.

[0050] Where the present invention is not described shall apply to the prior art.

[0051] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A sleeping posture recognition system, characterized in that: It includes a pressure pad, a signal acquisition system, and a host computer. A flexible piezoresistive sensing unit is provided on the pressure pad. The flexible piezoresistive sensing unit is connected to the signal acquisition system. The signal acquisition system is connected to the host computer through multiple serial ports and sends the collected pressure signals to the host computer. A sleeping posture recognition model is integrated in the host computer, and the current sleeping posture is presented in real time by using the sleeping posture recognition model. The pressure pad includes 6 small pressure pads of 200 cm², with a sensing density as high as 9 sensor points / cm². Each small pressure pad includes a top electrode layer, a bottom electrode layer, and an intermediate fiber matrix dielectric layer. The top electrode layer includes a top polyimide film, and copper electrodes are horizontally arranged at intervals below the top polyimide film. The bottom electrode layer includes a bottom polyimide film, and copper electrodes are vertically arranged at intervals above the bottom polyimide film. The intermediate fiber matrix dielectric layer includes a conductive fabric. When the fabric is compressed, the distance between the upper and lower electrode layers becomes smaller, further reducing the resistance between the upper and lower electrode layers. The intersection positions of the top electrode layer and the bottom electrode layer form 1 flexible piezoresistive sensing unit. The number of flexible piezoresistive sensing unit points on each small pressure pad is 3600, with a density of 9 points / cm², totaling 21600 points, constructing a pressure pad with a high-density large-scale sensing unit arrangement. The signal acquisition system includes 6 circuit boards to realize the acquisition of pressure signals from different pressure pads. Each circuit board is provided with a signal measurement circuit. The signal measurement circuit includes an MCU processor, multiple decoders, an analog switch, a voltage follower circuit, and a low-pass filter. The MCU processor controls the analog switch through multiple decoders to connect different rows and columns of the small pressure pad to determine the position of the sensing unit points to be collected. The generated signal passes through a voltage follower circuit, and an ADC converts the analog signal into a digital signal. After the original sensing unit data obtained is low-pass filtered, it is uploaded to the host computer through USB. A voltage source cross-interchange circuit is provided between each small pressure pad and the circuit board. The voltage source cross-interchange circuit includes a flexible flat cable. The copper electrodes of the top electrode layer and the bottom electrode layer of the small pressure pad are respectively connected to the circuit board through the flexible flat cable and form an electrical connection with the multiplex analog switch. The two flexible flat cables are connected through a reference resistor and a single-pole double-throw switch S2 to form a voltage source cross-interchange circuit. The voltage source cross-interchange circuit controls the power supply polarity of the target sensing unit through the single-pole double-throw switch. The host computer eliminates the cross-coupling of the high-density sensor array through the following decoupling algorithm formula: ; ; ; where VCC is the power supply voltage, R ref is the reference resistor, R point is the resistance of the measurement point P, R othor is the equivalent resistance generated by cross - coupling. Vout1 and Vout2 are the results of two measurements before and after the cross - swapping of the voltage source respectively. Through the voltage source cross - swapping strategy, based on the basic principle of keeping the current through the resistor under test unchanged, combined with the application of the analog switch chip for the circuit design of timely release of parasitic charges in the circuit and the decoupling algorithm, zero - crosstalk acquisition of the original sensing signal is ensured.

2. The sleep posture recognition system according to claim 1, wherein: The sleep posture recognition model includes: an input module, which is connected to convolutional kernels with sizes of 1×1, 3×3, and 5×5, and a max pooling layer; the outputs of the 3 convolutional kernels are connected to a fusion module, and the output of the max pooling layer is connected to the input of the fusion module through a 3×3 convolutional kernel. The output of the fusion module is connected to a 5×5 convolutional kernel, the output of this convolutional kernel is connected to a max pooling layer, the output of the max pooling layer is then connected to a 3×3 convolutional kernel, the output of this convolutional kernel is then connected to a max pooling layer, the output of the max pooling layer is connected to a global average pooling layer, and the global average pooling layer is then connected to a batch normalization layer; the batch normalization layer is then connected to a global average pooling layer, and finally connected to an output layer.

3. The sleep posture recognition system according to claim 1, wherein: The implementation process of the sleep posture recognition model is as follows: Use the input module to input an image, and input the image data into 3 convolutional kernels with different sizes to extract sleep posture information at different scales; Input the image data into the max pooling layer, and use the pooling layer to extract features with spatial invariance at the same time; Use the fusion module to calculate the outputs of the 3 convolutional kernels with different sizes and the max pooling layer in the depth dimension to extract temporal features; Use a global average pooling to map the multi-dimensional features to a one-dimensional vector; Use the output layer to obtain the probability distribution of different sleep postures.

Images