Pressure sensor data identification method, mattress control method and related device

Abstract

The embodiment of the present invention proposes a method for identifying pressure sensor data, a mattress control method and related devices, which relate to the field of smart home. By pre-training a sleeping position recognition network and a part recognition network and a key point recognition network under different sleeping positions, when identifying the pressure sensor data, the acquired human body pressure data is first input into the sleeping position recognition network for recognition to obtain the corresponding human body sleeping position, and then the human body pressure data is input into the part recognition network and the key point recognition network corresponding to the human body sleeping position for recognition, and the different human body parts and key points under the human body sleeping position are identified. Since different part recognition networks and key point recognition networks are used for part recognition and key point recognition for different sleeping positions, accurate recognition of human body parts and key points can be achieved in different sleeping position scenarios, thereby providing a strong guarantee for improving the intelligent service capabilities of the mattress and improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of smart home, and in particular to a pressure sensor data recognition method, a mattress control method and related devices. Background Art

[0002] With the development of smart homes, people are paying more and more attention to the quality of their daily lives, especially for smart mattresses in terms of preventing pressure sores, comfort, and intelligent adjustment.

[0003] However, in the prior art, the recognition of pressure sensor data for the mattress has a problem of low accuracy, resulting in low accuracy of intelligent operation of the mattress, which reduces the user experience. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a pressure sensor data recognition method, a mattress control method and related devices, which can accurately recognize the pressure sensor data, thereby providing a strong guarantee for improving the intelligent service capabilities of the mattress.

[0005] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0006] In a first aspect, the present invention provides a method for identifying pressure sensor data, the method comprising:

[0007] Obtaining human body pressure data collected by a pressure sensor provided on a smart mattress;

[0008] Inputting the human body pressure data into a pre-trained sleeping posture recognition network to obtain the human body sleeping posture;

[0009] The human body pressure data is respectively input into the part recognition network and the key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained through pre-training.

[0010] In an optional embodiment, the sleeping posture recognition network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a fully connected layer and a first classification layer;

[0011] The step of inputting the human body pressure data into a pre-trained sleeping posture recognition network to obtain the human body sleeping posture comprises:

[0012] Inputting the human body pressure data into a pre-trained sleeping posture recognition network, performing a first depthwise separable convolution operation on the input human body pressure data using the first convolution layer, and inputting the matrix data after the first depthwise separable convolution operation into the first pooling layer;

[0013] Using the first pooling layer to perform a first pooling operation on the matrix data after the first depthwise separable convolution operation, and inputting the matrix data after the first pooling operation into the second convolution layer;

[0014] Using the second convolution layer to perform a second depth-separable convolution operation on the matrix data after the first pooling operation, and inputting the matrix data after the second depth-separable convolution operation into the second pooling layer;

[0015] Using the second pooling layer to perform a second pooling operation on the matrix data after the second depthwise separable convolution operation, and inputting the matrix data after the second pooling operation into the fully connected layer;

[0016] Using the fully connected layer to perform a fully connected operation on the matrix data after the second pooling operation, and inputting the matrix data after the fully connected operation into the first classification layer;

[0017] The first classification layer is used to perform a classification operation on the matrix data after the full connection operation to obtain a first classification result, and the sleeping posture of the human body is determined according to the first classification result.

[0018] In an optional embodiment, the human sleeping positions include supine, prone and lateral positions, and the part recognition network and key point recognition network corresponding to the supine position have the same network structure and different network parameters; the part recognition network and key point recognition network corresponding to the prone position have the same network structure and different network parameters; the part recognition network and key point recognition network corresponding to the lateral position have the same network structure and different network parameters.

[0019] In an optional embodiment, the supine position part recognition network and the supine position key point recognition network corresponding to the supine position both include a first convolutional neural network, a third pooling layer, a second convolutional neural network, a fourth pooling layer, a first upsampling layer, a third convolutional layer, and a second classification layer;

[0020] The human body pressure data is respectively input into a part recognition network and a key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained by pre-training, including:

[0021] When the sleeping posture of the human body is a supine position, the human body pressure data is respectively input into the supine position part recognition network and the supine position key point recognition network, and a first convolution operation is performed on the input human body pressure data using the first convolution neural network of each of the supine position part recognition network and the supine position key point recognition network, and the matrix data after the first convolution operation is input into the corresponding third pooling layer;

[0022] Using the third pooling layer of each of the supine position part recognition network and the supine position key point recognition network, a third pooling operation is performed on the received matrix data after the first convolution operation, and the matrix data after the third pooling operation is input into the corresponding second convolutional neural network;

[0023] Using the second convolutional neural network of each of the supine position part recognition network and the supine position key point recognition network, a second convolution operation is performed on the received matrix data after the third pooling operation, and the matrix data after the second convolution operation is input into the corresponding fourth pooling layer;

[0024] Using the fourth pooling layer of each of the supine position part recognition network and the supine position key point recognition network, perform a fourth pooling operation on the received matrix data after the second convolution operation, and input the matrix data after the fourth pooling operation into the corresponding first upsampling layer;

[0025] Using the first upsampling layers of the supine position part recognition network and the supine position key point recognition network, respectively, a first upsampling operation is performed on the received matrix data after the fourth pooling operation, and the matrix data after the first upsampling operation is input into the corresponding third convolutional layer;

[0026] Using the third convolutional layers of the supine position part recognition network and the supine position key point recognition network, respectively, a third convolutional operation is performed on the received matrix data after the first upsampling operation, and the matrix data after the third convolutional operation is input into the corresponding second classification layer;

[0027] The second classification layer of the supine position part recognition network and the supine position key point recognition network is used to perform a classification operation on the received matrix data after the third convolution operation to obtain a second classification result, and the human body parts and key points in the supine position are obtained according to the second classification results output by the supine position part recognition network and the supine position key point recognition network.

[0028] In an optional embodiment, the prone position part recognition network and the prone position key point recognition network corresponding to the prone position both include a third convolutional neural network, a fifth pooling layer, a fourth convolutional neural network, a sixth pooling layer, a second upsampling layer, a fourth convolutional layer and a third classification layer;

[0029] The human body pressure data is respectively input into a part recognition network and a key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained by pre-training, including:

[0030] When the sleeping posture of the human body is a prone position, the human body pressure data is respectively input into the prone position part recognition network and the prone position key point recognition network, and a fourth convolution operation is performed on the input human body pressure data using the third convolutional neural network of each of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the fourth convolution operation is input into the corresponding fifth pooling layer;

[0031] Using the fifth pooling layer of each of the prone position part recognition network and the prone position key point recognition network, perform a fifth pooling operation on the received matrix data after the fourth convolution operation, and input the matrix data after the fifth pooling operation into the corresponding fourth convolutional neural network;

[0032] Using the fourth convolutional neural network of each of the prone position part recognition network and the prone position key point recognition network, perform a fifth convolution operation on the received matrix data after the fifth pooling operation, and input the matrix data after the fifth convolution operation into the corresponding sixth pooling layer;

[0033] Using the sixth pooling layer of each of the prone position part recognition network and the prone position key point recognition network, perform a sixth pooling operation on the received matrix data after the fifth convolution operation, and input the matrix data after the sixth pooling operation into the corresponding second upsampling layer;

[0034] Using the second upsampling layers of the prone position part recognition network and the prone position key point recognition network, respectively, a second upsampling operation is performed on the received matrix data after the sixth pooling operation, and the matrix data after the second upsampling operation is input into the corresponding fourth convolutional layer;

[0035] Using the fourth convolutional layers of the prone position part recognition network and the prone position key point recognition network, respectively, a sixth convolutional operation is performed on the received matrix data after the second upsampling operation, and the matrix data after the sixth convolutional operation is input into the corresponding third classification layer;

[0036] The third classification layer of the prone position part recognition network and the prone position key point recognition network is used to perform a classification operation on the received matrix data after the sixth convolution operation to obtain a third classification result, and the human body parts and key points in the prone position are obtained according to the third classification results output by the prone position part recognition network and the prone position key point recognition network.

[0037] In an optional embodiment, the lateral position part recognition network and the lateral position key point recognition network corresponding to the lateral position both include a fifth convolutional neural network, a seventh pooling layer, a sixth convolutional neural network, an eighth pooling layer, a third upsampling layer, a fifth convolutional layer and a fourth classification layer;

[0038] The human body pressure data is respectively input into a part recognition network and a key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained by pre-training, including:

[0039] When the sleeping posture of the human body is a lateral position, the human body pressure data is respectively input into the lateral position part recognition network and the lateral position key point recognition network, and a seventh convolution operation is performed on the input human body pressure data using the fifth convolutional neural network of each of the lateral position part recognition network and the lateral position key point recognition network, and the matrix data after the seventh convolution operation is input into the corresponding seventh pooling layer;

[0040] Using the seventh pooling layer of each of the lateral position part recognition network and the lateral position key point recognition network, perform the seventh pooling operation on the received matrix data after the seventh convolution operation, and input the matrix data after the seventh pooling operation into the corresponding sixth convolutional neural network;

[0041] Using the sixth convolutional neural network of each of the lateral position part recognition network and the lateral position key point recognition network, an eighth convolution operation is performed on the received matrix data after the seventh pooling operation, and the matrix data after the eighth convolution operation is input into the corresponding eighth pooling layer;

[0042] Using the eighth pooling layer of the lateral position part recognition network and the lateral position key point recognition network, the eighth pooling operation is performed on the received matrix data after the eighth convolution operation, and the matrix data after the eighth pooling operation is input into the corresponding third upsampling layer;

[0043] Using the third upsampling layers of the lateral position part recognition network and the lateral position key point recognition network, respectively, a third upsampling operation is performed on the received matrix data after the eighth pooling operation, and the matrix data after the third upsampling operation is input into the corresponding fifth convolutional layer;

[0044] Using the fifth convolutional layer of each of the lateral position part recognition network and the lateral position key point recognition network, a ninth convolutional operation is performed on the received matrix data after the third upsampling operation, and the matrix data after the ninth convolutional operation is input into the corresponding fourth classification layer;

[0045] The fourth classification layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform a classification operation on the received matrix data after the ninth convolution operation to obtain a fourth classification result, and the human body parts and key points in the lateral position are obtained according to the fourth classification results output by the lateral position part recognition network and the lateral position key point recognition network.

[0046] In a second aspect, the present invention provides a mattress control method, the method comprising:

[0047] Receiving a human body part and a human body key point sent by a server; the human body part and the human body key point are identified according to the pressure sensor data identification method described in any one of the aforementioned implementation modes;

[0048] Displaying the human body parts and the human body key points on a pressure heat map corresponding to the pressure sensor;

[0049] When a control instruction issued by the user based on the human body parts and the human body key points displayed on the pressure heat map is received, a corresponding control operation is performed on the smart mattress according to the control instruction.

[0050] In a third aspect, the present invention provides a device for identifying pressure sensor data, the device comprising:

[0051] A data acquisition module, used to acquire human body pressure data collected by a pressure sensor provided on the smart mattress;

[0052] A sleeping posture recognition module, used for inputting the human body pressure data into a pre-trained sleeping posture recognition network to obtain a human body sleeping posture;

[0053] The part and key point recognition module is used to input the human body pressure data into the part recognition network and key point recognition network corresponding to the human body sleeping posture respectively, to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained through pre-training.

[0054] In a fourth aspect, the present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for identifying pressure sensor data as described in any one of the aforementioned embodiments are implemented, or the mattress control method as described in the aforementioned embodiments are implemented.

[0055] In a fifth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the pressure sensor data identification method as described in any one of the aforementioned embodiments, or implements the mattress control method as described in the aforementioned embodiments.

[0056] The pressure sensor data recognition method, mattress control method and related device provided by the embodiment of the present invention pre-trains the sleeping position recognition network and the part recognition network and key point recognition network under different sleeping positions. When recognizing the pressure sensor data, the acquired human body pressure data is first input into the sleeping position recognition network for recognition to obtain the corresponding human body sleeping position, and then the human body pressure data is input into the part recognition network and key point recognition network corresponding to the human body sleeping position for recognition, and the different human body parts and key points under the human body sleeping position are recognized, so as to know the specific positions of different human body parts and key points on the mattress, and can also be used for subsequent intelligent operation of the mattress after visualization. Since different part recognition networks and key point recognition networks are used for part recognition and key point recognition for different sleeping positions, accurate recognition of human body parts and key points can be achieved in different sleeping position scenarios, thereby providing a strong guarantee for improving the intelligent service capabilities of the mattress and improving the user experience.

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 A schematic diagram of an application scenario applicable to an embodiment of the present invention is shown;

[0060] Figure 2 A schematic diagram showing a pressure heat map generated by a coarse-grained sensor;

[0061] Figure 3 A schematic diagram showing an error display caused by an accuracy problem of a pressure sensor system;

[0062] Figure 4 A schematic diagram showing a pressure sensor arrangement;

[0063] Figure 5 A structural schematic diagram of a smart mattress is shown;

[0064] Figure 6 A schematic diagram showing an annotation interface of a data annotation software;

[0065] Figure 7 Another schematic diagram of the annotation interface of the data annotation software is shown;

[0066] Figure 8 A schematic diagram of a grid sheet is shown;

[0067] Fig. 9 A schematic diagram showing visualization of supine position part annotation data;

[0068] Fig.10 A schematic diagram showing the visualization of prone position annotation data;

[0069] Fig.11 A schematic diagram showing the visualization of the data of the lateral position annotation is shown;

[0070] Fig.12 A schematic diagram showing the visualization of key point annotation data in the supine position;

[0071] Fig.13 A schematic diagram showing the visualization of prone position key point annotation data;

[0072] Fig.14 A schematic diagram showing the visualization of key point annotation data in the lateral position is shown;

[0073] Fig.15 A schematic diagram showing visualization of pressure sensor data;

[0074] Fig.16 A schematic diagram showing a flow chart of a method for identifying pressure sensor data provided by an embodiment of the present invention;

[0075] Fig.17 A structural schematic diagram of a sleeping posture recognition network is shown;

[0076] Fig.18 A structural diagram of MobileNetV1 Block is shown;

[0077] Fig.19 A structural schematic diagram of a supine position part recognition network and a supine position key point recognition network is shown;

[0078] Fig. 20 A structural diagram of a Fire Module is shown;

[0079] Fig.21 A structural schematic diagram of a prone position part recognition network and a prone position key point recognition network is shown;

[0080] Fig. 22 A schematic diagram of the structure of Shuffle V1Block is shown;

[0081] Fig.23A structural schematic diagram of a lateral position part recognition network and a lateral position key point recognition network is shown;

[0082] Fig.24 A schematic diagram of a flow chart of a mattress control method provided by an embodiment of the present invention is shown;

[0083] Fig.25 An example diagram of a mattress control process is shown;

[0084] Fig.26 A functional module diagram of a pressure sensor data recognition device provided by an embodiment of the present invention is shown;

[0085] Fig. 27 A block diagram of an electronic device provided by an embodiment of the present invention is shown.

[0086] Icons: 100 - server; 200 - user terminal; 300 - smart mattress; 400 - pressure sensor data recognition device; 410 - data acquisition module; 420 - sleeping posture recognition module; 430 - body part and key point recognition module; 810 - memory; 820 - processor; 830 - communication module. DETAILED DESCRIPTION

[0087] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0088] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0089] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0090] Please refer to Figure 1 , is a schematic diagram of an application scenario applicable to an embodiment of the present invention. Figure 1 As shown, the server 100, the user terminal 200, and the smart mattress 300 are all communicatively connected to each other.

[0091] The server 100 may be a local server, a cloud server or other server, and the specific server type may not be limited in the embodiment of the present invention.

[0092] The user terminal 200 may include a personal computer (PC), a tablet computer, a smart phone, a personal digital assistant (PDA), etc., which are not limited here. The user terminal 200 can exchange information with the server 100 through wireless methods such as 2G / 3G / 4G / 5G / WiFi. Of course, the connection method between the user terminal 200 and the server 100 may not be limited in the embodiment of the present invention. In some embodiments, the user terminal 200 can also be used to interact with the user, so that the user can perform some control operations of the smart mattress 300 through the user terminal 200.

[0093] The smart mattress 300 is provided with a pressure sensor, which can collect the pressure on the surface of each air bag of the smart mattress 300. When the user lies on the smart mattress 300, the pressure data collected by the pressure sensor can be transmitted to the server 100. The server 100 recognizes the pressure sensor data, and the user can perform intelligent operations on the mattress using the user terminal 200.

[0094] The accuracy of mattress pressure sensor data recognition is the cornerstone of intelligent mattress pressure sore prevention algorithm, comfort algorithm, and intelligent mattress adjustment. An efficient and accurate recognition method can provide a strong guarantee for improving the intelligent service capabilities of mattresses. However, (1) the pressure heat map generated by coarse-grained sensor data (such as Figure 2 As shown in the figure, (A), (B), (C), and (D) are prone, supine, right side, and left side positions respectively. It is difficult for the human eye to distinguish where each tiny key point is distributed, so it is difficult to get accurate labeling. This bottleneck problem increases the difficulty of training the recognition algorithm model and reduces the recognition accuracy of the recognition algorithm model. (2) Due to the inevitable error problem of the sensor (such as Figure 3 As shown in the figure, the error caused by the accuracy problem of the pressure sensor system is (A) error in the transmission process, (B) pressure error of the frame, and (C) pressure error of the part (left hand). This results in a lot of pressure errors in the collected sensor data, which brings a lot of noise to the training of the artificial intelligence neural network. At the same time, during the use of the mattress, there will inevitably be some pressure weights in unpredictable positions, causing abnormalities or noise in the pressure heat map. These two situations will have an impact and error on the recognition method, reducing the accuracy of the recognition model. (3) The recognition method is an identification link in the application of intelligent mattresses. It is necessary to use artificial intelligence methods to achieve real-time prediction, which has great requirements on the time complexity of the algorithm. However, if the scale of the network model is reduced, the prediction accuracy of the model will be reduced. It can be seen that efficient and accurate annotation of pressure heat maps, noise preprocessing of pressure data, and lightweight and high-precision network models are the basic requirements for the application of intelligent mattresses.

[0095] The embodiment of the present invention pre-trains 7 recognition networks for the recognition of pressure sensor data, namely, a sleeping posture recognition network, a supine position part recognition network, a prone position part recognition network, a lateral position part recognition network, a supine key point recognition network, a prone position key point recognition network, and a lateral position key point recognition network. In actual application, the pressure sensor data is first recognized by the sleeping posture recognition network to obtain the corresponding human sleeping posture; then, it is recognized by the part recognition network (supine position part recognition network, prone position part recognition network or lateral position part recognition network) and the key point recognition network (supine position key point recognition network, prone position key point recognition network or lateral position key point recognition network) corresponding to the human sleeping posture, thereby obtaining different human body parts and human body key points under the human sleeping posture, so as to facilitate knowing the specific positions of different human body parts and human body key points on the mattress, and after visualization, it can also be used for subsequent mattress intelligent operation.

[0096] The following is a detailed description of the sleeping posture recognition network involved in the embodiment of the present invention, and the training process of the body part recognition network and the key point recognition network under different sleeping postures.

[0097] The embodiment of the present invention utilizes the collected 8×21 grid pressure data (visualized pressure data) and recognizes the data to output an artificial intelligence solution for sleeping posture, human body parts and key points of the human body. Figure 4 and Figure 5As shown, the pressure sensors are evenly distributed on the 90CM×190CM bed. The effective sensing area of ​​the pressure sensors is 80×189CM, which can be composed of 3 80×63CM pressure sensors. Each pressure sensor can collect 8×7 grid pressure data. The pressure sensors collect pressure data in real time to form 8×21 grid pressure data.

[0098] For the collection and labeling of pressure sensor data, the embodiment of the present invention adopts a data labeling software that can be installed on a terminal such as a mobile phone or a tablet, such as Figure 6 and Figure 7 As shown in Figure 1, the data annotation software can visualize the data of the pressure sensor on the mattress in real time. At the same time, an interactive annotation module is designed on the data annotation software to perform real-time annotation according to the actual sleeping posture, body parts and key points of the person lying on the mattress.

[0099] In order to be able to more accurately mark on the data marking software, the embodiment of the present invention designs a grid sheet of the same size as the pressure sensor (size 8×21, such as Figure 8 ), the grid sheets and the actual pressure sensors are in one-to-one correspondence, so that different key points of the human body can be mapped to different grid sheets, and the grid sheets and the annotation matrix on the data annotation software are also in one-to-one correspondence, so that the human body parts and key points can be correctly labeled.

[0100] In this embodiment, it is necessary to collect 7 types of data and corresponding annotation data to support the model training of the sleeping posture recognition network, the body part recognition algorithm and the key point recognition algorithm.

[0101] For sleeping position data and corresponding annotated data, sleeping positions are divided into four types: supine, prone, left side, and right side. During the annotation process, we searched for men and women with different BMIs (Body Mass Index), different ages, and different body types for experimental annotation. At the same time, we collected various supine, prone, left side, and right side sleeping positions, covering basically all common sleeping positions in medicine, making the collected data comprehensive.

[0102] For supine position data and corresponding annotation data, such as Fig. 9 As shown in the figure, there are 17 types of supine positions: head, neck, left shoulder, right shoulder, waist, buttocks, left thigh, right thigh, left calf, left calf, left hind arm, right hind arm, left forearm, right forearm, left heel, right heel, back. During the labeling process, men and women with different BMI, different ages and different body shapes were found for experimental labeling, and their 17 subdivided parts were labeled. The experimental labeling covers basically all different common supine sleeping positions in medicine, making the collected data comprehensive.

[0103] For prone position data and corresponding annotation data, such as Fig.10 As shown in the figure, there are 16 parts of the prone position: head, neck, left chest, right chest, waist, buttocks, left thigh, right thigh, left calf, right calf, left hind arm, right hind arm, left forearm, right forearm, left heel, and right heel. During the labeling process, men and women with different BMIs, ages, and body shapes were found for experimental labeling, and their 16 subdivided parts were labeled. The experimental labeling covers basically all the different prone sleeping positions commonly used in medicine, making the collected data comprehensive.

[0104] For the lateral position data and the corresponding annotation data, such as Fig.11 As shown in the figure, there are 16 types of side-lying positions: head, neck, shoulder, waist, buttocks, left thigh, right thigh, left calf, right calf, left heel, right heel, back, left hind arm, right hind arm, left forearm, and right forearm. During the labeling process, men and women with different BMIs, ages, and body shapes were found for experimental labeling, and their 16 subdivided parts were labeled. The experimental labeling covers basically all the different side-lying sleeping positions commonly used in medicine, making the collected data comprehensive.

[0105] For the supine position key point data and the corresponding annotation data, such as Fig.12 As shown in the figure, there are 7 key points of supine position: headrest, shoulder blade, left elbow, right elbow, sacrum, left heel, and right heel. During the annotation process, men and women with different BMI, different ages, and different body shapes were found for experimental annotation, and their 7 subdivided key points were annotated. The experimental annotation covers basically all different common supine sleeping positions in medicine, making the collected data comprehensive.

[0106] For the prone position key point data and the corresponding annotation data, such as Fig.13 As shown in the figure, there are 8 key points of prone position: head, chest, left knee, right knee, left toe, right toe, left pelvis, and right pelvis. During the annotation process, we searched for men and women with different BMI, different ages, and different body shapes for experimental annotation, and annotated their 8 subdivided key points. The experimental annotation covers basically all different prone sleeping positions commonly used in medicine, making the collected data comprehensive.

[0107] For the lateral position key point data and the corresponding annotation data, such as Fig.14 As shown in the figure, the key points of the side-lying position are divided into 6 types: ear, acromion, rib, hip, knee, and ankle. During the labeling process, men and women with different BMI, different ages, and different body shapes were found for experimental labeling, and their 6 subdivided key points were labeled. The experimental labeling covers basically all the different side-lying sleeping positions commonly used in medicine, making the collected data comprehensive.

[0108] The above 7 types of data are all collected and labeled in pairs. The labeled sleeping position data includes pressure sensor data and corresponding sleeping position labeling data, the labeled body part data includes pressure sensor data and corresponding body part labeling data, and the labeled key point data includes pressure sensor data and corresponding key point labeling data. The above data collection and labeling scheme can more accurately label and collect pressure data, providing high-quality data resources for model training.

[0109] In practical applications, considering that there may be errors in manually labeled data, in order to facilitate the inspection of whether the labeled data is wrong, the pressure sensor data and the labeled data can be visualized. The data collected by the pressure sensor is distributed in [0, 68], and 90% of the normal human pressure data is distributed in [16, 40]. Using the formula Process the pressure sensor data and normalize it to the range of 1 to 255; where x max Indicates the maximum value in the collected pressure sensor data, x min represents the minimum value in the collected pressure sensor data, x represents the pressure sensor data before processing, and X represents the pressure sensor data after processing. Then, the matrix size is expanded to 800x2100 using the bilinear difference method, and finally RGB processing is performed. The red depth is dyed according to the size of each pressure sensor data to obtain the pressure heat map corresponding to the pressure sensor data (such as Fig.15 shown).

[0110] The labeled data is a digital matrix of 8x21. In this embodiment, the labeled data needs to be digitized. The processing rules of the labeled data are as follows:

[0111] For sleeping position data, the labeled data are assigned a value of 0 for supine position, 1 for prone position, 2 for left side position, and 3 for right side position.

[0112] For the supine position data, the labeled data are assigned 1 to the neck, 2 to the left shoulder, 3 to the right shoulder, 4 to the waist, 5 to the buttocks, 6 to the left thigh, 7 to the right thigh, 8 to the left calf, 9 to the left calf, 10 to the left hind arm, 11 to the right hind arm, 12 to the left forearm, 13 to the right forearm, 14 to the left heel, 15 to the right heel, 16 to the back, 17 to the head, and 0 to the remaining areas on the mattress where there are no human parts.

[0113] For the prone position data, the labeled data are assigned 1 to the head, 2 to the neck, 3 to the left chest, 4 to the right chest, 5 to the waist, 6 to the buttocks, 7 to the left thigh, 8 to the right thigh, 9 to the left calf, 10 to the right calf, 11 to the left hind arm, 12 to the right hind arm, 13 to the left forearm, 14 to the right forearm, 15 to the left heel, 16 to the right heel, and the rest of the areas on the mattress where there are no human parts are assigned 0.

[0114] For the side-lying position data, the labeled data are assigned 1 to the head, 2 to the neck, 3 to the shoulder, 4 to the waist, 5 to the buttocks, 6 to the left thigh, 7 to the right thigh, 8 to the left calf, 9 to the right calf, 10 to the left heel, 11 to the right heel, 12 to the back, 13 to the left hind arm, 14 to the right hind arm, 15 to the left forearm, 16 to the right forearm, and the rest of the areas on the mattress where there are no human parts are assigned 0.

[0115] For the key point data of the supine position, the labeled data is assigned a value of 1 for the headrest, 2 for the shoulder blade, 3 for the left elbow, 4 for the right elbow, 5 for the sacrum, 6 for the left heel, and 7 for the right heel. The rest of the areas on the mattress that do not have key points of the human body are assigned a value of 0.

[0116] For the key point data of the prone position, the labeled data are assigned a value of 1 for the head, 2 for the chest, 3 for the left knee, 4 for the right knee, 5 for the left toe, 6 for the right toe, 7 for the left pelvis, and 8 for the right pelvis. The rest of the areas on the mattress that do not have key points of the human body are assigned a value of 0.

[0117] For the key point data of the side-lying position, the labeled data is assigned a value of 1 for the ear, 2 for the acromion, 3 for the ribs, 4 for the hips, 5 for the knees, 6 for the ankles, and 0 for the remaining areas on the mattress that do not have key points of the human body.

[0118] The processed labeled data is processed by RGB and converted into a visualization. Finally, a corresponding check is performed based on the pressure heat map corresponding to the pressure sensor data and the visualization of the labeled data to determine whether there are any labeling errors and make corrections. In this way, the accuracy of the labeled data can be effectively improved.

[0119] In addition, considering the inevitable error problem of the pressure sensor data itself, the pressure sensor data is noise preprocessed in this embodiment: the outermost data of the original data of the pressure sensor is set to 0, all the pressure sensor data is cleaned, and the pressure sensor data with a histogram mean less than 16 is deleted. The model obtained by neural network training is used to predict the pressure sensor data, manually screen the erroneous data and the data with a prediction confidence less than 50%, check and correct its labeled data, and at the same time, the noise type of the predicted erroneous data is counted, and then the data set is corrected a priori based on the statistics. In this way, the accuracy of the labeled data can be further improved, and the noise caused by the error problem of the pressure sensor to the training of the network model can be effectively avoided, thereby affecting the accuracy of the network model.

[0120] In this embodiment, after completing the processing of 7 types of pressure sensor data and the corresponding annotation data, the pressure sensor data and its corresponding sleeping posture annotation data can be used as a training set to train a pre-designed sleeping posture recognition network. The trained sleeping posture recognition network can be used to realize the recognition of human sleeping posture.

[0121] The supine pressure sensor data and its corresponding supine part annotation data are used as training sets to train the pre-designed supine part recognition network. The trained supine part recognition network can be used to recognize different human body parts in the supine position.

[0122] The prone position pressure sensor data and its corresponding prone position part annotation data are used as a training set to train a pre-designed prone position part recognition network. The trained prone position part recognition network can be used to recognize different human body parts in the prone position.

[0123] The lateral position pressure sensor data and its corresponding lateral position part annotation data are used as a training set to train a pre-designed lateral position part recognition network. The trained lateral position part recognition network can be used to recognize different human body parts in the lateral position.

[0124] The supine pressure sensor data and its corresponding supine key point annotation data are used as training sets to train the pre-designed supine key point recognition network. The trained supine key point recognition network can be used to recognize different key points of the human body in the supine position.

[0125] The prone position pressure sensor data and its corresponding prone position key point annotation data are used as training sets to train the pre-designed prone position key point recognition network. The trained prone position key point recognition network can be used to recognize different key points of the human body in the prone position.

[0126] The lateral position pressure sensor data and its corresponding lateral position key point annotation data are used as a training set to train the pre-designed lateral position key point recognition network. The trained lateral position key point recognition network can be used to recognize different human key points in the lateral position.

[0127] Below, the recognition process of pressure sensor data is described in detail based on the trained sleeping posture recognition network, the body part recognition network under different sleeping postures, and the key point recognition network.

[0128] Please refer to Fig.16 , is a flow chart of a method for identifying pressure sensor data provided by an embodiment of the present invention. It should be noted that the method for identifying pressure sensor data in the embodiment of the present invention is not based on Fig.16 The specific order below is not limited. It should be understood that in other embodiments, the order of some steps in the method for identifying pressure sensor data in the embodiment of the present invention can be interchanged according to actual needs, or some steps can be omitted or deleted. The method for identifying pressure sensor data can be composed of Figure 1 The server 100 shown in FIG. Fig.16 The specific process shown is described in detail.

[0129] Step S101, obtaining human body pressure data collected by a pressure sensor provided on the smart mattress.

[0130] In this embodiment, the server 100 can obtain human body pressure data collected by a pressure sensor provided on the smart mattress 300 .

[0131] Step S102, inputting the human body pressure data into a pre-trained sleeping posture recognition network to obtain the human body sleeping posture.

[0132] In this embodiment, when the server 100 recognizes the human body pressure data, it can input the human body pressure data into a pre-trained sleeping position recognition network to recognize the human body sleeping position and obtain the current human body sleeping position.

[0133] Step S103, inputting the human body pressure data into the part recognition network and the key point recognition network corresponding to the human body sleeping posture respectively, to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained by pre-training.

[0134] In this embodiment, different part recognition networks and key point recognition networks are pre-trained for different sleeping positions. After the corresponding human sleeping position is recognized using the sleeping position recognition network, the part recognition network and key point recognition network corresponding to the human sleeping position can be called, and the human body pressure data can be input into the part recognition network for part recognition to obtain the human body parts; the human body pressure data can be input into the key point recognition network for key point recognition to obtain the human body key points.

[0135] For example, when the identified human sleeping position is supine, the human body pressure data is input into the supine position part recognition network for part recognition to obtain the human body parts in the supine position; the human body pressure data is input into the supine key point recognition network for key point recognition to obtain the human body key points in the supine position. Similarly, when the identified human sleeping position is prone, the human body pressure data is input into the prone position part recognition network for part recognition to obtain the human body parts in the prone position; the human body pressure data is input into the prone position key point recognition network for key point recognition to obtain the human body key points in the prone position. When the identified human sleeping position is lateral (left side or right side), the human body pressure data is input into the lateral position part recognition network for part recognition to obtain the human body parts in the lateral position; the human body pressure data is input into the lateral position key point recognition network for key point recognition to obtain the human body key points in the left side or right side.

[0136] In practical applications, considering that the collected human body pressure data may have pressure errors, the human body pressure data can be subjected to noise preprocessing before being input into the sleeping posture recognition network, the body part recognition network and the key point recognition network. For example, the outermost circle data of the human body pressure data can be set to 0. This can effectively improve the recognition accuracy of the network.

[0137] It can be seen that the method for identifying pressure sensor data provided by the embodiment of the present invention, by pre-training the sleeping position recognition network and the part recognition network and key point recognition network under different sleeping positions, first inputs the acquired human body pressure data into the sleeping position recognition network for recognition to obtain the corresponding human body sleeping position, and then inputs the human body pressure data into the part recognition network and key point recognition network corresponding to the human body sleeping position for recognition, and identifies the different human body parts and key points under the human body sleeping position. Since different part recognition networks and key point recognition networks are used for part recognition and key point recognition for different sleeping positions, accurate recognition of human body parts and key points can be achieved in different sleeping position scenarios, thereby providing a strong guarantee for improving the intelligent service capabilities of the mattress and enhancing the user experience.

[0138] In one embodiment, the sleeping posture recognition network can be used Fig.17 The structure shown. Fig.17As shown, the sleeping posture recognition network includes a first convolution layer, a first pooling layer, a second convolution layer, a second pooling layer, a fully connected layer and a first classification layer. After the human body pressure data is input into the sleeping posture recognition network, it will be processed by the first convolution layer, the first pooling layer, the second convolution layer, the second pooling layer, the fully connected layer and the first classification layer in sequence, and then the human body sleeping posture is determined according to the result output by the first classification layer. Specifically, the above step S102 may include:

[0139] The human body pressure data is input into a pre-trained sleeping posture recognition network, and the first convolution layer is used to perform a first depth-separable convolution operation on the input human body pressure data, and the matrix data after the first depth-separable convolution operation is input into the first pooling layer; the first pooling layer is used to perform a first pooling operation on the matrix data after the first depth-separable convolution operation, and the matrix data after the first pooling operation is input into the second convolution layer; the second convolution layer is used to perform a second depth-separable convolution operation on the matrix data after the first pooling operation, and the matrix data after the second depth-separable convolution operation is input into the second pooling layer; the second pooling layer is used to perform a second pooling operation on the matrix data after the second depth-separable convolution operation, and the matrix data after the second pooling operation is input into the fully connected layer; the fully connected layer is used to perform a fully connected operation on the matrix data after the second pooling operation, and the matrix data after the fully connected operation is input into the first classification layer; the first classification layer is used to perform a classification operation on the matrix data after the fully connected operation to obtain a first classification result, and the human body sleeping posture is determined according to the first classification result.

[0140] In this embodiment, the sleeping posture recognition network is a lightweight network that can perform data analysis on pressure sensor data in real time. The main module of its network structure can be implemented according to MobileNetV1 Block, wherein MobileNetV1Block includes a depthwise separable convolution layer (DW) with a convolution kernel of 3×3, a batch normalization layer (Batch Normalization, BN), a rectified linear unit (ReLU), a convolution layer with a convolution kernel of 1×1, a batch normalization layer, and a rectified linear unit, such as Fig.18 As shown. The first convolution layer and the second convolution layer can be implemented by using the depth-separable convolution layer in MobileNetV1 Block.

[0141] In an example, as shown in Table 1, Input represents the size and dimension of the matrix data input to each layer, Operator represents what operation is performed, Output represents the size and dimension of the output of this layer, and Filter Shape represents the size and dimension of the convolution kernel.

[0142] Table 1

[0143]

[0144] For example, if the size of the human body pressure data collected by the pressure sensor is 21x8, then before inputting the human body pressure data into the sleeping position recognition network, the human body pressure data needs to be cropped to adapt to the network input, and the last row of data is cropped from 21×8 to 20×8, and noise preprocessing is performed before inputting the sleeping position recognition network. Among them, the human body pressure data input to the first convolutional layer is a matrix data of 20×8×1 (i.e., a matrix of 20 in length and 8 in width). After the first depth-separable convolution operation, the matrix data of 20×8×16 is output to the first pooling layer. After the first pooling layer performs the first pooling operation on the 20×8×16 matrix data, it outputs the 10×4×16 matrix data to the second convolutional layer; after the second convolutional layer performs the second depth-separable convolution operation on the 10×4×16 matrix data, it outputs the 10×4×32 matrix data to the second pooling layer; after the second pooling layer performs the second pooling operation on the 10×4×32 matrix data, it outputs the 5×2×32 matrix data to the fully connected layer; after the fully connected layer performs the fully connected operation on the 5×2×32 matrix data, it outputs the 1×1×4 matrix data to the first classification layer; the first classification layer performs a classification operation on the 1×1×4 matrix data, and the output first classification result can be 4 probability values ​​belonging to each sleeping position category, and based on the size of the 4 probability values, it is determined that the human sleeping position is supine, prone, left lateral position or right lateral position.

[0145] In this embodiment, human sleeping positions include supine, prone and lateral positions, among which the part recognition network and key point recognition network corresponding to the supine position have the same network structure and different network parameters; the part recognition network and key point recognition network corresponding to the prone position have the same network structure and different network parameters; the part recognition network and key point recognition network corresponding to the lateral position have the same network structure and different network parameters.

[0146] It can be understood that the part recognition network and key point recognition network under different sleeping positions are trained in the training phase using the labeled part data under different sleeping positions and the labeled key point data. Since the input data in the training phase are different, the final network parameters are also different.

[0147] In one embodiment, the supine position part recognition network and the supine position key point recognition network corresponding to the supine position can be adopted Fig.19 The structure of Fig.19As shown, the supine position part recognition network and the supine position key point recognition network both include a first convolutional neural network, a third pooling layer, a second convolutional neural network, a fourth pooling layer, a first upsampling layer, a third convolutional layer, and a second classification layer. After the human body pressure data is input into the supine position part recognition network and the supine position key point recognition network, it will be processed by the first convolutional neural network, the third pooling layer, the second convolutional neural network, the fourth pooling layer, the first upsampling layer, the third convolutional layer, and the second classification layer of the supine position part recognition network and the supine position key point recognition network respectively in sequence, and then the human body parts in the supine position are determined according to the results output by the second classification layer of the supine position part recognition network, and the human body key points in the supine position are determined according to the results output by the second classification layer of the supine position key point recognition network. Specifically, the above-mentioned step S103 may include:

[0148] When the human body is in the supine position, the human body pressure data is respectively input into the supine position part recognition network and the supine position key point recognition network, and the first convolutional neural network of the supine position part recognition network and the supine position key point recognition network is used to perform the first convolution operation on the input human body pressure data, and the matrix data after the first convolution operation is input into the corresponding third pooling layer. The third pooling layer of the supine position part recognition network and the supine position key point recognition network is used to perform the third pooling operation on the received matrix data after the first convolution operation, and the matrix data after the third pooling operation is input into the corresponding second convolutional neural network. The second convolutional neural network of the supine position part recognition network and the supine position key point recognition network is used to perform the second convolution operation on the received matrix data after the third pooling operation, and the matrix data after the second convolution operation is input into the corresponding fourth pooling layer. The fourth pooling layer of the supine position part recognition network and the supine position key point recognition network is used to perform the fourth pooling operation on the received matrix data after the second convolution operation, and the matrix data after the fourth pooling operation is input into the corresponding first upsampling layer. Using the first upsampling layer of the supine position part recognition network and the supine position key point recognition network, the first upsampling operation is performed on the received matrix data after the fourth pooling operation, and the matrix data after the first upsampling operation is input into the corresponding third convolution layer. Using the third convolution layer of the supine position part recognition network and the supine position key point recognition network, the third convolution operation is performed on the received matrix data after the first upsampling operation, and the matrix data after the third convolution operation is input into the corresponding second classification layer. Using the second classification layer of the supine position part recognition network and the supine position key point recognition network, the matrix data after the third convolution operation is classified to obtain the second classification result, and according to the second classification results output by the supine position part recognition network and the supine position key point recognition network, the human body parts and human body key points in the supine position are obtained.

[0149] In this embodiment, the supine position part recognition network and the supine position key point recognition network are lightweight networks that can perform supine position part recognition and supine position key point recognition on pressure sensor data in real time. The main module of the network structure is SqueezeNet Block, which is composed of Fire Module, batch normalization layer, rectified linear unit, convolution layer with convolution kernel of 1×1, batch normalization layer, and rectified linear unit in sequence. The structure of Fire Module is as follows: Fig. 20 As shown, the first convolutional neural network and the second convolutional neural network can be implemented using SqueezeNet Block.

[0150] In an example, as shown in Table 2, Input represents the size and dimension of the matrix data input to each layer, Operator represents what operation is performed, Output represents the size and dimension of the output of this layer, Filter Shape represents the size and dimension of the convolution kernel, and the number of the first upsampling layer is 2.

[0151] Table 2

[0152] Input Operator Output Filter Shape 20×8×1 The first convolutional neural network (SqueezeNet Block) 20×8×16 3×3×16 20×8×16 The third pooling layer (Pooling Layers) 10×4×16 2×2 10×4×16 Second Convolutional Neural Network (SqueezeNet Block) 10×4×32 3×3×32 10×4×32 Fourth Pooling Layers 5×2×32 2×2 5×2×32 First upsampling layer (Upsample) 10×4×16 Upsample 10×4×16 First upsampling layer (Upsample) 20×8×32 Upsample 20×8×32 The third convolutional layer (ConV) 20×8×17 1×1×17 20×8×17 Second classification layer (Softmax) 20×8 Segment

[0153] For example, the human body pressure data collected by the pressure sensor is cropped, the last row of data is cropped, the size is cropped from 21×8 to 20×8, and noise preprocessing is performed, and then the data is input into the supine position recognition network and the supine position key point recognition network for recognition. Among them, the human body pressure data input into the first convolutional neural network is a 20×8×1 matrix data. After the first convolutional neural network performs the first convolution operation, the 20×8×16 matrix data is output to the third pooling layer. After the third pooling layer performs the third pooling operation on the 20×8×16 matrix data, the 10×4×16 matrix data is output to the second convolutional neural network. After the second convolutional neural network performs the second convolution operation on the 10×4×16 matrix data, the 10×4×32 matrix data is output to the fourth pooling layer. After the fourth pooling layer performs the fourth pooling operation on the 10×4×32 matrix data, the 5×2×32 matrix data is output to the first first upsampling layer. After the first upsampling layer performs the first upsampling operation on the 5×2×32 matrix data, it outputs the 10×4×16 matrix data to the second first upsampling layer. After the second first upsampling layer performs the first upsampling operation on the 10×4×16 matrix data, it outputs the 20×8×32 matrix data to the third convolution layer. After the third convolution layer performs the third convolution operation on the 20×8×32 matrix data, it outputs the 20×8×17 matrix data to the second classification layer. The second classification layer performs a classification operation on the 20×8×17 matrix data and outputs a second classification result. Among them, the second classification result output by the supine position part recognition network is the probability value of each part category, and the human body parts in the supine position are identified based on the probability value of each part category; the second classification result output by the supine position key point recognition network is the probability value of each key point category, and the human body key points in the supine position are identified based on the probability value of each key point category.

[0154] In one embodiment, the prone position part recognition network and the prone position key point recognition network corresponding to the prone position can be adopted Fig.21 The structure of Fig.21 As shown, the prone position part recognition network and the prone position key point recognition network both include a third convolutional neural network, a fifth pooling layer, a fourth convolutional neural network, a sixth pooling layer, a second upsampling layer, a fourth convolutional layer and a third classification layer. After the human body pressure data is input into the prone position part recognition network and the prone position key point recognition network, it will be processed in sequence by the third convolutional neural network, the fifth pooling layer, the fourth convolutional neural network, the sixth pooling layer, the second upsampling layer, the fourth convolutional layer and the third classification layer of the prone position part recognition network and the prone position key point recognition network, and then the human body parts in the prone position are determined according to the results output by the third classification layer of the prone position part recognition network, and the human body key points in the prone position are determined according to the results output by the third classification layer of the prone position key point recognition network. Specifically, the above-mentioned step S103 may include:

[0155] When the human body is in a prone position, the human body pressure data is respectively input into the prone position part recognition network and the prone position key point recognition network, and the third convolutional neural network of the prone position part recognition network and the prone position key point recognition network is used to perform a fourth convolution operation on the input human body pressure data, and the matrix data after the fourth convolution operation is input into the corresponding fifth pooling layer. The fifth pooling operation is performed on the received matrix data after the fourth convolution operation using the fifth pooling layer of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the fifth pooling operation is input into the corresponding fourth convolutional neural network. The fifth convolution operation is performed on the received matrix data after the fifth pooling operation using the fourth convolutional neural network of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the fifth convolution operation is input into the corresponding sixth pooling layer. The sixth pooling operation is performed on the received matrix data after the fifth convolution operation using the sixth pooling layer of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the sixth pooling operation is input into the corresponding second upsampling layer. The second upsampling layer of the prone position part recognition network and the prone position key point recognition network is used to perform a second upsampling operation on the received matrix data after the sixth pooling operation, and the matrix data after the second upsampling operation is input into the corresponding fourth convolution layer. The fourth convolution layer of the prone position part recognition network and the prone position key point recognition network is used to perform a sixth convolution operation on the received matrix data after the second upsampling operation, and the matrix data after the sixth convolution operation is input into the corresponding third classification layer. The third classification layer of the prone position part recognition network and the prone position key point recognition network is used to perform a classification operation on the received matrix data after the sixth convolution operation to obtain a third classification result, and according to the third classification results output by the prone position part recognition network and the prone position key point recognition network, the human body parts and human body key points in the prone position are obtained.

[0156] In this embodiment, the prone position recognition network and the prone position key point recognition network are lightweight networks that can perform prone position recognition and prone key point recognition on pressure sensor data in real time. The main module of its network structure is Shuffle V1 Block, which is composed of a group convolution layer (GroupConvolution, GC) with a convolution kernel of 1×1, a batch normalization layer, a rectified linear unit, a feature communication layer (Channel Shuffle), a depth-separable convolution layer, a batch normalization layer, a group convolution layer with a convolution kernel of 1×1, a batch normalization layer, an ADD layer (this layer performs the operation of superimposing the original input with the output of the previous layer), and a rectified linear unit, such as Fig. 22As shown. Among them, the third convolutional neural network and the fourth convolutional neural network can be implemented by using ShuffleV1 Block.

[0157] In an example, as shown in Table 3, Input represents the size and dimension of the matrix data input to each layer, Operator represents what operation is performed, Output represents the size and dimension of the output of this layer, Filter Shape represents the size and dimension of the convolution kernel, and the number of the second upsampling layer is 2.

[0158] Table 3

[0159] Input Operator Output Filter Shape 20×8×1 The third convolutional neural network (ShuffleV1 Block) 20×8×16 3×3×16 20×8×16 Fifth Pooling Layers 10×4×16 2×2 10×4×16 Fourth Convolutional Neural Network (ShuffleV1 Block) 10×4×32 3×3×32 10×4×32 Pooling Layers 5×2×32 2×2 5×2×32 The second upsampling layer (Upsample) 10×4×16 Upsample 10×4×16 The second upsampling layer (Upsample) 20×8×32 Upsample 20×8×32 Fourth convolutional layer (ConV) 20×8×16 1×1×16 20×8×16 The third classification layer (Softmax) 20×8 Segment

[0160] For example, the human body pressure data collected by the pressure sensor is cropped, the last row of data is cropped, the size is cropped from 21×8 to 20×8, and noise preprocessing is performed, and then the data is input into the prone position recognition network and the prone position key point recognition network for recognition. Among them, the human body pressure data input into the third convolutional neural network is a 20×8×1 matrix data. After the third convolutional neural network performs the fourth convolution operation, the matrix data of 20×8×16 is output to the fifth pooling layer. After the fifth pooling layer performs the fifth pooling operation on the 20×8×16 matrix data, the matrix data of 10×4×16 is output to the fourth convolutional neural network. After the fourth convolutional neural network performs the fifth convolution operation on the 10×4×16 matrix data, the matrix data of 10×4×32 is output to the sixth pooling layer. After the sixth pooling layer performs the sixth pooling operation on the 10×4×32 matrix data, the matrix data of 5×2×32 is output to the first second upsampling layer. After the first second upsampling layer performs the second upsampling operation on the 5×2×32 matrix data, it outputs the 10×4×16 matrix data to the second second upsampling layer. After the second second upsampling layer performs the second upsampling operation on the 10×4×16 matrix data, it outputs the 20×8×32 matrix data to the fourth convolution layer. After the fourth convolution layer performs the sixth convolution operation on the 20×8×32 matrix data, it outputs the 20×8×16 matrix data to the third classification layer. The third classification layer performs a classification operation on the 20×8×16 matrix data and outputs the third classification result. Among them, the third classification result output by the prone position part recognition network is the probability value of each part category, and the human body parts in the prone position are identified based on the probability value of each part category; the third classification result output by the prone position key point recognition network is the probability value of each key point category, and the human body key points in the prone position are identified based on the probability value of each key point category.

[0161] In one embodiment, the lateral position part recognition network and the lateral position key point recognition network corresponding to the lateral position can be used Fig.23 The structure of Fig.23As shown, the lateral position part recognition network and the lateral position key point recognition network both include a fifth convolutional neural network, a seventh pooling layer, a sixth convolutional neural network, an eighth pooling layer, a third upsampling layer, a fifth convolutional layer, and a fourth classification layer. After the human body pressure data is input into the lateral position part recognition network and the lateral position key point recognition network, it will be processed in turn by the fifth convolutional neural network, the seventh pooling layer, the sixth convolutional neural network, the eighth pooling layer, the third upsampling layer, the fifth convolutional layer, and the fourth classification layer of the lateral position part recognition network and the lateral position key point recognition network, and then the human body parts in the lateral position are determined according to the results output by the fourth classification layer of the lateral position part recognition network, and the human body key points in the lateral position are determined according to the results output by the fourth classification layer of the lateral position key point recognition network. Specifically, the above-mentioned step S103 may include:

[0162] When the human body sleeping posture is a side-lying position, the human body pressure data is respectively input into the side-lying position part recognition network and the side-lying position key point recognition network, and the fifth convolutional neural network of the side-lying position part recognition network and the side-lying position key point recognition network is used to perform the seventh convolution operation on the input human body pressure data, and the matrix data after the seventh convolution operation is input into the corresponding seventh pooling layer. The seventh pooling layer of the side-lying position part recognition network and the side-lying position key point recognition network is used to perform the seventh pooling operation on the received matrix data after the seventh convolution operation, and the matrix data after the seventh pooling operation is input into the corresponding sixth convolutional neural network. The sixth convolutional neural network of the side-lying position part recognition network and the side-lying position key point recognition network is used to perform the eighth convolution operation on the received matrix data after the seventh pooling operation, and the matrix data after the eighth convolution operation is input into the corresponding eighth pooling layer. The eighth pooling layer of the side-lying position part recognition network and the side-lying position key point recognition network is used to perform the eighth pooling operation on the received matrix data after the eighth convolution operation, and the matrix data after the eighth pooling operation is input into the corresponding third upsampling layer. The third upsampling layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform a third upsampling operation on the received matrix data after the eighth pooling operation, and the matrix data after the third upsampling operation is input into the corresponding fifth convolution layer. The fifth convolution layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform a ninth convolution operation on the received matrix data after the third upsampling operation, and the matrix data after the ninth convolution operation is input into the corresponding fourth classification layer. The fourth classification layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform a classification operation on the received matrix data after the ninth convolution operation to obtain a fourth classification result, and according to the fourth classification results output by the lateral position part recognition network and the lateral position key point recognition network, the human body parts and human body key points in the lateral position are obtained.

[0163] In this embodiment, when the human body's sleeping position is the left side lying position or the right side lying position, the lateral position part recognition network and the lateral position key point recognition network are used for recognition. Among them, the lateral position part recognition network and the lateral position key point recognition network are lightweight networks that can perform lateral position part recognition and lateral position key point recognition on pressure sensor data in real time. The main module of its network structure is MobileNetV1 Block, which is composed of a depth-separable convolution layer with a convolution kernel of 3×3, a batch normalization layer, a rectified linear unit, a convolution layer with a convolution kernel of 1×1, a batch normalization layer, and a rectified linear unit. Among them, the fifth convolutional neural network and the sixth convolutional neural network can be implemented using MobileNetV1 Block.

[0164] In an example, as shown in Table 4, Input represents the size and dimension of the matrix data input to each layer, Operator represents what operation is performed, Output represents the size and dimension of the output of this layer, Filter Shape represents the size and dimension of the convolution kernel, and the number of the third upsampling layer is 2.

[0165] Table 4

[0166] Input Operator Output Filter Shape 20×8×1 The fifth convolutional neural network (MobileNetV1 Block) 20×8×16 3×3×16 20×8×16 Pooling Layers 10×4×16 2×2 10×4×16 The sixth convolutional neural network (MobileNetV1 Block) 10×4×32 3×3×32 10×4×32 Pooling Layers 5×2×32 2×2 5×2×32 The third upsampling layer (Upsample) 10×4×16 Upsample 10×4×16 The third upsampling layer (Upsample) 20×8×32 Upsample 20×8×32 Fifth convolutional layer (ConV) 20×8×16 1×1×16 20×8×16 Fourth classification layer (Softmax) 20×8 Segment

[0167] For example, the human body pressure data collected by the pressure sensor is cropped, the last row of data is cropped, the size is cropped from 21×8 to 20×8, and noise preprocessing is performed, and then the data is input into the lateral position recognition network and the lateral position key point recognition network for recognition. Among them, the human body pressure data input into the fifth convolutional neural network is a 20×8×1 matrix data. After the fifth convolutional neural network performs the seventh convolution operation, the matrix data of 20×8×16 is output to the seventh pooling layer. After the seventh pooling layer performs the seventh pooling operation on the 20×8×16 matrix data, the matrix data of 10×4×16 is output to the sixth convolutional neural network. After the sixth convolutional neural network performs the eighth convolution operation on the 10×4×16 matrix data, the matrix data of 10×4×32 is output to the eighth pooling layer. After the eighth pooling layer performs the eighth pooling operation on the 10×4×32 matrix data, the matrix data of 5×2×32 is output to the first third upsampling layer. After the first third upsampling layer performs the third upsampling operation on the 5×2×32 matrix data, it outputs the 10×4×16 matrix data to the second third upsampling layer. After the second third upsampling layer performs the third upsampling operation on the 10×4×16 matrix data, it outputs the 20×8×32 matrix data to the fifth convolution layer. After the fifth convolution layer performs the ninth convolution operation on the 20×8×32 matrix data, it outputs the 20×8×16 matrix data to the fourth classification layer. The fourth classification layer performs a classification operation on the 20×8×16 matrix data and outputs the fourth classification result. Among them, the fourth classification result output by the lateral position part recognition network is the probability value of each part category, and the human body parts in the lateral position are identified based on the probability value of each part category; the fourth classification result output by the lateral position key point recognition network is the probability value of each key point category, and the human body key points in the lateral position are identified based on the probability value of each key point category.

[0168] In this embodiment, after experimental measurement, the above-mentioned sleeping position recognition network, body part recognition network and key point recognition network are used to perform sleeping position recognition, body part recognition and body key point recognition respectively, and excellent recognition rates can be achieved. Among them, the sleeping position recognition network based on the lightweight network can achieve an accuracy of 98% for sleeping position recognition; the supine position part recognition network based on image segmentation can achieve an accuracy of 90.13% for supine body part recognition; the lateral position part recognition network based on image segmentation can achieve an accuracy of 89.16% for left-side body part recognition; the lateral position part recognition network based on image segmentation can achieve an accuracy of 91.90% for right-side body part recognition; the prone position part recognition network based on image segmentation can achieve an accuracy of 86.95% for prone body part recognition. The supine key point recognition network based on posture estimation can achieve an accuracy of 88.33% for the key point recognition of the human body in the supine position; the lateral key point recognition network based on posture estimation can achieve an accuracy of 92.57% for the key point recognition of the human body in the left lateral position; the lateral key point recognition network based on posture estimation can achieve an accuracy of 89.97% for the key point recognition of the human body in the right lateral position; the prone key point recognition network based on posture estimation can achieve an accuracy of 91.83% for the key point recognition of the human body in the prone position. In addition, the above-mentioned sleeping posture recognition network, body part recognition network and key point recognition network all use lightweight networks, so the running speed can achieve the performance of real-time prediction.

[0169] In actual applications, after the server 100 completes the recognition of the sleeping posture, body parts and key points of the human body, it can send the recognized body parts and key points of the human body to the user terminal 200 for visualization, so that the user can use the intelligent operation of the mattress. Fig.24 The embodiment of the present invention further provides a mattress control method, which can be applied to the above-mentioned user terminal 200. Fig.24 The specific process shown is described in detail.

[0170] Step S201, receiving human body parts and key points sent by a server.

[0171] The human body parts and key points can be identified according to the aforementioned pressure sensor data identification method.

[0172] Step S202, displaying the human body parts and key points on the pressure heat map corresponding to the pressure sensor.

[0173] In this embodiment, the human body pressure data collected by the pressure sensor can be sent to the user terminal 200, and the user terminal 200 can visualize the received human body pressure data to obtain a corresponding pressure heat map. After receiving the human body parts and human body key points identified by the server 100, each human body part and human body key point is displayed on the pressure heat map.

[0174] Step S203, when a control instruction issued by the user based on the human body parts and key points displayed on the pressure heat map is received, a corresponding control operation is performed on the smart mattress according to the control instruction.

[0175] In this embodiment, the user terminal 200 can clearly know the pressure conditions of different parts and key points of the human body on the smart mattress 300 based on the pressure heat map showing the parts and key points of the human body. When there is a need to adjust the pressure, a control instruction can be issued on the user terminal 200 based on the parts and key points of the human body displayed on the pressure heat map. The user terminal 200 performs corresponding control operations on the smart mattress 300 according to the control instruction to achieve intelligent adjustment.

[0176] It can be seen that the mattress control method provided by the embodiment of the present invention can be based on Fig.25 The process shown is carried out, that is, after the human body pressure data collected by the pressure sensor is transmitted to the server, it is first identified by the sleeping posture recognition network to obtain the corresponding human sleeping posture, and then the human body pressure data is input into the human body sleeping posture corresponding part recognition network and the key point recognition network for recognition, and the different human body parts and human body key points under the human sleeping posture are identified. The server sends the human body parts and human body key points to the user terminal, and after visualization processing, the user can know the specific positions of different human body parts and key points on the smart mattress, which is convenient for the subsequent intelligent adjustment of the mattress pressure, effectively improving the intelligent service capabilities of the mattress and enhancing the user experience.

[0177] In order to execute the corresponding steps in the above embodiments and various possible methods, a method for implementing a pressure sensor data recognition device is provided below. Fig.26 , which is a functional module diagram of a pressure sensor data recognition device 400 provided in an embodiment of the present invention. It should be noted that the basic principle and technical effects of the pressure sensor data recognition device 400 provided in this embodiment are the same as those of the above-mentioned embodiments. For the sake of brief description, for parts not mentioned in this embodiment, reference can be made to the corresponding contents in the above-mentioned embodiments. The pressure sensor data recognition device 400 includes: a data acquisition module 410, a sleeping posture recognition module 420, and a body part and key point recognition module 430.

[0178] The data acquisition module 410 is used to acquire human body pressure data collected by a pressure sensor provided on the smart mattress.

[0179] It can be understood that the data acquisition module 410 can execute the above step S101.

[0180] The sleeping posture recognition module 420 is used to input human body pressure data into a pre-trained sleeping posture recognition network to obtain a human body sleeping posture.

[0181] It can be understood that the sleeping posture recognition module 420 can execute the above step S102.

[0182] The part and key point recognition module 430 is used to input the human body pressure data into the part recognition network and key point recognition network corresponding to the human body sleeping posture, and obtain the human body parts and key points under the human body sleeping posture; the part recognition network and key point recognition network are obtained through pre-training.

[0183] It can be understood that the part and key point identification module 430 can execute the above step S103.

[0184] Optionally, the sleeping posture recognition network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a fully connected layer and a first classification layer.

[0185] The sleeping posture recognition module 420 is specifically used to input human body pressure data into a pre-trained sleeping posture recognition network, use the first convolution layer to perform a first depth-separable convolution operation on the input human body pressure data, and input the matrix data after the first depth-separable convolution operation into the first pooling layer; use the first pooling layer to perform a first pooling operation on the matrix data after the first depth-separable convolution operation, and input the matrix data after the first pooling operation into the second convolution layer; use the second convolution layer to perform a second depth-separable convolution operation on the matrix data after the first pooling operation, and input the matrix data after the second depth-separable convolution operation into the second pooling layer; use the second pooling layer to perform a second pooling operation on the matrix data after the second depth-separable convolution operation, and input the matrix data after the second pooling operation into the fully connected layer; use the fully connected layer to perform a fully connected operation on the matrix data after the second pooling operation, and input the matrix data after the fully connected operation into the first classification layer; use the first classification layer to perform a classification operation on the matrix data after the fully connected operation to obtain a first classification result, and determine the human sleeping posture according to the first classification result.

[0186] Optionally, the human sleeping position includes supine, prone and lateral positions, and the part recognition network and key point recognition network corresponding to the supine position have the same network structure and different network parameters; the part recognition network and key point recognition network corresponding to the prone position have the same network structure and different network parameters; the part recognition network and key point recognition network corresponding to the lateral position have the same network structure and different network parameters.

[0187] Optionally, the supine position part recognition network and the supine position key point recognition network corresponding to the supine position both include a first convolutional neural network, a third pooling layer, a second convolutional neural network, a fourth pooling layer, a first upsampling layer, a third convolutional layer and a second classification layer.

[0188] The part and key point recognition module 430 can be used to input the human body pressure data into the supine part recognition network and the supine key point recognition network respectively when the human body is in the supine position, and use the first convolution neural network of the supine part recognition network and the supine key point recognition network to perform a first convolution operation on the input human body pressure data, and input the matrix data after the first convolution operation into the corresponding third pooling layer. Use the third pooling layer of the supine part recognition network and the supine key point recognition network to perform a third pooling operation on the received matrix data after the first convolution operation, and input the matrix data after the third pooling operation into the corresponding second convolution neural network. Use the second convolution neural network of the supine part recognition network and the supine key point recognition network to perform a second convolution operation on the received matrix data after the third pooling operation, and input the matrix data after the second convolution operation into the corresponding fourth pooling layer. Using the fourth pooling layer of each of the supine position part recognition network and the supine position key point recognition network, the fourth pooling operation is performed on the received matrix data after the second convolution operation, and the matrix data after the fourth pooling operation is input into the corresponding first upsampling layer. Using the first upsampling layer of each of the supine position part recognition network and the supine position key point recognition network, the first upsampling operation is performed on the received matrix data after the fourth pooling operation, and the matrix data after the first upsampling operation is input into the corresponding third convolution layer. Using the third convolution layer of each of the supine position part recognition network and the supine position key point recognition network, the third convolution operation is performed on the received matrix data after the first upsampling operation, and the matrix data after the third convolution operation is input into the corresponding second classification layer. Using the second classification layer of each of the supine position part recognition network and the supine position key point recognition network, the classification operation is performed on the received matrix data after the third convolution operation to obtain the second classification result, and according to the second classification results outputted by the supine position part recognition network and the supine position key point recognition network, the human body parts and human body key points in the supine position are obtained.

[0189] Optionally, the prone position part recognition network and the prone position key point recognition network corresponding to the prone position both include a third convolutional neural network, a fifth pooling layer, a fourth convolutional neural network, a sixth pooling layer, a second upsampling layer, a fourth convolutional layer and a third classification layer.

[0190] The part and key point recognition module 430 can be used to input the human body pressure data into the prone position part recognition network and the prone position key point recognition network respectively when the human body is in a prone position, and perform a fourth convolution operation on the input human body pressure data using the third convolution neural network of the prone position part recognition network and the prone position key point recognition network, and input the matrix data after the fourth convolution operation into the corresponding fifth pooling layer. Perform a fifth pooling operation on the received matrix data after the fourth convolution operation using the fifth pooling layer of the prone position part recognition network and the prone position key point recognition network, and input the matrix data after the fifth pooling operation into the corresponding fourth convolution neural network. Perform a fifth convolution operation on the received matrix data after the fifth pooling operation using the fourth convolution neural network of the prone position part recognition network and the prone position key point recognition network, and input the matrix data after the fifth convolution operation into the corresponding sixth pooling layer. The sixth pooling operation is performed on the received matrix data after the fifth convolution operation by using the respective sixth pooling layers of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the sixth pooling operation is input into the corresponding second upsampling layer. The second upsampling operation is performed on the received matrix data after the sixth pooling operation by using the respective second upsampling layers of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the second upsampling operation is input into the corresponding fourth convolution layer. The sixth convolution operation is performed on the received matrix data after the second upsampling operation by using the respective fourth convolution layers of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the sixth convolution operation is input into the corresponding third classification layer. The third classification layer of the prone position part recognition network and the prone position key point recognition network is used to perform a classification operation on the received matrix data after the sixth convolution operation to obtain a third classification result, and according to the third classification results outputted by the prone position part recognition network and the prone position key point recognition network, the human body parts and human body key points in the prone position are obtained.

[0191] Optionally, the lateral position part recognition network and the lateral position key point recognition network corresponding to the lateral position both include a fifth convolutional neural network, a seventh pooling layer, a sixth convolutional neural network, an eighth pooling layer, a third upsampling layer, a fifth convolutional layer and a fourth classification layer.

[0192] The part and key point recognition module 430 can be used to input the human body pressure data into the lateral position part recognition network and the lateral position key point recognition network respectively when the human body sleeping posture is the lateral position, and use the fifth convolutional neural network of the lateral position part recognition network and the lateral position key point recognition network to perform the seventh convolution operation on the input human body pressure data, and input the matrix data after the seventh convolution operation into the corresponding seventh pooling layer. Use the seventh pooling layer of the lateral position part recognition network and the lateral position key point recognition network to perform the seventh pooling operation on the received matrix data after the seventh convolution operation, and input the matrix data after the seventh pooling operation into the corresponding sixth convolutional neural network. Use the sixth convolutional neural network of the lateral position part recognition network and the lateral position key point recognition network to perform the eighth convolution operation on the received matrix data after the seventh pooling operation, and input the matrix data after the eighth convolution operation into the corresponding eighth pooling layer. The eighth pooling layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform the eighth pooling operation on the received matrix data after the eighth convolution operation, and the matrix data after the eighth pooling operation is input into the corresponding third upsampling layer. The third upsampling layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform the third upsampling operation on the received matrix data after the eighth pooling operation, and the matrix data after the third upsampling operation is input into the corresponding fifth convolution layer. The fifth convolution layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform the ninth convolution operation on the received matrix data after the third upsampling operation, and the matrix data after the ninth convolution operation is input into the corresponding fourth classification layer. The fourth classification layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform the classification operation on the received matrix data after the ninth convolution operation, and the fourth classification result is obtained, and the human body parts and human body key points in the lateral position are obtained according to the fourth classification results output by the lateral position part recognition network and the lateral position key point recognition network.

[0193] It can be seen that the pressure sensor data recognition device provided by the embodiment of the present invention, through pre-training the sleeping posture recognition network and the part recognition network and key point recognition network under different sleeping postures, when recognizing the pressure sensor data, the data acquisition module obtains the human body pressure data collected by the pressure sensor set on the intelligent mattress, the sleeping posture recognition module inputs the human body pressure data into the pre-trained sleeping posture recognition network to obtain the human body sleeping posture, and the part and key point recognition module inputs the human body pressure data into the part recognition network and key point recognition network corresponding to the human body sleeping posture respectively, to obtain the human body parts and human body key points under the human body sleeping posture, so as to facilitate knowing the specific positions of different human body parts and human body key points on the mattress, and after visualization, it can also be used for subsequent mattress intelligent operation. Since different part recognition networks and key point recognition networks are used for part recognition and key point recognition for different sleeping postures, accurate recognition of human body parts and human body key points can be achieved in different sleeping posture scenarios, thereby providing a strong guarantee for improving the intelligent service capabilities of the mattress and improving the user experience.

[0194] Please refer to Fig. 27 , is a block diagram of an electronic device that can implement the above-mentioned server 100 and user terminal 200 provided in an embodiment of the present invention. The electronic device includes a memory 810, a processor 820 and a communication module 830. The memory 810, the processor 820 and the communication module 830 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0195] The memory 810 is used to store programs or data. The memory 810 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0196] The processor 820 is used to read / write the data or program stored in the memory 810 and execute the corresponding functions. For example, when the computer program stored in the memory 810 is executed by the processor 820, the pressure sensor data recognition method or mattress control method disclosed in the above embodiments can be implemented. The communication module 830 is used to establish a communication connection between the electronic device and other communication terminals through the network, and to send and receive data through the network.

[0197] It should be understood that Fig. 27 The structure shown is only a schematic diagram of the server structure. The server may also include Fig. 27 More or fewer components as shown, or with Fig. 27 Different configurations are shown. Fig. 27 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0198] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by the processor 820, the pressure sensor data recognition method or the mattress control method disclosed in the above embodiments is implemented.

[0199] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0200] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0201] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0202] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying pressure sensor data, characterized in that: The method comprises: Obtaining human body pressure data collected by a pressure sensor provided on a smart mattress; Inputting the human body pressure data into a pre-trained sleeping posture recognition network to obtain the human body sleeping posture; The human body pressure data is respectively input into a part recognition network and a key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained by pre-training; The human sleeping posture includes a supine position, and the part recognition network and the key point recognition network corresponding to the supine position have the same network structure and different network parameters; the supine position part recognition network and the supine position key point recognition network corresponding to the supine position both include a first convolutional neural network, a third pooling layer, a second convolutional neural network, a fourth pooling layer, a first upsampling layer, a third convolutional layer, and a second classification layer; The human body pressure data is respectively input into the body part recognition network and the key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and human body key points under the human body sleeping posture, including: When the sleeping posture of the human body is a supine position, the human body pressure data is respectively input into the supine position part recognition network and the supine position key point recognition network, and a first convolution operation is performed on the input human body pressure data using the first convolution neural network of each of the supine position part recognition network and the supine position key point recognition network, and the matrix data after the first convolution operation is input into the corresponding third pooling layer; Using the third pooling layer of each of the supine position part recognition network and the supine position key point recognition network, a third pooling operation is performed on the received matrix data after the first convolution operation, and the matrix data after the third pooling operation is input into the corresponding second convolutional neural network; Using the second convolutional neural network of each of the supine position part recognition network and the supine position key point recognition network, a second convolution operation is performed on the received matrix data after the third pooling operation, and the matrix data after the second convolution operation is input into the corresponding fourth pooling layer; Using the fourth pooling layer of each of the supine position part recognition network and the supine position key point recognition network, perform a fourth pooling operation on the received matrix data after the second convolution operation, and input the matrix data after the fourth pooling operation into the corresponding first upsampling layer; Using the first upsampling layers of the supine position part recognition network and the supine position key point recognition network, respectively, a first upsampling operation is performed on the received matrix data after the fourth pooling operation, and the matrix data after the first upsampling operation is input into the corresponding third convolutional layer; Using the third convolutional layers of the supine position part recognition network and the supine position key point recognition network, respectively, a third convolutional operation is performed on the received matrix data after the first upsampling operation, and the matrix data after the third convolutional operation is input into the corresponding second classification layer; The second classification layer of the supine position part recognition network and the supine position key point recognition network is used to perform a classification operation on the received matrix data after the third convolution operation to obtain a second classification result, and the human body parts and key points in the supine position are obtained according to the second classification results output by the supine position part recognition network and the supine position key point recognition network.

2. The method according to claim 1, characterized in that The sleeping posture recognition network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a fully connected layer and a first classification layer; The step of inputting the human body pressure data into a pre-trained sleeping posture recognition network to obtain the human body sleeping posture comprises: Inputting the human body pressure data into a pre-trained sleeping posture recognition network, performing a first depthwise separable convolution operation on the input human body pressure data using the first convolution layer, and inputting the matrix data after the first depthwise separable convolution operation into the first pooling layer; Using the first pooling layer to perform a first pooling operation on the matrix data after the first depthwise separable convolution operation, and inputting the matrix data after the first pooling operation into the second convolution layer; Using the second convolution layer to perform a second depth-separable convolution operation on the matrix data after the first pooling operation, and inputting the matrix data after the second depth-separable convolution operation into the second pooling layer; Using the second pooling layer to perform a second pooling operation on the matrix data after the second depthwise separable convolution operation, and inputting the matrix data after the second pooling operation into the fully connected layer; Using the fully connected layer to perform a fully connected operation on the matrix data after the second pooling operation, and inputting the matrix data after the fully connected operation into the first classification layer; The first classification layer is used to perform a classification operation on the matrix data after the full connection operation to obtain a first classification result, and the sleeping posture of the human body is determined according to the first classification result.

3. The method according to claim 1, characterized in that The human sleeping positions also include prone position and lateral position; the part recognition network and key point recognition network corresponding to the prone position have the same network structure and different network parameters; the part recognition network and key point recognition network corresponding to the lateral position have the same network structure and different network parameters.

4. The method according to claim 3, characterized in that The prone position part recognition network and the prone position key point recognition network corresponding to the prone position both include a third convolutional neural network, a fifth pooling layer, a fourth convolutional neural network, a sixth pooling layer, a second upsampling layer, a fourth convolutional layer and a third classification layer; The human body pressure data is respectively input into the body part recognition network and the key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and key points under the human body sleeping posture; The part recognition network and the key point recognition network are obtained by pre-training, including: When the sleeping posture of the human body is a prone position, the human body pressure data is respectively input into the prone position part recognition network and the prone position key point recognition network, and a fourth convolution operation is performed on the input human body pressure data using the third convolutional neural network of each of the prone position part recognition network and the prone position key point recognition network, and the matrix data after the fourth convolution operation is input into the corresponding fifth pooling layer; Using the fifth pooling layer of each of the prone position part recognition network and the prone position key point recognition network, perform a fifth pooling operation on the received matrix data after the fourth convolution operation, and input the matrix data after the fifth pooling operation into the corresponding fourth convolutional neural network; Using the fourth convolutional neural network of each of the prone position part recognition network and the prone position key point recognition network, perform a fifth convolution operation on the received matrix data after the fifth pooling operation, and input the matrix data after the fifth convolution operation into the corresponding sixth pooling layer; Using the sixth pooling layer of each of the prone position part recognition network and the prone position key point recognition network, perform a sixth pooling operation on the received matrix data after the fifth convolution operation, and input the matrix data after the sixth pooling operation into the corresponding second upsampling layer; Using the second upsampling layers of the prone position part recognition network and the prone position key point recognition network, respectively, a second upsampling operation is performed on the received matrix data after the sixth pooling operation, and the matrix data after the second upsampling operation is input into the corresponding fourth convolutional layer; Using the fourth convolutional layers of the prone position part recognition network and the prone position key point recognition network, respectively, a sixth convolutional operation is performed on the received matrix data after the second upsampling operation, and the matrix data after the sixth convolutional operation is input into the corresponding third classification layer; The third classification layer of the prone position part recognition network and the prone position key point recognition network is used to perform a classification operation on the received matrix data after the sixth convolution operation to obtain a third classification result, and the human body parts and key points in the prone position are obtained according to the third classification results output by the prone position part recognition network and the prone position key point recognition network.

5. The method according to claim 3, characterized in that: The lateral position part recognition network and the lateral position key point recognition network corresponding to the lateral position both include a fifth convolutional neural network, a seventh pooling layer, a sixth convolutional neural network, an eighth pooling layer, a third upsampling layer, a fifth convolutional layer and a fourth classification layer; The human body pressure data is respectively input into the body part recognition network and the key point recognition network corresponding to the human body sleeping posture to obtain the human body parts and key points under the human body sleeping posture; The part recognition network and the key point recognition network are obtained by pre-training, including: When the sleeping posture of the human body is a lateral position, the human body pressure data is respectively input into the lateral position part recognition network and the lateral position key point recognition network, and a seventh convolution operation is performed on the input human body pressure data using the fifth convolutional neural network of each of the lateral position part recognition network and the lateral position key point recognition network, and the matrix data after the seventh convolution operation is input into the corresponding seventh pooling layer; Using the seventh pooling layer of each of the lateral position part recognition network and the lateral position key point recognition network, perform the seventh pooling operation on the received matrix data after the seventh convolution operation, and input the matrix data after the seventh pooling operation into the corresponding sixth convolutional neural network; Using the sixth convolutional neural network of each of the lateral position part recognition network and the lateral position key point recognition network, an eighth convolution operation is performed on the received matrix data after the seventh pooling operation, and the matrix data after the eighth convolution operation is input into the corresponding eighth pooling layer; Using the eighth pooling layer of the lateral position part recognition network and the lateral position key point recognition network, the eighth pooling operation is performed on the received matrix data after the eighth convolution operation, and the matrix data after the eighth pooling operation is input into the corresponding third upsampling layer; Using the third upsampling layers of the lateral position part recognition network and the lateral position key point recognition network, respectively, a third upsampling operation is performed on the received matrix data after the eighth pooling operation, and the matrix data after the third upsampling operation is input into the corresponding fifth convolutional layer; Using the fifth convolutional layer of each of the lateral position part recognition network and the lateral position key point recognition network, a ninth convolutional operation is performed on the received matrix data after the third upsampling operation, and the matrix data after the ninth convolutional operation is input into the corresponding fourth classification layer; The fourth classification layer of the lateral position part recognition network and the lateral position key point recognition network is used to perform a classification operation on the received matrix data after the ninth convolution operation to obtain a fourth classification result, and the human body parts and key points in the lateral position are obtained according to the fourth classification results output by the lateral position part recognition network and the lateral position key point recognition network.

6. A mattress control method, characterized in that: The method comprises: Receiving human body parts and human body key points sent by a server; the human body parts and the human body key points are identified by the pressure sensor data identification method according to any one of claims 1-5; Displaying the human body parts and the human body key points on a pressure heat map corresponding to the pressure sensor; When a control instruction issued by the user based on the human body parts and the human body key points displayed on the pressure heat map is received, a corresponding control operation is performed on the smart mattress according to the control instruction.

7. A pressure sensor data recognition device, characterized in that: The device comprises: A data acquisition module, used to acquire human body pressure data collected by a pressure sensor provided on the smart mattress; A sleeping posture recognition module, used for inputting the human body pressure data into a pre-trained sleeping posture recognition network to obtain a human body sleeping posture; A part and key point recognition module, used to input the human body pressure data into a part recognition network and a key point recognition network corresponding to the human body sleeping posture, respectively, to obtain the human body parts and key points under the human body sleeping posture; the part recognition network and the key point recognition network are obtained by pre-training; The human sleeping posture includes a supine position, and the network structures of the part recognition network and the key point recognition network corresponding to the supine position are the same and the network parameters are different; the supine position part recognition network and the supine position key point recognition network corresponding to the supine position both include a first convolutional neural network, a third pooling layer, a second convolutional neural network, a fourth pooling layer, a first upsampling layer, a third convolutional layer, and a second classification layer; the part and key point recognition module is used to input the human body pressure data into the supine position part recognition network and the supine position key point recognition network respectively when the human sleeping posture is a supine position, and use the supine position part recognition network and the key point recognition network to respectively classify the human body pressure data. The first convolutional neural network of each of the supine key point recognition networks performs a first convolution operation on the input human body pressure data, and inputs the matrix data after the first convolution operation into the corresponding third pooling layer; the third pooling layer of each of the supine position part recognition network and the supine key point recognition network performs a third pooling operation on the received matrix data after the first convolution operation, and inputs the matrix data after the third pooling operation into the corresponding second convolutional neural network; the second convolutional neural network of each of the supine position part recognition network and the supine key point recognition network performs a third pooling operation on the received matrix data after the third pooling operation, and inputs the matrix data after the third pooling operation into the corresponding second convolutional neural network; The second convolution operation is performed on the received matrix data after the second convolution operation, and the matrix data after the second convolution operation is input into the corresponding fourth pooling layer; the fourth pooling layer of the supine position part recognition network and the supine position key point recognition network is used to perform the fourth pooling operation on the received matrix data after the second convolution operation, and the matrix data after the fourth pooling operation is input into the corresponding first upsampling layer; the first upsampling layer of the supine position part recognition network and the supine position key point recognition network is used to perform the first upsampling operation on the received matrix data after the fourth pooling operation, and the matrix data after the first upsampling operation is input into the corresponding third convolution layer; The third convolution operation is performed on the received matrix data after the first upsampling operation using the third convolution layer of the supine position part recognition network and the supine position key point recognition network, and the matrix data after the third convolution operation is input into the corresponding second classification layer; the second classification layer of the supine position part recognition network and the supine position key point recognition network is used to perform a classification operation on the received matrix data after the third convolution operation to obtain a second classification result, and the human body parts and key points in the supine position are obtained according to the second classification results outputted by the supine position part recognition network and the supine position key point recognition network.

8. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the method for identifying pressure sensor data as described in any one of claims 1 to 5 are implemented, or the mattress control method as described in claim 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for identifying pressure sensor data according to any one of claims 1 to 5 are implemented, or the mattress control method according to claim 6 is implemented.

Images