Posture recognition method and device, electronic equipment and storage medium

Abstract

The application provides a posture recognition method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: obtaining target data related to the posture of a target object in a detection area based on positioning of the target object; performing time-frequency domain feature extraction on the target data to obtain first target features and second target features; the first target features are frequency domain expressions of the posture of the target object in the detection area, and the second target features are time domain expressions of the posture of the target object in the detection area; and the posture of the target object is recognized according to the first target features and the second target features to obtain a posture recognition result. The application solves the problem of low accuracy of posture recognition in related technologies.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a posture recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of computer technology, posture recognition is increasingly being used to assist users in various information analyses. For example, sleep posture recognition can help users analyze sleep state and sleep stages.

[0003] Currently, posture recognition is mainly achieved based on various types of detection devices, such as cameras, radar devices, and sensors. Regardless of the type of detection device, the accuracy of posture recognition will be affected. For example, pressure sensors are easily affected by environmental factors, leading to detection failure and consequently impacting the accuracy of posture recognition.

[0004] Therefore, how to improve the accuracy of pose recognition remains to be solved. Summary of the Invention

[0005] This application provides a posture recognition method, apparatus, electronic device, and storage medium, which can solve the problem of low accuracy in posture recognition in related technologies. The technical solution is as follows:

[0006] According to one aspect of this application, a pose recognition method includes: obtaining target data related to the pose of the target object in a detection region based on the localization of the target object; extracting time-frequency domain features from the target data to obtain a first target feature and a second target feature; wherein the first target feature is a frequency domain representation of the pose of the target object in the detection region, and the second target feature is a time domain representation of the pose of the target object in the detection region; and recognizing the pose of the target object based on the first target feature and the second target feature to obtain a pose recognition result.

[0007] According to one aspect of this application, a posture recognition device includes: a data acquisition module, configured to obtain target data related to the posture of the target object in a detection area based on the localization of the target object; a feature extraction module, configured to perform time-frequency domain feature extraction on the target data to obtain a first target feature and a second target feature; wherein the first target feature is a frequency domain representation of the posture of the target object in the detection area, and the second target feature is a time domain representation of the posture of the target object in the detection area; and a posture recognition module, configured to recognize the posture of the target object based on the first target feature and the second target feature to obtain a posture recognition result.

[0008] In an exemplary embodiment, the feature extraction module includes: a frequency domain transformation unit, configured to perform frequency domain transformation on the target data to obtain multiple frequency domain signals of the target data in the frequency domain, each frequency domain signal corresponding to a frequency point in the frequency domain; and a time domain transformation unit, configured to perform frequency domain feature extraction on the frequency domain signals corresponding to each frequency point in a set frequency band to obtain a first target feature; and to perform time domain transformation on the frequency domain signals corresponding to each frequency point in the set frequency band to obtain a second target feature.

[0009] In an exemplary embodiment, the pose recognition module includes: a feature fusion unit, configured to fuse the first target feature and the second target feature to obtain a joint target feature; and a pose prediction unit, configured to input the joint target feature into a pose recognition model to predict the pose category and obtain the pose recognition result.

[0010] In an exemplary embodiment, the apparatus further includes: a model training module, configured to train a base model based on joint features of samples in a training set and their corresponding sample labels to obtain the pose recognition model, wherein the sample labels indicate the pose category to which the corresponding joint feature belongs; the model training module includes: a sample input unit, configured to input a current joint feature of a sample in the training set into the base model, perform pose recognition on the joint feature of the sample through the base model, and obtain a pose prediction result; a loss calculation unit, configured to obtain a corresponding loss value based on the difference between the pose prediction result and the sample label corresponding to the current joint feature of the sample; and a convergence unit, configured to adjust the gradient of each model parameter in the base model according to the loss value and continue training until the gradient of each model parameter satisfies a set convergence condition, thereby training the pose recognition model from the base model.

[0011] In an exemplary embodiment, the apparatus further includes: a signal acquisition module, configured to acquire echo signals matching the number of echo signals of the set window length from a plurality of echo signals based on a sliding window of a set window length; the plurality of echo signals are multiple radar signals emitted by the detection device during the positioning process, which are reflected by the target object; a spectrum analysis module, configured to perform spectrum analysis on the acquired echo signals to obtain multiple distance data, the distance data being used to indicate the radial distance between the target object and the detection device in the detection area; and a stationary detection module, configured to determine whether the target object in the detection area is stationary based on the multiple distance data; if not, to control the sliding window to continue sliding among the multiple echo signals.

[0012] In an exemplary embodiment, the posture is a sleeping posture, and the posture recognition result is used to indicate the sleeping posture of the target object in the detection area.

[0013] In one exemplary embodiment, the device further includes: a first automatic control module, configured to notify the device to perform an action corresponding to the sleeping posture indicated by the posture recognition result.

[0014] In one exemplary embodiment, the device further includes: a vital sign detection module, configured to perform vital sign detection on the target object based on the sleeping posture indicated by the posture recognition result, and obtain vital sign data of the target object, the vital sign data being used to indicate the vital signs of the target object; a sleep staging module, configured to determine a sleep staging result based on the vital signs of the target object indicated by the vital sign data and the confidence level configured for the vital signs of the target object in the sleeping posture; and a second automatic control module, configured to notify the device to perform an action corresponding to the sleep stage indicated by the sleep staging result.

[0015] In one exemplary embodiment, the apparatus further includes a configuration module, configured to configure the credibility of the vital signs of the target object in the sleeping position based on the sleeping position indicated by the posture recognition result, the first target feature, and / or the second target feature.

[0016] According to one aspect of this application, an electronic device includes: at least one processor, at least one memory, and at least one communication bus, wherein a computer program is stored in the memory, and the processor reads the computer program from the memory via the communication bus; when the computer program is executed by the processor, it implements the attitude recognition method as described above.

[0017] According to one aspect of this application, a storage medium stores a computer program thereon, which, when executed by a processor, implements the posture recognition method as described above.

[0018] According to one aspect of this application, a computer program product includes a computer program stored in a storage medium, a processor of a computer device reads the computer program from the storage medium, and the processor executes the computer program, causing the computer device to implement the posture recognition method as described above when executed.

[0019] The beneficial effects of the technical solution provided in this application are:

[0020] In the above technical solution, after obtaining target data related to the pose of the target object in the detection area, time-frequency domain features are extracted from the target data to obtain the first target feature and the second target feature, which are used as the basis for recognizing the pose of the target object to obtain the pose recognition result. The first target feature is the frequency domain expression of the pose of the target object in the detection area, and the second target feature is the time domain expression of the pose of the target object in the detection area. That is to say, by using the first target feature and the second target feature, i.e. the different expressions of the pose of the target object in the detection area in the time-frequency domain, the pose of the target object can be predicted comprehensively, and a high recognition accuracy can be obtained, thereby solving the problem of low accuracy of pose recognition in related technologies. Attached Figure Description

[0021] To more clearly illustrate the technical solutions provided in this application, the accompanying drawings used in the description of the various embodiments of this application will be briefly introduced below.

[0022] Figure 1 This is a schematic diagram of the implementation environment according to the embodiments of this application;

[0023] Figure 2 This is a flowchart illustrating a pose recognition method according to an exemplary embodiment;

[0024] Figure 3 yes Figure 2 A flowchart of step 370 in one embodiment corresponds to the following example;

[0025] Figure 4 This is a flowchart illustrating another pose recognition method according to an exemplary embodiment;

[0026] Figure 5 yes Figure 4 A flowchart of step 410 in one embodiment corresponds to the following example;

[0027] Figure 6 This is a schematic diagram illustrating the movement of a sliding window according to an exemplary embodiment;

[0028] Figure 7 This is a flowchart illustrating another pose recognition method according to an exemplary embodiment;

[0029] Figure 8 yes Figure 3 A flowchart of step 373 in one embodiment corresponds to the following example;

[0030] Figure 9 This is a structural block diagram of an attitude recognition device according to an exemplary embodiment;

[0031] Figure 10This is a hardware structure diagram of a server according to an exemplary embodiment;

[0032] Figure 11 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0034] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0035] The following is an introduction and explanation of several terms used in this application:

[0036] CFAR stands for Constant False-Alarm Rate.

[0037] FFT stands for Fast Fourier Transform.

[0038] IFFT stands for Inverse Fast Fourier Transform.

[0039] SNR stands for Signal-Noise Ratio.

[0040] PCA stands for Principal Component Analysis.

[0041] SVM stands for Support Vector Machine.

[0042] CNN stands for Convolutional Neural Networks.

[0043] LSTM stands for Long Short-term Memory.

[0044] GRU stands for Gated Recurrent Unit.

[0045] Currently, users can perform various information analyses using data obtained from different types of detection devices, such as analyzing their own sleep patterns. Specifically, piezoelectric sensors are used to collect fluctuation signals in the user's chest cavity to detect vital signs such as heart rate, respiratory rate, and body movement, and then these vital signs are used to analyze the user's sleep state, sleep stage, and so on.

[0046] Generally, the accuracy of detecting a user's vital signs varies depending on their sleeping position. For example, if the radar device detects vertically downwards, the chest rise and fall caused by breathing and heartbeat is greater when lying flat, resulting in a more accurate detection of respiratory and heart rates. When lying on one's side, the chest rise and fall caused by breathing and heartbeat is smaller within the radar's radial direction compared to lying flat, thus weakening breathing and heartbeat and affecting the accuracy of respiratory and heart rate readings. Lying on the left side results in the greatest radial distance between the heart and the radar device, with the weakest heartbeat, and therefore the lowest accuracy for heart rate readings.

[0047] Therefore, judgments of sleep status and analysis of sleep stages based on the aforementioned vital signs may be inaccurate. To more accurately analyze a user's sleep status and sleep stages, sleep posture recognition should be performed during vital sign detection, thus providing effective data for sleep status judgment and sleep stage analysis.

[0048] However, as mentioned earlier, the accuracy of sleep posture recognition is not high regardless of whether it is based on cameras, radar devices, or sensors. Furthermore, cameras raise privacy concerns, radar devices require contour detection during sleep posture recognition, leading to excessive costs, while sensors are easily affected by environmental factors or prone to damage, affecting the effectiveness of detection and further reducing the accuracy of sleep posture recognition.

[0049] As can be seen from the above, taking sleeping posture as an example, the relevant technologies still have the drawback of low accuracy in recognizing sleeping posture. Similarly, posture recognition based on various types of detection devices also has the limitation of low accuracy.

[0050] Therefore, the posture recognition method provided in this application can effectively improve the accuracy of posture recognition. Accordingly, the posture recognition method is applicable to posture recognition devices, which can be deployed on electronic devices. For example, the electronic device can be a computer device configured with a von Neumann architecture, including but not limited to desktop computers, laptops, tablets, servers, etc. Alternatively, the electronic device can also be a detection device, such as a human body sensor.

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0052] Figure 1 This is a schematic diagram of an implementation environment for a gesture recognition method. The implementation environment includes a user terminal 110, a smart device 130, a gateway 150, a server 170, and a router 190.

[0053] Specifically, user terminal 110, which can also be considered as user terminal or terminal, can deploy (or install) the client associated with smart device 130. This user terminal 110 can be an electronic device such as a smartphone, tablet, laptop, desktop computer, smart control panel, or other device with display and control functions, without limitation here.

[0054] The client, associated with the smart device 130, is essentially where the user registers an account and configures the smart device 130. For example, the configuration includes adding a device identifier to the smart device 130, so that when the client runs on the user terminal 110, it can provide the user with functions such as device display and device control of the smart device 130. This client can be in the form of an application or a web page. Correspondingly, the interface for displaying the device on the client can be in the form of a program window or a web page, and there is no limitation here.

[0055] Smart device 130 is deployed in gateway 150 and communicates with gateway 150 through its own configured communication module, thereby being controlled by gateway 150. It should be understood that smart device 130 generally refers to one of multiple smart devices 130. This application embodiment only uses smart device 130 as an example; that is, this application embodiment does not limit the number or type of smart devices deployed in gateway 150. In one application scenario, smart device 130 is deployed in gateway 150 by accessing it through a local area network. The process of smart device 130 accessing gateway 150 through a local area network includes: gateway 150 first establishes a local area network, and smart device 130 joins the local area network established by gateway 150 by connecting to it. This local area network includes, but is not limited to, ZIGBEE or Bluetooth. Among them, the smart device 130 can be a smart printer, smart fax machine, smart camera, smart air conditioner, smart door lock, smart light, or electronic devices such as human body sensor, door and window sensor, temperature and humidity sensor, water immersion sensor, natural gas alarm, smoke alarm, wall switch, wall socket, wireless switch, wireless wall sticker switch, cube controller, curtain motor, etc., equipped with a communication module.

[0056] The interaction between user terminal 110 and smart device 130 can be achieved through a local area network (LAN) or a wide area network (WAN). In one application scenario, user terminal 110 establishes a wired or wireless communication connection with gateway 150 via router 190, such as Wi-Fi, allowing user terminal 110 and gateway 150 to be deployed on the same LAN, thus enabling user terminal 110 to interact with smart device 130 via the LAN path. In another application scenario, user terminal 110 establishes a wired or wireless communication connection with gateway 150 via server 170, such as 2G, 3G, 4G, 5G, or Wi-Fi, allowing user terminal 110 and gateway 150 to be deployed on the same WAN, thus enabling user terminal 110 to interact with smart device 130 via the WAN path.

[0057] The server-side 170 can also be considered as the cloud, cloud platform, platform side, server side, etc. This server-side 170 can be a single server, a server cluster consisting of multiple servers, or a cloud computing center consisting of multiple servers, in order to better provide backend services to a massive number of user terminals 110. For example, backend services include gesture recognition services.

[0058] Taking the pose recognition service provided by server 170 as an example, the pose recognition process is explained as follows:

[0059] For the smart device 130, by locating the target object, it can obtain target data related to the pose of the target object in the detection area, and then send the target data to the server 170 to request the server 170 to provide pose recognition services.

[0060] As the smart device 130 interacts with the server 170 via a local area network or a wide area network, the server 170 can receive the target data sent by the smart device 130, and then extract time-frequency domain features from the target data to obtain the first target feature and the second target feature. The first target feature and the second target feature are combined to identify the posture of the target object, resulting in a posture recognition result with a high recognition accuracy.

[0061] Based on the posture recognition results with high accuracy, it can better assist users in various information analyses, such as the analysis of sleep state and sleep stage based on sleeping posture recognition.

[0062] Please see Figure 2 This application provides a posture recognition method, which is applicable to electronic devices, specifically, such devices. Figure 1 The server-side 170 shown in the implementation environment can also be... Figure 1 The illustration shows a smart device 130 in the implementation environment, such as a human body sensor.

[0063] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0064] like Figure 2 As shown, the method may include the following steps:

[0065] Step 310: Based on the localization of the target object, obtain target data related to the pose of the target object in the detection area.

[0066] First, it should be noted that the target object refers to any object that can appear within the detection area, specifically an object with posture, such as a person, robot, or animal. The detection area refers to the region where the detection device can emit effective detection signals towards the target object.

[0067] Secondly, the location of the target object is achieved by a detection device. In one possible implementation, the detection device is installed directly above the target object. For example, if the target object is a person, the detection device could be installed directly above the person's chest, such as on the ceiling.

[0068] Taking a human body sensor equipped with millimeter-wave radar as an example, the millimeter-wave radar in the human body sensor is equipped with an antenna array, which includes a transmitting antenna and a receiving antenna. The human body sensor can transmit millimeter-wave signals to the target object through the transmitting antenna and receive the echo signal formed by the millimeter-wave signal transmitted by the target object through the receiving antenna. After relevant processing, the target object can be located.

[0069] By locating the target object, the detection device can obtain the target object's position data within the detection area. This position data indicates the target object's location within the detection area. For a target object maintaining its orientation, in one possible implementation, the position data includes, but is not limited to: the target object's orientation within the detection area and the radial distance between the target object and the detection device within the detection area.

[0070] Therefore, the target data, which is related to the attitude of the target object in the detection area, is extracted from the position data of the target object in the detection area. In one possible implementation, the target data is described by the azimuth angle of the receiving antenna, which can be a horizontal angle or a pitch angle, used to indicate the attitude of the target object in the detection area.

[0071] For example, the location data of the target object in the detection area can be represented as Where, ρ * θ represents the radial distance between the target object and the detection device within the detection area. * This represents the horizontal angle used to indicate the orientation of the target object within the detection area. This represents the pitch angle of the target object within the detection area, used to indicate its attitude. Therefore, target data can be extracted from this position data, represented as... In one possible implementation, if the posture is a sleeping position, the target data specifically indicates the horizontal and pitch angles of the target object's sleeping posture within the detection area.

[0072] It should be noted that the inventors recognized that if the target object is not present in the detection area, but the detection device still attempts to locate it, the power consumption of the detection device will be high, requiring frequent battery replacements and affecting its lifespan, which is detrimental to reducing the hardware cost of posture recognition. Therefore, in one possible implementation, the presence of the target object in the detection area is first determined before locating it. For example, the detection device uses a two-dimensional CFAR (Constant False-Alarm Rate) algorithm to detect the target object in the detection area. If the detection device detects a target object in the detection area, it then locates that target object. Of course, in other embodiments, it is not limited to the same detection device performing target object detection and location; target object detection and location can also be implemented using different detection devices, which is not intended to constitute a specific limitation.

[0073] Step 350: Extract time-frequency domain features from the target data to obtain the first target feature and the second target feature.

[0074] The first target feature refers to the feature of the frequency domain signal corresponding to the frequency point in the set frequency band, i.e., the frequency domain feature. It can also be considered as the frequency domain expression of the attitude of the target object in the detection area. The second target feature refers to the feature of the time domain signal after the frequency domain signal is transformed by the time domain, i.e., the time domain feature. It can also be considered as the time domain expression of the attitude of the target object in the detection area.

[0075] It should be understood that the pose of the target object in the detection area will be different in the time and frequency domain after time-frequency domain transformation. That is, the first target feature is different and the second target feature is also different. In other words, the first target feature accurately describes the pose of the target object in the detection area from the frequency domain perspective, while the second target feature accurately describes the pose of the target object in the detection area from the time domain perspective.

[0076] In one possible implementation, the time-frequency domain feature extraction process specifically refers to: performing a frequency domain transformation on the target data to obtain multiple frequency domain signals of the target data in the frequency domain; extracting frequency domain features from the frequency domain signals corresponding to each frequency point in the set frequency band to obtain the first target feature; and performing a time domain transformation on the frequency domain signals corresponding to each frequency point in the set frequency band to obtain the second target feature. Each frequency domain signal corresponds to a specific frequency point in the frequency domain.

[0077] The frequency domain is a coordinate system that describes the frequency characteristics of a signal, and can describe the signal quantity within a specified frequency band. A frequency point can be a specific frequency, and frequency can be a quantity describing the frequency of an object's vibration.

[0078] In one possible implementation, frequency domain transformation refers to Fast Fourier Transform (FFT). Specifically, assuming the target data is represented as x(n), performing frequency domain transformations such as FFT on the target data yields a frequency domain signal x(k), where n represents the number of vectors in the target data; and k represents the frequency point number of the frequency domain signal, which can also be considered as the k-th frequency point in the frequency domain corresponding to the frequency domain signal.

[0079] In one possible implementation, windowing is applied to the target data so that the frequency domain transformation is based on the processed target data. This prevents spectral leakage caused by waveform discontinuities in the target data, thereby improving recognition accuracy. The windowing process can be either a Hamming window or a rectangular window; no specific limitation is made here.

[0080] In one possible implementation, the frequency bands include a respiratory frequency band and a heart rate frequency band. Specifically, the respiratory frequency band refers to 0.1Hz to 0.6Hz; the heart rate frequency band refers to 0.8Hz to 4Hz.

[0081] In one possible implementation, the time-domain transformation refers to the inverse fast Fourier transform. Specifically, assuming the frequency domain signal x(k) corresponding to the k-th frequency point in the frequency domain belongs to a set frequency band, then time-domain transformations such as the inverse fast Fourier transform are performed on the frequency domain signals corresponding to each frequency point in the set frequency band to obtain the time-domain signal corresponding to the set frequency band. For example, if the set frequency band is the breathing frequency band, the corresponding time-domain signal is the breathing signal x. b (n), or, setting the frequency band as the heartbeat frequency band, the corresponding time-domain signal is the heartbeat signal x. h (n), where n represents the number of vectors in the time-domain signal.

[0082] Taking sleeping posture as an example, the features of the first target and the features of the second target are illustrated below:

[0083] In one possible implementation, the first target feature includes, but is not limited to, frequency domain features such as: respiratory frequency band energy, heartbeat frequency band energy, energy ratio between respiratory and heartbeat frequency bands, peak signal-to-noise ratio of respiratory frequency band, peak signal-to-noise ratio of heartbeat frequency band, peak signal-to-noise ratio ratio between respiratory and heartbeat frequency bands, ratio between different peaks in the heartbeat frequency band, and ratio between frequency points corresponding to different peaks in the heartbeat frequency band.

[0084] The frequency domain characteristics are as follows:

[0085] 1. Energy in the respiratory frequency band: Where, k b,s k represents the starting frequency of the breathing band. b,e The frequency point representing the end of the breathing frequency band is denoted by x(k), which represents the frequency domain signal corresponding to the k-th frequency point in the breathing frequency band.

[0086] 2. Heart rate frequency energy: Where, k h,s k represents the starting frequency of the heartbeat band. h,e Let x(k) represent the termination frequency of the heartbeat frequency band, and let x(k) represent the frequency domain signal corresponding to the k-th frequency point in the heartbeat frequency band.

[0087] 3. Energy ratio between respiratory rate and heart rate: R bh,E =E b / E h Among them, E b E represents the energy of the breathing frequency band. h Indicates the energy of the heartbeat frequency band;

[0088] 4. Peak signal-to-noise ratio (SNR) in the respiratory frequency band: SNT b =|X(k) b，1 )| / M;k b,1 X(k) represents the frequency point where the maximum peak value found through peak search is located in the breathing frequency band. b,1 ) represents frequency point k b,1 The corresponding frequency domain signal, where M represents the noise floor mean, is based on the termination frequency k of the breathing band. b,e The subsequent frequency point k b,e +1 to the maximum frequency point k in the frequency domain e The frequency domain signal x(k) corresponding to each frequency point k can be obtained by calculating the average value of the peak values ​​|X(k)| of these frequency domain signals using the following formula:

[0089] 5. Peak Signal-to-Noise Ratio (SNR) in the Heartbeat Band h =|X(k) h，1 )| / M;k h,1 X(k) represents the frequency point where the maximum peak value found through peak search is located in the heartbeat frequency band. h,1 ) represents frequency point k h,1 The corresponding frequency domain signal, M represents the noise floor mean, which is based on the termination frequency k of the heartbeat frequency band. h,e The subsequent frequency point k h,e +1 to the maximum frequency point k in the frequency domain e The frequency domain signal x(k) corresponding to each frequency point k can be obtained by calculating the average value of the peak values ​​|X(k)| of these frequency domain signals using the following formula:

[0090] 6. Peak signal-to-noise ratio ratio between the respiratory frequency band and the heart rate frequency band: R bh，SNR =SNR b / SNR h Among them, SNR bPeak signal-to-noise ratio (SNR) represents the respiratory frequency band. h This represents the peak signal-to-noise ratio in the heartbeat frequency band;

[0091] 7. The ratio between the highest and second-highest heart rate: R h，12 =|X(k) h，1 )| / |X(k h，2 )|;

[0092] 8. The ratio between the frequencies corresponding to the highest and second-highest heart rate peaks: R h，k12 =k h，1 / k h，2 ; where k h，1 k represents the frequency point where the maximum peak value is located in the heartbeat frequency band. h，2 This indicates the frequency point where the second largest peak value in the heartbeat frequency band is located.

[0093] In one possible implementation, the second target feature includes, but is not limited to, time-domain features such as average peak respiratory rate, average peak heart rate, average peak ratio between respiratory and heart rate, average symmetry ratio of respiratory waveforms, respiratory rate variability, and heart rate variability.

[0094] The specific time-domain features are as follows:

[0095] 1. Average peak respiratory rate: In the respiratory signal xb(n), all peaks n are found through peak search. b，peak (m), m = 1, ..., N b，peak N b，peak Indicates respiratory signal x b The number of all peaks in (n); the average peak respiratory rate is calculated using the following formula.

[0096] 2. Average peak heart rate: In the heart rate signal x h In (n), all peaks n are found through peak search. h，peak (m), m = 1, ..., N h，peak N h，peak Represents the heartbeat signal x h The number of all peaks in (n); the average peak heart rate is calculated using the following formula.

[0097] 3. The ratio of the mean peak respiration rate to the mean peak heart rate: R bh，P =M b，peak / M h，peak Among them, M b，peak M represents the mean peak respiratory rate. h，peak This indicates the average peak heart rate;

[0098] 4. Average symmetry ratio of respiratory waveforms: In the respiratory signal xb In (n), all troughs n are found through peak search. b，valley (m), m = 1, ..., N b，valley n b，valley (m) represents the respiratory signal x b The number of all troughs in (n); the time difference between a trough and a crest is calculated using the following formula:

[0099] n valley，peak (m)=n b，valley (m)-n b，peak (m), m=1,...,min(N) b，valley N b，peak );

[0100] The time difference between the peak and the trough can be calculated using the following formula:

[0101] n peak，valley (m)=n b，peak (m+1)-n b，valley (m), m=1,...,min(N) b，valley N b，peak );

[0102] Based on the above time difference, the average symmetry ratio of the respiratory waveform is calculated using the following formula:

[0103] R b，symm =mean(n) valley，peak (m) / n peak，valley (m));

[0104] 5. Respiratory rate variability: The time difference n between adjacent peaks in the respiratory signal xb(n) is calculated. b，peak (m+1)-n b，peak (m), and the calculated time difference n between adjacent troughs. b，valley (m+1)-n b，valley (m);

[0105] Based on the aforementioned time difference, the respiratory rate variability is calculated using the following formula:

[0106]

[0107] 6. Heart rate variability: In the heart rate signal xh(n), all troughs n are found through peak search. h，valley (m), m = 1, ..., N h，valley N h，valley Represents the heartbeat signal x h The number of all troughs in (n); in the heartbeat signal, the time difference n between adjacent peaks is calculated.h，peak (m+1)-n h，peak (m), and the calculated time difference n between adjacent troughs. h，valley (m+1)-n h，valley (m);

[0108] Based on the aforementioned time difference, heart rate variability is calculated using the following formula:

[0109]

[0110] Of course, in other embodiments, it is not limited to the frequency domain features and time domain features mentioned above. Other time and frequency domain features that are beneficial to attitude recognition can also be used. For example, it can be frequency domain features in the power spectrum or higher-order spectrum, or other effective waveform features. This is not intended to constitute a specific limitation.

[0111] Step 370: Based on the first target feature and the second target feature, the pose of the target object is identified to obtain the pose recognition result.

[0112] The posture recognition result is used to indicate the posture category of the target object. In one possible implementation, if the posture is a sleeping position, the posture category includes lying flat, lying on the left side, lying on the right side, lying on the back, and lying face down.

[0113] In one possible implementation, pose recognition is achieved by calling a pose recognition model. Of course, in other embodiments, pose recognition can also be implemented using a pose recognition algorithm, such as Principal Component Analysis (PCA).

[0114] Figure 3 A flowchart illustrating the pose recognition process in one embodiment is shown, such as... Figure 3 As shown, specifically, step 370 may include the following steps:

[0115] Step 371: Fuse the first target feature with the second target feature to obtain the joint target feature.

[0116] The fusion can be achieved by adding the first target feature to the second target feature, specifically, the joint target feature = the first target feature + the second target feature; or it can be achieved by concatenating the first target feature to the second target feature, specifically, the joint target feature = [the first target feature, the second target feature], without any limitation here.

[0117] Taking splicing as an example, based on the 8 frequency domain features and 6 time domain features involved in step 350, the joint feature F of the target can be expressed as follows:

[0118] F = [E]b E h R bh，E SNR b SNR h R bh，SNR R h，12 R b，k12 M b，peak M h，peak R bh，P R b，symm W b W h ].

[0119] Step 373: Input the joint features of the target into the pose recognition model to predict the pose category and obtain the pose recognition result.

[0120] Among them, pose category prediction refers to calculating the probability that the joint features of a target belong to different pose categories.

[0121] Taking sleeping posture as an example, posture categories include lying flat, lying on the left side, and lying on the right side. Assume the probability that the target's joint feature belongs to lying flat is P1, the probability of lying on the left side is P2, and the probability of lying on the right side is P3. Assume P1 is the largest, then the prediction result is that the target's joint feature belongs to lying flat. Furthermore, if P1 ≥ 0.8 (a decision condition is set), then the prediction result is taken as the posture recognition result, that is, the posture recognition result indicates that the target object's posture category is lying flat.

[0122] It should be noted that the decision condition is set to prevent misidentification of pose categories, thereby further improving the accuracy of pose recognition. This decision condition can be flexibly set according to the actual needs of the application scenario, and is not limited here. This decision condition can be regarded as a rejection criterion for whether to reject the prediction result as the pose recognition result. In the example above, if P1 < 0.8, then the prediction result is rejected as the pose recognition result.

[0123] After determining the pose category of the target object based on the pose recognition results, corresponding intelligent services can be provided to the target object based on the pose category.

[0124] In one application scenario, the device is notified to perform an action corresponding to the posture category. For example, if the user's posture category is lying flat, the gateway can send a start command to the humidifier if the user lies flat for more than a set time (e.g., 20 minutes), thereby controlling the humidifier to turn on and preventing the user from snoring and causing a dry throat due to prolonged lying flat.

[0125] By utilizing the first target feature and the second target feature, i.e. the different expressions of the target object's pose in the detection area in the time-frequency domain, through the above process, the pose of the target object can be predicted comprehensively, and a high recognition accuracy can be obtained.

[0126] Please see Figure 4 In an exemplary embodiment, prior to step 330, the method may further include the following steps:

[0127] Step 410: Determine whether the target object in the detection area is stationary.

[0128] It is understandable that if the posture is a sleeping posture, then the sleeping posture refers to the sleeping position. When the target object enters a sleeping state, there is no obvious movement. Therefore, before performing posture recognition, it is possible to detect whether the target object is stationary in the detection area.

[0129] If the target object is detected to be stationary in the detection area, it means that the target object has no obvious movement, that is, the target object has entered a sleep state. Only then will the pose recognition of the target object be performed, that is, step 430 will be executed.

[0130] Conversely, if the target object is detected to be not stationary in the detection area, it means that the target object has not entered a sleep state, and there is no need to perform pose recognition on the target object.

[0131] In this approach, pose recognition is only performed on stationary target objects within the detection area, which helps reduce the workload of electronic devices and thus improves their processing efficiency.

[0132] In one possible implementation, such as Figure 5 As shown, the detection process may include the following steps:

[0133] Step 411: Based on a sliding window with a set window length, obtain echo signals from multiple echo signals that match the number of echo signals with the set window length.

[0134] Among them, multiple echo signals are formed by the reflection of multiple radar signals emitted by the detection equipment during the positioning process by the target object.

[0135] Taking the human body sensor as an example, for the millimeter-wave radar configured in the human body sensor, the transmitting antenna of the millimeter-wave radar can transmit multiple millimeter-wave signals (i.e., radar signals) to the target object in each time period. Correspondingly, the millimeter-wave radar can receive multiple echo signals formed by the reflection of the multiple millimeter-wave signals by the target object through the receiving antenna.

[0136] As can be seen from the above, the multiple echo signals involved in step 411 can be multiple consecutive echo signals in the same time period, or multiple consecutive echo signals in different time periods, without any limitation here.

[0137] Figure 6 A schematic diagram illustrating the movement of the sliding window is shown. Figure 6 In this context, assuming the set window length of the sliding window 401 is 3, then for multiple echo signals 402, based on the set window length of the sliding window 401, three echo signals 403 of matching quantity can be obtained from the multiple echo signals 402.

[0138] Step 413: Perform spectrum analysis on the acquired echo signal to obtain multiple distance data.

[0139] Among them, the distance data is used to indicate the radial distance between the target object and the detection device in the detection area.

[0140] Specifically, spectrum analysis refers to the process of mixing, fast Fourier transform, and modulus calculation of echo signals to obtain position data.

[0141] As mentioned earlier, for a target object maintaining its posture, position data includes, but is not limited to: the target object's posture within the detection area and the radial distance between the target object and the detection device within the detection area. Therefore, distance data can be extracted from the position data. For example, the target object's position data within the detection area is represented as... Where, ρ * θ represents the radial distance between the target object and the detection device within the detection area. * This represents the horizontal angle used to indicate the orientation of the target object within the detection area. This represents the pitch angle of the target object within the detection area, used to indicate its attitude. Therefore, distance data, denoted as θ, can be extracted from this position data. * .

[0142] Step 415: Based on multiple distance data, determine whether the target object in the detection area is stationary.

[0143] Specifically, if the radial distances indicated by multiple distance data are consistent, then the target object in the detection area is determined to be stationary.

[0144] Conversely, if at least one distance data indicates a radial distance that is inconsistent with the radial distances indicated by other distance data, it is determined that the target object in the detection area is not stationary. In this case, the control sliding window continues to slide among the multiple echo signals.

[0145] Continue reading Figure 6For the first two echo signals, the radial distance indicated by the distance data is a. Starting from the third echo signal, the radial distance indicated by the distance data is b. Therefore, for the first three echo signals obtained by the sliding window 401, if it is determined that the target object in the detection area is not stationary, the sliding window 401 is controlled to continue sliding from the echo signal 403 corresponding to the distance data with inconsistent radial distances, that is, the sliding window 401 is updated to the sliding window 401'.

[0146] Step 430: Based on the localization of the stationary target object in the detection area, target data related to the pose of the target object in the detection area is obtained.

[0147] Once a target object is detected as stationary within the detection area, its position data within the detection area can be obtained by locating the stationary target object. This position data can then be used to extract the target data. As mentioned earlier, the position data of the target object within the detection area is represented as... The target data related to the pose of the target object in the detection area is represented as follows: In one possible implementation, the target data, described by the pitch angle of the target object in the detection area to indicate its attitude, can be further represented as: Where n represents the number of vectors in the target data.

[0148] Step 450: Perform interference cancellation preprocessing on the target data.

[0149] Interference cancellation preprocessing includes unwinding and difference calculation.

[0150] Specifically, unwinding refers to the process of reducing or adding 2π to the phase value of the target data when the phase value is greater than π or less than -π, so that the phase value of the target data is always kept between -π and π.

[0151] Subtraction processing refers to calculating the difference between the current target data and the next target data, and using this difference as the processed target data. For example, the processed target data is represented as x(n), and the current target data is represented as... The second target data is represented as but,

[0152] Of course, in other embodiments, interference cancellation preprocessing can also be implemented by algorithms such as high-pass filters and wavelet transforms, which is not a specific limitation here.

[0153] Therefore, after completing the interference cancellation preprocessing, time-frequency domain features can be extracted from the processed target data to obtain the first target feature and the second target feature, i.e., step 350 is executed.

[0154] Therefore, by performing interference cancellation preprocessing on target data, signal offsets in the target data can be eliminated, such as multipath interference caused by echo signals formed by reflections from non-target objects in millimeter-wave radar, which helps to further improve the accuracy of attitude recognition.

[0155] In one exemplary embodiment, the pose recognition model for pose recognition is trained based on a base model, which can be a machine learning model or a deep learning model, including but not limited to: one or any combination of networks such as Long Short-Term Memory (LSTM), Gate Recurrent Unit (GRU), Long-Term Dependencies (LTD), SVM (Support Vector Machine), CNN (Convolutional Neural Networks), and RNN (Recurrent Neural Network).

[0156] In one possible implementation, the pose recognition model is trained based on a base model. Specifically, this means that the base model is trained using the joint features of samples in the training set and their corresponding sample labels to obtain the pose recognition model. The sample labels indicate the pose category to which the corresponding joint features belong.

[0157] It should be noted that the extraction process of sample joint features is basically the same as the extraction process of target joint features, and will not be described again here.

[0158] Please see Figure 7 In one exemplary embodiment, the model training process may include the following steps:

[0159] Step 510: Input the joint features of the current sample in the training set into the base model, and use the base model to perform pose recognition on the joint features of the sample to obtain the pose prediction result.

[0160] In the process of pose recognition based on the joint features of samples, forward propagation can be used to process the input joint features of samples. Forward propagation is the process of predicting the pose category of the sample to which the joint features belong based on the joint features of samples. Accordingly, the output result is used to indicate the predicted pose category of the sample to which the joint features belong.

[0161] Step 530: Based on the difference between the pose prediction result and the sample label corresponding to the joint features of the current sample, obtain the corresponding loss value.

[0162] Specifically, in the process of calculating the difference between the pose prediction result and the sample label, the loss value of each layer can be obtained through backpropagation.

[0163] Backpropagation is the process of calculating the loss value of each layer based on the difference between the output of each layer and the sample label.

[0164] In one possible implementation, backpropagation involves using a loss function to calculate the loss value of each layer based on the difference between the output of each layer and the sample label. This loss function can be a mean squared error loss function, a cross-entropy loss function, a regression loss function, etc., and is not limited here.

[0165] Step 550: Adjust the gradients of each model parameter in the base model according to the loss value and continue training until the gradients of each model parameter meet the set convergence conditions. Then, the pose recognition model is obtained by training the base model.

[0166] The model parameters are initialized when the base model begins training and are continuously updated during subsequent training. Specifically, during parameter updates, the gradients of the parameters in each layer of the base model are calculated based on the loss value, and the model parameters are updated based on these gradients to obtain a trained pose recognition model. This initialization can be any of random initialization, orthogonal initialization, or zero initialization; no specific limitation is made here.

[0167] If the gradients of each model parameter satisfy the set convergence condition, then a trained pose recognition model is obtained based on the updated model parameters.

[0168] Conversely, if the gradients of each model parameter do not meet the set convergence conditions, the joint features of the next sample in the training set will be input into the base model for training until the gradients of each model parameter meet the set convergence conditions. Then, the trained pose recognition model will be obtained from the base model.

[0169] The convergence condition can refer to the gradients of each model parameter no longer changing, the number of iterations reaching an iteration threshold, or the completion of training for all batches of samples. This convergence condition can be flexibly set according to the actual needs of the application scenario and is not limited here. It should be noted that, to maximize training efficiency and increase the stability of model parameter convergence, all samples can be divided into several batches for batch training, thus achieving parameter tuning of the Long Short-Term Memory network.

[0170] One possible implementation is to use the Adam (Adaptive Motion) algorithm to update the parameters of each model.

[0171] After updating the parameters of each model, return to step 510, input the joint features of the last sample in the training set into the Long Short-Term Memory network, and continue training the model on the Long Short-Term Memory network.

[0172] After the model training is completed, the pose recognition model has the ability to recognize poses and can identify the pose of the target object based on the joint features of the target.

[0173] In one exemplary embodiment, the posture is a sleeping posture, and the posture recognition result is used to indicate the sleeping posture of the target object in the detection area.

[0174] Since sleeping posture is a continuous posture, using a long short-term memory network as the basic model for training and predicting posture recognition models can help improve the accuracy of sleeping posture recognition.

[0175] The Long Short-Term Memory (LSTM) network consists of multiple layers, each containing an LSTM unit. Each LSTM unit includes a forget gate, an input gate, a tanh layer, and an output layer.

[0176] Specifically, the forgetting gate f t Used to determine the input x of the t-th LSTM unit t and the hidden layer state h of the (t-1)th LSTM unit t-1 Determine the cell state C of the (t-1)th LSTM cell. t-1 Discard some information; input gate i t Used to determine the input x of the t-th LSTM unit t and the hidden layer state h of the (t-1)th LSTM unit t-1 Determine the cell state C of the (t-1)th LSTM cell. t-1 The middle layer stores some information; the tanh layer is used to determine the cell state C that can be increased up to the t-th LSTM cell. t candidate values Thus, through the forgetting gate f t Input gate i t With the addition of the tanh layer, the cell state c of the (t-1)th LSTM cell can be obtained. t-1 Update the cell state C of the t-th LSTM cell. t The calculation formula is shown in (1):

[0177]

[0178] Where σ(·) represents the sigmoid function, W f W i and W C These are the forget gates f in the t-th LSTM unit. t Input gate i t And the weights for cell state updates, b f b i and b c This is the corresponding bias.

[0179] Furthermore, the output layer is used to output the output of the t-th LSTM unit based on the above output results. t and hidden layer state h t The calculation formula is shown in (2):

[0180]

[0181] Where σ(·) represents the sigmoid function, W o and b o is the weight and bias of the output gate in the t-th LSTM unit.

[0182] It is worth mentioning that in this embodiment, when the posture is a sleeping position, the posture category includes at least three types: lying flat, lying on the left side, and lying on the right side. Therefore, unlike the standard Long Short-Term Memory (LSTM) network for binary classification, the LSTM network in this embodiment essentially adds an extra linear layer to the standard LSTM network. This is done by passing the output of the last layer in the standard LSTM network (assuming it contains t LSTM units, then the last layer is the t-th LSTM unit) to... t Input this linear layer, calculate the output result of this linear layer, i.e., o d =Softmax(W d ·o t +b d ), where W d and b d These are the weights and biases of the linear layer, respectively.

[0183] Therefore, based on the output of the linear layer, the probability of o is the highest. d This allows the system to identify the sleeping position of the target object within the detection area.

[0184] Now combined Figure 8 The following is a detailed explanation of the sleeping posture recognition process performed by the posture recognition model trained using a long short-term memory network as the base model:

[0185] Training process:

[0186] The first step is to construct the training and testing sets for the Long Short-Term Memory (LSTM) network.

[0187] In this process, sample data under different sleeping positions are collected, and the joint features of the samples are obtained according to steps 310 to 330. The sample labels are then labeled simultaneously, thereby forming a training set and a test set with a ratio of 4:1. In one possible implementation, the sample labels include at least lying flat (0), lying on the left side (1), and lying on the right side (2).

[0188] The second step is to initialize the model parameters of each layer (or LSTM unit) in the Long Short-Term Memory network. In one possible implementation, the initialization of the model parameters includes: orthogonal initialization of the weights and zero initialization of the biases.

[0189] The third step is to perform forward propagation to calculate the output of each layer (or LSTM unit) in the Long Short-Term Memory network.

[0190] Specifically, the output of the (t-1)th LSTM unit is used as the input of the tth LSTM unit, and the output of the tth LSTM unit is calculated until the last linear layer has completed its calculation.

[0191] The fourth step is to select cross-entropy as the loss function for the long short-term memory network and calculate the loss value of each layer through backpropagation.

[0192] The fifth step is to calculate the gradient of each model parameter in the Long Short-Term Memory network based on the loss value of each layer, and update the model parameters by means of an optimization algorithm when the gradient of each model parameter does not meet the set convergence condition.

[0193] In one possible implementation, the optimization algorithm includes the Adam algorithm.

[0194] The sixth step involves iterating through steps one through five until the gradients of the model parameters in the long short-term memory network meet the set convergence conditions. The pose recognition model is then obtained from the long short-term memory network.

[0195] To maximize training efficiency, one possible approach is to increase the stability of convergence for each model parameter by employing batch training. In one round of training (meaning all samples are trained once), all samples are divided into batches, and it is determined whether the gradients of each model parameter meet the set convergence conditions before proceeding to the next round of training.

[0196] Prediction process:

[0197] like Figure 8 As shown, the pose recognition process based on a pose recognition model trained by a long short-term memory network can include the following steps:

[0198] Step 3731: Input the joint features of the target into the first layer of the pose recognition model to obtain the output of the first layer.

[0199] Step 3733: Input the output of the first layer into the second layer of the pose recognition model to obtain the output of the second layer, and so on until the output of the last layer of the pose recognition model is obtained.

[0200] The output results are used to indicate the pose category of the target object to which the joint features of the target belong.

[0201] Step 3735: If the output of the last layer satisfies the set decision conditions, then the output of the last layer is taken as the pose recognition result.

[0202] Under the above embodiments, pose recognition based on long short-term memory networks is realized, which has better generalization ability for pose detection processes with continuity, thereby ensuring the accuracy of pose recognition.

[0203] In one exemplary embodiment, after step 370, the method may further include the following steps:

[0204] Based on the posture category, first target feature, and / or second target feature indicated by the posture recognition results, the credibility configuration is performed for the vital signs of the target object in different postures.

[0205] As mentioned earlier, if the posture is a sleeping position, the accuracy of the detected vital signs of the target object varies under different sleeping positions. Therefore, the sleep state judgment and sleep stage analysis based on the vital signs will be biased. Therefore, in this embodiment, after obtaining the posture category indicated by the posture recognition result, the credibility of the target object's vital signs under different postures will be configured by combining the first target feature and the second target feature, so as to assist the user in conducting more effective information analysis.

[0206] Taking sleeping posture as an example and vital signs including respiratory rate and heart rate as an example, the process of configuring the credibility of respiratory rate and heart rate of the target subject under different sleeping postures is explained as follows:

[0207] In one possible implementation, based on the sleeping posture indicated by the posture recognition results, the credibility of the target object's respiratory rate and heart rate under different sleeping postures is configured.

[0208] Table 1 Reliability based on attitude category configuration

[0209]

[0210]

[0211] As can be seen from Table 1, the reliability of respiratory rate and heart rate is highest when the sleeping position is supine.

[0212] In one possible implementation, the credibility of the respiratory rate and heart rate of the target object under different sleeping positions is configured based on the sleeping position indicated by the posture recognition result and the peak signal-to-noise ratio of the set frequency band (first target feature).

[0213] First, the peak signal-to-noise ratio of the set frequency band is divided into high signal-to-noise ratio, medium signal-to-noise ratio, and low signal-to-noise ratio.

[0214] Assume τ bl，SNR τ is the lower limit threshold of the signal-to-noise ratio in the breathing frequency band. bh，SNR Let be the upper threshold for the signal-to-noise ratio (SNR) of the respiratory frequency band. Then, for the first target feature, the peak signal-to-noise ratio (SNR) of the respiratory frequency band... b In terms of SNR i ≤τ bl，SNR If τ is low, it is considered a low signal-to-noise ratio. bl，SNR <SNR b <τ bh，SNR If SNR is considered as the signal-to-noise ratio, then... b ≥τ bh，SNR If it is high, it is considered to have a high signal-to-noise ratio.

[0215] Similarly, suppose r hlSNR τ is the lower limit threshold of the signal-to-noise ratio in the heartbeat frequency band. hh，SNR Let be the upper threshold of the signal-to-noise ratio (SNR) for the heartbeat frequency band. Then, for the first target feature, the peak SNR (Signal-to-Noise Ratio) of the heartbeat frequency band... h In terms of SNR h ≤τ hl，SNR If τ is low, it is considered a low signal-to-noise ratio. hl，SNR <SNR h <τ hh，SNR If SNR is considered as the signal-to-noise ratio, then... h ≥τ hh，SNR If it is high, it is considered to have a high signal-to-noise ratio.

[0216] Therefore, the reliability of respiratory rate and heart rate can be configured by combining sleeping position, high signal-to-noise ratio, medium signal-to-noise ratio, and low signal-to-noise ratio.

[0217] Table 2 Respiratory rate reliability based on sleep position and peak signal-to-noise ratio of respiratory frequency band.

[0218]

[0219] As can be seen from Table 2, when the sleeping position is supine, the reliability of the respiratory rate is highest when the peak signal-to-noise ratio of the respiratory frequency band is high.

[0220] Table 3 Heart Rate Confidentiality Based on Sleep Position and Peak Signal-to-Noise Ratio of Heart Rate Bands

[0221]

[0222]

[0223] As can be seen from Table 3, when the sleeping position is supine, the confidence level of the heart rate is highest when the peak signal-to-noise ratio in the heart rate frequency band is high.

[0224] Once the reliability of the target's vital signs under different sleeping positions is determined, the user's sleep state and sleep stage can be analyzed more accurately based on this reliability.

[0225] Specifically, in one possible implementation, the sleep stage analysis process may include the following steps: based on the sleeping posture indicated by the posture recognition results, perform vital sign detection on the target object to obtain the target object's vital sign data; determine the sleep staging result based on the vital signs of the target object indicated by the vital sign data and the confidence level configured for the target object's vital signs in the sleeping posture; and notify the device to perform the action corresponding to the sleep stage indicated by the sleep staging result.

[0226] Among them, vital signs data are used to indicate the vital signs of the target subject, such as respiratory rate and heart rate. Sleep staging results are used to indicate the sleep stages of the target subject.

[0227] In one possible implementation, the sleep stages include a deep sleep stage and a light sleep stage. It should be understood that the deep sleep stage indicates that the target subject's sleep state is a deep sleep state; the light sleep stage indicates that the target subject's sleep state is a light sleep state.

[0228] After determining the sleep stage of the target subject based on the sleep staging results, corresponding intelligent services can be provided to the target subject based on the sleep stage.

[0229] In one application scenario, the device is notified to perform actions corresponding to the sleep stage. For example, if the user is in a light sleep stage, the gateway sends control commands to a smart light or smart speaker to adjust the brightness of the smart light or to play soothing music to help the user quickly enter a deep sleep stage, thereby improving the user's sleep quality.

[0230] For example, before the alarm clock rings, the gateway sends corresponding control commands to the alarm clock according to the user's different sleep stages to control the alarm clock to play different ringtones, and / or the gateway sends corresponding control commands to the smart light to control the smart light to turn on and adjust different brightness to wake up the user.

[0231] Through the combination of the above embodiments, the reliability of the vital signs of the target object under different sleeping positions can be configured, which can provide more accurate data for analyzing the user's sleep state and sleep stage, thereby greatly improving the reliability of intelligent services and enhancing the user experience.

[0232] The following are embodiments of the apparatus described in this application, which can be used to execute the posture recognition method involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the posture recognition method involved in this application.

[0233] Please see Figure 9 This application provides a posture recognition device 900, including but not limited to: a data acquisition module 910, a feature extraction module 950, and a posture recognition module 970.

[0234] The data acquisition module 910 is used to obtain target data related to the orientation of the target object in the detection area based on the positioning of the target object.

[0235] The feature extraction module 950 is used to extract time-frequency domain features from the target data to obtain the first target feature and the second target feature; the first target feature is the frequency domain representation of the target object's pose in the detection area, and the second target feature is the time domain representation of the target object's pose in the detection area.

[0236] The posture recognition module 970 is used to recognize the posture of the target object based on the first target feature and the second target feature, and obtain the posture recognition result.

[0237] In an exemplary embodiment, the feature extraction module includes: a frequency domain transformation unit, configured to perform frequency domain transformation on the target data to obtain multiple frequency domain signals of the target data in the frequency domain, each frequency domain signal corresponding to a frequency point in the frequency domain; and a time domain transformation unit, configured to obtain a first target feature from the frequency domain signal corresponding to the frequency point in a set frequency band, and obtain a second target feature by performing time domain transformation on the frequency domain signal corresponding to the frequency point in the set frequency band.

[0238] In an exemplary embodiment, the pose recognition module includes: a feature fusion unit, configured to fuse the first target feature and the second target feature to obtain a joint target feature; and a pose prediction unit, configured to input the joint target feature into a pose recognition model to predict the pose category and obtain the pose recognition result.

[0239] In an exemplary embodiment, the apparatus further includes: a model training module, configured to train a base model based on joint features of samples in a training set and their corresponding sample labels to obtain the pose recognition model, wherein the sample labels indicate the pose category to which the corresponding joint feature belongs; the model training module includes: a sample input unit, configured to input a current joint feature of a sample in the training set into the base model, and perform pose recognition on the joint feature of the sample through the base model to obtain a pose prediction result; a loss calculation unit, configured to calculate a corresponding loss value through backpropagation based on the pose prediction result and the sample label corresponding to the current joint feature of the sample; and a convergence unit, configured to adjust the gradient of each model parameter in the base model according to the loss value and continue training until the gradient of each model parameter satisfies a set convergence condition, thereby training the pose recognition model from the base model.

[0240] In an exemplary embodiment, the apparatus further includes: a signal acquisition module, configured to acquire echo signals matching the number of echo signals of the set window length from a plurality of echo signals based on a sliding window of a set window length; the plurality of echo signals are multiple radar signals emitted by the detection device during the positioning process, which are reflected by the target object; a spectrum analysis module, configured to perform spectrum analysis on the acquired echo signals to obtain multiple distance data, the distance data being used to indicate the radial distance between the target object and the detection device in the detection area; and a stationary detection module, configured to determine whether the target object in the detection area is stationary based on the multiple distance data; if not, to control the sliding window to continue sliding among the multiple echo signals.

[0241] In an exemplary embodiment, the posture is a sleeping posture, and the posture recognition result is used to indicate the sleeping posture of the target object in the detection area.

[0242] In one exemplary embodiment, the device further includes: a first automatic control module, configured to notify the device to perform an action corresponding to the sleeping posture indicated by the posture recognition result.

[0243] In one exemplary embodiment, the device further includes: a vital sign detection module, configured to perform vital sign detection on the target object based on the sleeping posture indicated by the posture recognition result, and obtain vital sign data of the target object, the vital sign data being used to indicate the vital signs of the target object; a sleep staging module, configured to determine a sleep staging result based on the vital signs of the target object indicated by the vital sign data and the confidence level configured for the vital signs of the target object in the sleeping posture; and a second automatic control module, configured to notify the device to perform an action corresponding to the sleep stage indicated by the sleep staging result.

[0244] In one exemplary embodiment, the apparatus further includes a configuration module, configured to configure the credibility of the vital signs of the target object in the sleeping position based on the sleeping position indicated by the posture recognition result, the first target feature, and / or the second target feature.

[0245] It should be noted that the posture recognition device provided in the above embodiments is only illustrated by the division of the above functional modules when performing posture recognition. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the posture recognition device will be divided into different functional modules to complete all or part of the functions described above.

[0246] Furthermore, the posture recognition device and posture recognition method embodiments provided in the above embodiments belong to the same concept, and the specific way in which each module performs operations has been described in detail in the method embodiments, and will not be repeated here.

[0247] Figure 10 A schematic diagram of the structure of an electronic device is shown according to an exemplary embodiment. This electronic device is suitable for... Figure 1 The server-side configuration of the implementation environment is shown as 170.

[0248] It should be noted that this electronic device is merely an example adapted to this application and should not be construed as providing any limitation on the scope of use of this application. Furthermore, this electronic device should not be interpreted as requiring or depending on any specific feature. Figure 10 One or more components of the exemplary electronic device 2000 shown.

[0249] The hardware structure of electronic devices 2000 can vary significantly due to differences in configuration or performance, such as... Figure 10 As shown, the electronic device 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0250] Specifically, power supply 210 is used to provide operating voltage for various hardware devices on electronic device 2000.

[0251] Interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. For example, to perform... Figure 1 The diagram illustrates the interaction between the smart device 130 and the server 170 in the implementation environment.

[0252] Of course, in other examples adapted in this application, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. Figure 10 As shown, this does not constitute a specific limitation.

[0253] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0254] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 2000, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0255] Application 253 is a computer program that performs at least one specific task based on operating system 251, and may include at least one module ( Figure 10 (Not shown), each module may contain a computer program for the electronic device 2000. For example, the attitude recognition device may be considered as an application program 253 deployed on the electronic device 2000.

[0256] Data 255 can be photos, pictures, etc. stored on a disk, or it can be target data, etc., stored in memory 250.

[0257] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer programs stored in the memory 250, thereby performing operations and processing on massive amounts of data 255 stored in the memory 250. For example, the attitude recognition method may be implemented by the central processing unit 270 reading a series of computer programs stored in the memory 250.

[0258] Furthermore, this application can also be implemented through hardware circuits or a combination of hardware circuits and software. Therefore, the implementation of this application is not limited to any specific hardware circuit, software, or combination thereof.

[0259] Please see Figure 11 This application provides an electronic device 4000, which may include:

[0260] exist Figure 11 The electronic device 4000 includes at least one processor 4001, at least one communication bus 4002, and at least one memory 4003.

[0261] The processor 4001 and memory 4003 are connected, for example, via a communication bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0262] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0263] The communication bus 4002 may include a path for transmitting information between the aforementioned components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0264] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0265] The memory 4003 stores a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002.

[0266] When the computer program is executed by the processor 4001, it implements the posture recognition method in the above embodiments.

[0267] Furthermore, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the posture recognition method described in the above embodiments.

[0268] This application provides a computer program product comprising a computer program stored in a storage medium. A processor of a computer device reads the computer program from the storage medium and executes the computer program, causing the computer device to perform the gesture recognition methods described in the above embodiments.

[0269] Compared with related technologies, the method of comprehensively predicting the sleeping posture of a target subject by combining the time-frequency domain representations of vital signs such as respiration and heartbeat has a higher accuracy rate in sleep posture recognition. At the same time, it can accurately identify different sleeping postures using millimeter-wave radar without the need for additional sensors. Sleep posture recognition using long short-term memory networks has better generalization ability, which further helps to improve the accuracy of sleep posture recognition. In addition, the confidence level configured based on the vital signs of the target subject in different sleeping postures assists users in analyzing sleep state, sleep stage, etc., which helps to improve the accuracy of analysis.

[0270] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0271] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A pose recognition method, characterized in that, include: Multiple echo signals are acquired, and spectral analysis is performed on each echo signal to obtain multiple distance data; the distance data is used to indicate the radial distance between each target object in the detection area and the detection device; Based on the distance data, determine whether the target object is stationary; If the target object is determined to be stationary, target data related to the pose of the target object in the detection area is obtained based on the localization of the target object. The target data is subjected to time-frequency domain feature extraction to obtain a first target feature and a second target feature; the first target feature is the frequency domain representation of the pose of the target object in the detection area, and the second target feature is the time domain representation of the pose of the target object in the detection area. The first target feature and the second target feature are fused to obtain the joint target feature; The joint features of the target are input into the pose recognition model to identify the pose of the target object and obtain the pose recognition result; the pose includes sleeping posture, and the pose recognition result is used to indicate the sleeping posture category of the target object in the detection area.

2. The method as described in claim 1, characterized in that, The step of extracting time-frequency domain features from the target data to obtain first target features and second target features includes: The target data is transformed in the frequency domain to obtain multiple frequency domain signals of the target data in the frequency domain, and each frequency domain signal corresponds to a frequency point in the frequency domain. Frequency domain features are extracted from the frequency domain signals corresponding to each frequency point in the set frequency band to obtain the first target features; The second target feature is obtained by performing a time-domain transformation on the frequency domain signal corresponding to each frequency point in the set frequency band.

3. The method as described in claim 1, characterized in that, The method further includes: training the basic model based on the joint features of samples in the training set and their corresponding sample labels to obtain the pose recognition model, wherein the sample labels are used to indicate the pose category of the sample to which the corresponding joint features belong; The step of training the basic model based on the joint features of samples in the training set and their corresponding sample labels to obtain the pose recognition model includes: The joint features of the current sample in the training set are input into the base model, and the base model performs pose recognition on the joint features of the sample to obtain the pose prediction result. Based on the difference between the pose prediction result and the sample label corresponding to the joint features of the current sample, the corresponding loss value is obtained; The gradients of each model parameter in the base model are adjusted according to the loss value, and training continues until the gradients of each model parameter meet the set convergence condition. Then, the pose recognition model is obtained by training the base model.

4. The method as described in claim 1, characterized in that, Before obtaining target data related to the pose of the target object in the detection area based on the localization of the target object, the method further includes: Based on a sliding window with a set window length, echo signals matching the set window length are obtained from multiple echo signals; the multiple echo signals are multiple radar signals emitted by the detection device during the positioning process, which are reflected by the target object. The acquired echo signal was subjected to spectral analysis to obtain multiple distance data. Based on multiple distance data points, determine whether the target object in the detection area is stationary; If not, the sliding window is controlled to continue sliding among multiple echo signals.

5. The method as described in claim 1, characterized in that, The method further includes: The device is instructed to perform an action corresponding to the sleeping posture category indicated by the posture recognition result.

6. The method as described in claim 1, characterized in that, The method further includes: Based on the sleeping posture category indicated by the posture recognition result, vital signs detection is performed on the target object to obtain vital sign data of the target object, and the vital sign data is used to indicate the vital signs of the target object; The sleep staging result is determined based on the vital signs of the target object indicated by the vital sign data and the confidence level configured for the vital signs of the target object under the sleep position category; The device is instructed to perform an action corresponding to the sleep stage indicated by the sleep staging results.

7. The method as described in claim 6, characterized in that, Before determining the sleep staging result based on the vital signs of the target object indicated by the vital sign data and the confidence level configured for the vital signs of the target object in different sleeping positions, the method further includes: Based on the sleeping posture category indicated by the posture recognition result, the first target feature, and / or the second target feature, the credibility configuration of the vital signs of the target object under the sleeping posture category is performed.

8. A posture recognition device, characterized in that, include: The data acquisition module is used to acquire multiple echo signals and perform spectral analysis on each echo signal to obtain multiple distance data; the distance data is used to indicate the radial distance between each target object in the detection area and the detection device; based on the distance data, it is determined whether the target object is stationary; The feature extraction module is used to, when the target object is determined to be stationary, obtain target data related to the pose of the target object in the detection area based on the localization of the target object; perform time-frequency domain feature extraction on the target data to obtain a first target feature and a second target feature; the first target feature is a frequency domain representation of the pose of the target object in the detection area, and the second target feature is a time domain representation of the pose of the target object in the detection area; and fuse the first target feature and the second target feature to obtain a joint target feature. The posture recognition module is used to input the joint features of the target into the posture recognition model, recognize the posture of the target object, and obtain the posture recognition result; the posture includes sleeping posture, and the posture recognition result is used to indicate the sleeping posture category of the target object in the detection area.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, and at least one communication bus, wherein, The memory stores a computer program, and the processor reads the computer program from the memory via the communication bus; When the computer program is executed by the processor, it implements the posture recognition method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the posture recognition method as described in any one of claims 1 to 7.

Images