A behavior state determination method, apparatus, device, and medium

Abstract

The application provides a behavior state determination method and device, equipment and a medium, which are used to solve the problem of possible errors when measuring chest displacement, breathing displacement and heartbeat displacement based on the prior art. In the embodiment of the application, a first set number of chest displacements, breathing displacements and heartbeat displacements within a set time period are collected based on a millimeter wave radar, each second chest displacement vector, each second breathing displacement vector and each second heartbeat displacement vector are generated, each second chest displacement vector, each second breathing displacement vector and each second heartbeat displacement vector are superimposed, and a three-dimensional matrix is generated. The three-dimensional matrix is input into a neural network model to obtain a state value, thereby determining whether a user is in a detection state. This is because the measurement of breathing and heartbeat during movement also has an impact on the detected breathing and heartbeat, and therefore the current state of the user can be accurately determined based on the measured breathing and heartbeat, which can improve the accuracy of the measurement.

Description

Technical Field

[0001] This invention relates to the fields of radar signal processing and Internet of Things (IoT) technology, and in particular to a method, apparatus, device, and medium for determining behavioral states. Background Technology

[0002] Non-contact vital sign detection devices, which measure breathing and heart rate using millimeter-wave radar, are a relatively mature technology. However, from a product perspective, this technology still has some imperfections. One of them is that non-contact vital sign detection devices require the user to remain still when performing measurements using millimeter-wave radar, thus requiring the user's condition to be determined.

[0003] There are several methods for determining a user's status: one is active user control, where the user remains still when starting to use the non-contact vital sign detection device to measure breathing and heart rate; this method is inconvenient. Another method involves using other devices, such as infrared detectors, to determine the user's current status when they begin measuring breathing and heart rate. This additional device increases product cost. A third method uses the non-contact vital sign detection device itself, judging whether someone is in the detection area. If someone is in the detection area, the device is assumed to be in detection mode. This method does not consider personnel movement, which increases the error in measuring chest displacement, respiratory displacement, and heart rate displacement. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for determining behavioral states, which addresses the potential errors that may occur when measuring chest displacement, respiratory displacement, and heartbeat displacement using existing technologies.

[0005] Firstly, this application provides a method for determining a behavioral state, the method comprising:

[0006] Based on millimeter-wave radar data collection within a set time period, a first predetermined number of chest displacement, respiratory displacement, and heartbeat displacement are collected. A first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector are generated based on these measurements. A second predetermined number of components from the first chest displacement vector are then extracted to generate each second chest displacement vector. Similarly, a second predetermined number of components from the first respiratory displacement vector and a second predetermined number of components from the first heartbeat displacement vector are extracted to generate each second heartbeat displacement vector. Finally, each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector are superimposed to generate a three-dimensional matrix.

[0007] The three-dimensional matrix is ​​input into a pre-trained neural network model to obtain the state value output by the neural network model. Based on the state value, it is determined whether the user is in a detection state.

[0008] If the status value is greater than the set threshold, it is determined that the user is in a detection state.

[0009] Secondly, this application provides a behavior state determination device, the device comprising:

[0010] The generation module is used to generate a first set number of chest displacement, respiratory displacement, and heartbeat displacement data within a set time period based on millimeter-wave radar data acquisition; generate a first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector based on the first set number of chest displacement, respiratory displacement, and heartbeat displacement data, respectively; obtain a second set number of components from the first chest displacement vector to generate each second chest displacement vector; obtain a second set number of components from the first respiratory displacement vector to generate each second respiratory displacement vector; obtain a second set number of components from the first heartbeat displacement vector to generate each second heartbeat displacement vector; and superimpose each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector to generate a three-dimensional matrix.

[0011] The determination module is used to input the three-dimensional matrix into a pre-trained neural network model, obtain the state value output by the neural network model, and determine whether the user is in a detection state based on the state value; if the state value is greater than a set threshold, it is determined that the user is in a detection state.

[0012] Thirdly, this application provides an electronic device including a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of a behavior state determination method as described in any of the preceding claims.

[0013] Fourthly, this application provides a computer storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of any of the above-described behavior state determination methods.

[0014] In this embodiment, based on millimeter-wave radar data collection within a set time period, a first set number of chest displacement, respiratory displacement, and heartbeat displacement are collected to generate a first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector. Then, based on the first chest displacement vector, the first respiratory displacement vector, and the first heartbeat displacement vector, a second set number of components are obtained to generate each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector. These vectors are then superimposed to generate a three-dimensional matrix. The three-dimensional matrix is ​​input into a neural network model to obtain state values, thereby determining whether the user is in a detection state. This is because measuring breathing and heartbeat during movement also affects the detected breathing and heartbeat; therefore, based on the measured breathing and heartbeat, the user's current state can be accurately determined, thus improving the accuracy of the measurement. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1a A schematic diagram of a user's heart rate waveform in a detection state is provided for some embodiments of this application;

[0017] Figure 1b Schematic diagram of a user's respiratory waveform in a detection state provided for some embodiments of this application;

[0018] Figure 1c A schematic diagram of chest displacement information of a user in a detection state provided for some embodiments of this application;

[0019] Figure 1d A schematic diagram illustrating distance information of a user in a detection state, provided for some embodiments of this application;

[0020] Figure 2a This application provides schematic diagrams of heart rate waveforms of a user in a non-detection state, as shown in some embodiments.

[0021] Figure 2b Schematic diagrams of the breathing waveforms of a user in a non-detection state, provided for some embodiments of this application;

[0022] Figure 2c A schematic diagram illustrating chest displacement information of a user in a non-detection state, provided for some embodiments of this application;

[0023] Figure 2d A schematic diagram illustrating distance information of a user in a non-detection state, provided for some embodiments of this application;

[0024] Figure 3 A schematic diagram of a behavior state determination process provided for some embodiments of this application;

[0025] Figure 4 A schematic diagram illustrating the process of generating each second chest displacement vector by obtaining a second predetermined number of components in a first chest displacement vector, provided for some embodiments of this application;

[0026] Figure 5 A schematic diagram illustrating the process of obtaining a second predetermined number of components in a first respiratory displacement vector to generate each second respiratory displacement vector, provided for some embodiments of this application;

[0027] Figure 6 A schematic diagram illustrating the process of obtaining a second predetermined number of components in a first heartbeat displacement vector to generate each second heartbeat displacement vector, provided for some embodiments of this application;

[0028] Figure 7 Schematic diagrams of CNN network model structures provided for some embodiments of this application;

[0029] Figure 8 A schematic diagram illustrating the training process of a neural network model provided for some embodiments of this application;

[0030] Figure 9 Flowcharts for determining behavioral states provided for some embodiments of this application;

[0031] Figure 10 A schematic diagram of a behavior state determination device provided for some embodiments of this application;

[0032] Figure 11 This is a schematic diagram of an electronic device structure provided for some embodiments of this application. Detailed Implementation

[0033] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0034] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0035] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0036] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0037] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0038] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0039] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

[0040] In this embodiment, based on millimeter-wave radar data collection within a set time period, a first set number of chest displacement, respiratory displacement, and heartbeat displacement are collected to generate a first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector. Then, based on the first chest displacement vector, the first respiratory displacement vector, and the first heartbeat displacement vector, a second set number of components are obtained to generate each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector. These vectors are then superimposed to generate a three-dimensional matrix. The three-dimensional matrix is ​​input into a neural network model to obtain state values, thereby determining whether the user is in a detection state. This is because measuring breathing and heartbeat during movement also affects the detected breathing and heartbeat; therefore, based on the measured breathing and heartbeat, the user's current state can be accurately determined, thus improving the accuracy of the measurement.

[0041] Millimeter-wave radar measures respiration and heart rate, and in addition to outputting key indicators of respiration and heart rate, it can also output curves of chest displacement, respiration, and heart rate in the user. Figure 1a This is a schematic diagram of a user's heart rate waveform in a detection state, provided for some embodiments of this application. Figure 2a This application provides schematic diagrams of heart rate waveforms of a user in a non-detection state, as shown in some embodiments. Figure 1a , Figure 2a As shown, the first row shows heart rate (frequency), and the second row shows heart rate waveforms (frequency). The horizontal axis of the heart rate waveform represents time, and the vertical axis represents phase. In the same amount of time, the heart rate in the detection state is higher than that in the non-detection state. However, the phase fluctuation of the heart rate in the detection state is smoother than that of the heart rate in the non-detection state, indicating that the accuracy of heart rate measurement is higher when the user is in the detection state. Figure 1b This application provides schematic diagrams of respiratory rate waveforms of a user in a detection state, as shown in some embodiments. Figure 2b Schematic diagrams of the respiratory waveforms of a user in a non-detection state, provided for some embodiments of this application, such as... Figure 1b , Figure 2b As shown, the first row shows all respiration, and the second row shows all respiration waveforms. The horizontal axis of the respiration waveform is time, and the vertical axis is phase. Respiration in the detection state is higher than that in the non-detection state within the same time period. However, the phase fluctuation of respiration in the detection state is more gradual than that of respiration in the non-detection state, indicating that the accuracy of measuring respiration is higher when the user is in the detection state. Figure 1c This is a schematic diagram illustrating chest displacement information of a user in a detection state, provided for some embodiments of this application. Figure 2cThis application provides schematic diagrams of chest displacement information of a user in a non-detection state, as shown in some embodiments. Figure 1c , Figure 2c As shown, the horizontal axis of the chest displacement information graph is the data frame, and the vertical axis is the displacement. Under the same data frame, the fluctuation of the chest displacement in the detection state is smoother than that in the non-detection state, indicating that the accuracy of measuring chest displacement when the user is in the detection state is high. Figure 1d This is a schematic diagram illustrating distance information of a user in a detection state, provided for some embodiments of this application. Figure 2d This application provides schematic diagrams illustrating distance information for users in a non-detection state, as shown in some embodiments. Figure 1d , 2d As shown in the figure, the horizontal axis of the distance information graph represents distance, and the vertical axis represents amplitude. The distance information graph in the detection state shows large amplitude fluctuations at distances of 0.6-0.9 meters, while the distance information graph in the non-detection state shows large amplitude fluctuations at distances of 0-0.6 meters. The vertical fluctuations of the distance information in the non-detection state are more frequent than those in the detection state, indicating that the accuracy of distance measurement information is high when the user is in the detection state.

[0042] To improve the accuracy of measurements, this application provides a method, apparatus, device, and medium for determining behavioral states.

[0043] Figure 3 A schematic diagram of a behavior state determination process provided for some embodiments of this application, the process including:

[0044] S301: Based on millimeter-wave radar data collection within a set time period, a first set number of chest displacement, respiratory displacement, and heartbeat displacement are collected. A first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector are generated based on these first set number of chest displacement, respiratory displacement, and heartbeat displacement data, respectively. A second set number of components from the first chest displacement vector are obtained to generate each second chest displacement vector. A second set number of components from the first respiratory displacement vector are obtained to generate each second respiratory displacement vector. A second set number of components from the first heartbeat displacement vector are obtained to generate each second heartbeat displacement vector. The second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector are superimposed to generate a three-dimensional matrix.

[0045] The behavioral state determination method provided in this application is applied to an electronic device, which may be a contact-type vital sign detection device, a non-contact vital sign detection device, or the like. The electronic device is equipped with millimeter-wave radar.

[0046] Millimeter-wave radar can measure data at a single moment per chirp, accumulating data over a period of time after multiple chirps. Bandpass filtering is applied to the data to retain bands near common respiratory and cardiac displacements, allowing for separate analysis to obtain the respiratory and cardiac displacements and their respective curves. Generally, a chirp lasts only a few tens of microseconds, while the human respiratory cycle is typically a few seconds, and the heartbeat cycle a fraction of a second. Assuming a chirp duration of 40 μs, a respiratory cycle of 2 seconds, and a heartbeat cycle of 0.8 seconds, it's clear that the chirp duration is very short compared to breathing and heartbeat. Within one heartbeat cycle, the radar can collect approximately 20,000 chest displacement samples, which is too high a sampling rate for subsequent analysis. Therefore, to save computational resources, preprocessing of the detection data is necessary.

[0047] After the electronic device acquires a first predetermined number of chest displacements, respiratory displacements, and heartbeat displacements within a set time period using millimeter-wave radar, it generates a first chest displacement vector based on the first predetermined number of chest displacements, a first respiratory displacement vector based on the first predetermined number of respiratory displacements, and a first heartbeat displacement vector based on the first predetermined number of heartbeat displacements. Specifically, after acquiring the first predetermined number of chest displacements within the set time period using millimeter-wave radar, these chest displacements are sequentially used as each component in the first chest displacement vector according to the order of acquisition time. Correspondingly, the first respiratory displacement vector and the first heartbeat displacement vector are generated in the same way. Therefore, since the acquired chest displacements, respiratory displacements, and heartbeat displacements are all the first predetermined number, the resulting first chest displacement vector, first respiratory displacement vector, and first heartbeat displacement vector have the same length, that is, the dimensions of the first chest displacement vector, first respiratory displacement vector, and first heartbeat displacement vector are all the same.

[0048] Specifically, the time intervals for collecting chest displacement, respiratory displacement, and heart rate displacement are fixed. The collection processes for chest displacement, respiratory displacement, and heart rate are the same. The collection process for chest displacement will now be described. Since the collection interval between every two adjacent chest displacements is fixed, we can set it as tc. The length of the vector is also fixed. Therefore, based on the length of this vector, which is the first set number, we can determine the collection duration. The collection duration can be T. T / tc is the number of components contained in the first chest displacement vector, which is the dimension of the first chest displacement vector.

[0049] In this embodiment, the electronic device generates a first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector for a first predetermined number of chest displacements, respiratory displacements, and heartbeat displacements within a predetermined time period based on millimeter-wave radar data. Then, it obtains a second predetermined number of components from the first chest displacement vector to generate each second chest displacement vector, obtains a second predetermined number of components from the first respiratory displacement vector to generate each second respiratory displacement vector, and obtains a second predetermined number of components from the first heartbeat displacement vector to generate each second heartbeat displacement vector.

[0050] Since the processes for generating the second chest displacement vector, the second respiratory displacement vector, and the second heartbeat displacement vector are the same, the process for generating the second chest displacement vector will now be explained.

[0051] Specifically, a slice window can be pre-set, where the length of the slice window can be tp, and the interval between every two adjacent components in the chest displacement vector is tc. The second set number of components corresponding to each slice window in the first chest displacement vector is obtained. tp / tc / nm is the number of components contained in the second chest displacement vector. The number of components in each second chest displacement vector is the second set number, that is, the dimension of the second chest displacement vector is the second set number, which can be represented by np. The second set number of components in each slice window are used as the components of the second chest displacement vector to generate each second chest displacement vector.

[0052] For example, the first chest displacement vector can be (23, 34, 43, 45, 51, 56, 62, 64, 60, 58, 53, 47, 44, 39, 33, 28), and the second set quantity is 4, which means that the first chest displacement vector is divided into a second chest displacement vector with a dimension of 4. The resulting second chest displacement vectors are (23, 34, 43, 45), (51, 56, 62, 64), (60, 58, 53, 47), and (44, 39, 33, 28). Specifically, the first and second set quantities can be set according to requirements, and generally the first set quantity is an integer multiple of the second set quantity.

[0053] Each second chest displacement vector is concatenated into a two-dimensional chest displacement matrix F_Sc, each second respiratory displacement vector is concatenated into a two-dimensional respiratory displacement matrix F_Sb, and each second heartbeat displacement vector is concatenated into a two-dimensional heartbeat displacement matrix F_Sh. Specifically, each second chest displacement vector is generated by obtaining a second predetermined number of components from the first chest displacement vector. Since the number of components in each second chest displacement vector is the second predetermined number, the dimension of each second chest displacement vector is the same. Each second chest displacement vector is considered as a row vector, and each row vector can be concatenated sequentially according to row order to obtain a chest displacement vector group containing multiple row vectors. This chest displacement vector group is used as the two-dimensional chest displacement matrix F_Sc. The process of generating the two-dimensional respiratory displacement matrix F_Sb and the two-dimensional heartbeat displacement matrix F_Sh is the same as the process of generating the two-dimensional chest displacement matrix described above, and will not be repeated here.

[0054] After obtaining the two-dimensional matrix F_Sc of chest displacement, the two-dimensional matrix F_Sb of respiratory displacement, and the two-dimensional matrix F_Sh of heartbeat displacement, the two-dimensional matrix F_Sc of chest displacement, the two-dimensional matrix F_Sb of respiratory displacement, and the two-dimensional matrix F_Sh of heartbeat displacement are superimposed in sequence to generate a three-dimensional matrix.

[0055] To accurately generate the 3D matrix, the two-dimensional matrices of chest displacement, respiratory displacement, and heartbeat displacement are sequentially superimposed to form a 3D matrix. Specifically, because each chest displacement vector, respiratory vector, and heartbeat vector that makes up the chest displacement, respiratory displacement, and heartbeat displacement matrices has the same dimension, the resulting two-dimensional matrices are also of the same size; that is, each 2D matrix contains the same number of rows and columns. To generate the 3D matrix, the chest displacement, respiratory displacement, and heartbeat displacement matrices are respectively mapped to the R, G, and B channels. The mapping between which 2D matrix and which channel is assigned is randomized. Based on the 2D matrices corresponding to the R, G, and B channels, the components at corresponding positions in each channel are superimposed to obtain the 3D matrix. For example, each 2D matrix may have 10... When the matrix is ​​10, the resulting three-dimensional matrix is ​​10. 10 3. The component at each position in this three-dimensional matrix contains the component at the corresponding position in each two-dimensional component.

[0056] S302: Input the three-dimensional matrix into the pre-trained neural network model, obtain the state value output by the neural network model, and determine whether the user is in the detection state based on the state value.

[0057] In this embodiment, after generating the three-dimensional matrix, the three-dimensional matrix is ​​input into a pre-trained neural network model to obtain the state value output by the neural network model. Based on the state value, it is determined whether the user is in a detection state, where the state value can be a value between 0 and 1. The user's state can be determined based on the state value and a preset threshold. For example, 1 indicates that the user is fully in a detection state, and 0 indicates that the user is completely in a non-detection state. The threshold can be set to 0.7. If the state value is greater than 0.7, the user can be considered to be in a detection state.

[0058] In this embodiment, based on millimeter-wave radar data collection within a set time period, a first set number of chest displacement, respiratory displacement, and heartbeat displacement are collected to generate a first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector. Then, based on the first chest displacement vector, the first respiratory displacement vector, and the first heartbeat displacement vector, a second set number of components are obtained to generate each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector. These vectors are then superimposed to generate a three-dimensional matrix. The three-dimensional matrix is ​​input into a neural network model to obtain state values, thereby determining whether the user is in a detection state. This is because measuring breathing and heartbeat during movement also affects the detected breathing and heartbeat; therefore, based on the measured breathing and heartbeat, the user's current state can be accurately determined, thus improving the accuracy of the measurement.

[0059] To accurately update the first chest displacement vector, based on the above embodiments, in this embodiment, before obtaining the second predetermined number of components in the first chest displacement vector to generate each second chest displacement vector, the method further includes:

[0060] Based on the preset detection length, the average value of adjacent components in the first chest displacement vector that satisfy the detection length is calculated, and the first chest displacement vector is updated based on the chest displacement vector generated by each average value.

[0061] In this embodiment, to make the obtained first chest displacement vector more accurate, after obtaining the first chest displacement vector, a smoothing process can be performed on the first chest displacement vector based on each component in the first chest displacement vector. Specifically, a detection length can be preset, and the first chest displacement vector can be smoothed using this detection length. The length of the moving average window during smoothing is the preset detection length, which can be set to nm. Since the interval between every two adjacent components in the first chest displacement vector is fixed (tc), and the length of the first chest displacement vector is also fixed (a first preset number), the acquisition time can be determined, which can be T. By calculating the average value of adjacent components in the first chest displacement vector that satisfy the preset detection length, the first chest displacement vector is updated. T / tc / nm is the number of components contained in the smoothed first chest displacement vector, which is also the dimension of the smoothed first chest displacement vector.

[0062] For example, the first chest displacement vector can be (4, 6, 10, 16, 18, 20, 22, 26, 34, 44, 46, 52, 58, 64, 68, 60, 54, 50, 47, 43, 38, 34, 28, 22, 19, 17, 13, 11, 7, 5, 3, 1). The detection length is preset to 2. The first chest displacement vector is smoothed to update the first chest displacement vector. The updated first chest displacement vector is (5, 13, 19, 24, 39, 49, 61, 64, 52, 45, 36, 25, 18, 12, 6, 2).

[0063] Specifically, the first chest displacement vector can be smoothed using a pre-set moving average window. The length of the moving average window can be set to nm, so each average value can be calculated according to the following formula:

[0064] Sc(i) = mean(Sc0((i-1)) nm : i nm))

[0065] Where Sc(i) is the i-th chest displacement vector generated after smoothing according to the moving average window, mean() is the function to calculate the average value of adjacent components in the first chest displacement vector that satisfy the preset length of the moving average window, Sc0 is the first chest displacement vector, and Sc0((i-1)) is the first chest displacement vector. nm : i nm) is the component of the i-th moving average window in the first chest displacement vector.

[0066] To accurately segment the corresponding second chest displacement vector, based on the above embodiments, in this embodiment, obtaining a second predetermined number of components from the first chest displacement vector to generate each second chest displacement vector includes:

[0067] According to the preset second set number, the adjacent components in the first chest displacement vector that satisfy the second set number are divided into corresponding second chest displacement vectors.

[0068] In this embodiment, to accurately segment the first chest displacement vector into corresponding second chest displacement vectors, adjacent components satisfying the second preset number can be segmented into corresponding second chest displacement vectors according to a preset second set number. Specifically, a slice window tp is preset, and the first chest displacement vector is segmented through the slice window tp. The length of the slice window during segmentation can be tp, because the acquisition interval between every two adjacent components in the first chest displacement vector is fixed. Therefore, based on the length of tp, the second preset number of components contained in each slice window can be determined. By segmenting the adjacent components in the first chest displacement vector that satisfy the length of the slice window tp, the corresponding second chest displacement vector is obtained. A detection window for detecting the user's status can also be set. The length of this detection window can be tw, because the interval between every two adjacent components in the second chest displacement vector within the detection window is fixed and can be set to tc. Within each detection window tw, the second chest displacement vector will have nw = tw / tc / nm components.

[0069] For example, the first chest displacement vector is (6, 12, 19, 24, 36, 49, 60, 64, 52, 45, 36, 25, 16, 13, 7, 2). A second set number of slice windows with a pre-defined length is used, where the length of the slice window can be set to 4. The first chest displacement vector is then divided into corresponding second chest displacement vectors, and the resulting second chest displacement vectors are (6, 12, 19, 24), (36, 49, 60, 64), (52, 45, 36, 25), and (16, 13, 7, 2).

[0070] The obtained second chest displacement vector is subjected to Fast Fourier Transform (FFT) to obtain a chest displacement vector of the same dimension, FFT_Sc. According to the properties of FFT, the vector FFT_Sc is symmetrical from left to right. Therefore, to save computation, only the first half of the chest displacement vector is taken, which can be denoted as fft_Sc, fft_Sc = FFT_sc(1:np / 2). This vector is used to update the second chest displacement vector. After the first second chest displacement vector is updated, the slice window tp can be moved to the right by one step tf, where tf is the slice movement step and the slice movement step length is half the length of the slice window, i.e., tf = tp / 2. The above operation is repeated for Sc(np / 2 : 3np / 2) to generate a new vector fft_Sc.

[0071] For ease of explanation, Figure 4 A schematic diagram illustrating the process of generating each second chest displacement vector by obtaining a second predetermined number of components from the first chest displacement vector, as provided in some embodiments of this application, is shown below. Figure 4 The process includes the following steps:

[0072] S401: Based on the first set number of chest displacements collected by millimeter-wave radar within a set time period, generate a first chest displacement vector according to the first set number of chest displacements.

[0073] S402: Based on the preset detection length, calculate the average value of adjacent components in the first chest displacement vector that satisfy the detection length, and update the first chest displacement vector based on the chest displacement vector generated by each average value.

[0074] S403: According to the preset second set number, the adjacent components in the first chest displacement vector that satisfy the second set number are divided into corresponding second chest displacement vectors.

[0075] To accurately update the first respiratory displacement vector, based on the above embodiments, in this embodiment of the application, before obtaining the second predetermined number of components in the first respiratory displacement vector to generate each second respiratory displacement vector, the method further includes:

[0076] Based on the preset detection length, the average value of adjacent components in the first respiratory displacement vector that satisfy the detection length is calculated, and the first respiratory displacement vector is updated based on the respiratory displacement vector generated by each average value.

[0077] In this embodiment, to make the obtained first respiratory displacement vector more accurate, after obtaining the first respiratory displacement vector, a smoothing process can be performed on the first respiratory displacement vector based on each component in the first respiratory displacement vector. Specifically, a detection length can be preset, and the first respiratory displacement vector can be smoothed using this detection length. The length of the moving average window during smoothing is the preset detection length, which can be set to nm. Since the interval between every two adjacent components in the first respiratory displacement vector is fixed (tc), and the length of the first respiratory displacement vector is also fixed (a first preset number), the acquisition time can be determined, which can be T. By calculating the average value of adjacent components in the first respiratory displacement vector that satisfy the preset detection length, the first respiratory displacement vector is updated. T / tc / nm is the number of components contained in the smoothed first respiratory displacement vector, which is also the dimension of the smoothed first chest displacement vector.

[0078] For example, the first respiratory displacement vector can be (5, 9, 14, 16, 21, 25, 28, 30, 36, 44, 49, 53, 57, 65, 69, 61, 50, 48, 43, 39, 32, 26, 20, 18, 15, 13, 12, 10, 7, 5, 3, 1). The preset detection length is 2. The first respiratory displacement vector is smoothed to update it. The updated first respiratory displacement vector is (7, 15, 23, 29, 40, 51, 61, 65, 49, 41, 29, 19, 14, 11, 6, 2).

[0079] Specifically, the process of determining the vector generated based on each average value according to the average value calculation formula has been described in the above embodiments and will not be repeated here.

[0080] To accurately segment the corresponding second respiratory displacement vector, based on the above embodiments, in this embodiment, the step of obtaining a second predetermined number of components in the first respiratory displacement vector to generate each second respiratory displacement vector includes:

[0081] According to the preset second set number, the adjacent components in the first respiratory displacement vector that satisfy the second set number are divided into corresponding second respiratory displacement vectors.

[0082] In this embodiment, to accurately segment the first respiratory displacement vector into a corresponding second respiratory displacement vector, adjacent components that satisfy the second preset number can be segmented into corresponding second respiratory displacement vectors according to a preset second set number. Specifically, a slicing window is preset, and the first respiratory displacement vector is segmented through the slicing window. The length of the slicing window when segmenting the first respiratory displacement vector is a preset second set number. By segmenting adjacent components of the first respiratory displacement vector that satisfy the length of the slicing window, the corresponding second respiratory displacement vector is obtained.

[0083] The obtained second respiratory displacement vector is subjected to Fast Fourier Transform (FFT) to obtain a respiratory displacement vector of the same dimension, FFT_Sb. According to the properties of FFT, the vector FFT_Sb is symmetrical from left to right. Therefore, to save computation, only the first half of the respiratory displacement vector is taken, which can be denoted as fft_Sb, fft_Sb=FFT_sb(1:np / 2). This vector is used to update the second respiratory displacement vector. After the first second respiratory displacement vector is updated, the slice window tp can be moved to the right by one step tf, where tf is the slice movement step and the slice movement step length is half the length of the slice window, i.e., tf=tp / 2. The above operation is repeated for Sb(np / 2 : 3np / 2) to generate a new vector fft_Sb.

[0084] For ease of explanation, Figure 5 A schematic diagram illustrating the process of generating each second respiratory displacement vector by obtaining a second predetermined number of components in a first respiratory displacement vector, as provided in some embodiments of this application, is shown below. Figure 5 As shown, the process includes the following steps:

[0085] S501: Based on the first set number of respiratory displacements collected by millimeter-wave radar within a set time period, generate a first respiratory displacement vector according to the first set number of respiratory displacements;

[0086] S502: Based on the preset detection length, calculate the average value of adjacent components in the first respiratory displacement vector that satisfy the detection length, and update the first respiratory displacement vector based on the respiratory displacement vector generated by each average value;

[0087] S503: According to the preset second set number, the adjacent components in the first respiratory displacement vector that satisfy the second set number are divided into corresponding second respiratory displacement vectors.

[0088] To accurately update the first heartbeat displacement vector, based on the above embodiments, in this embodiment of the application, before obtaining the second predetermined number of components in the first heartbeat displacement vector to generate each second heartbeat displacement vector, the method further includes:

[0089] Based on the preset detection length, the average value of adjacent components in the first heartbeat displacement vector that satisfy the detection length is calculated, and the first heartbeat displacement vector is updated based on the heartbeat displacement vector generated by each average value.

[0090] In this embodiment, to make the obtained first heartbeat displacement vector more accurate, after obtaining the first heartbeat displacement vector, a smoothing process can be performed on the first heartbeat displacement vector based on each component in the vector. Specifically, a detection length can be preset, and the first heartbeat displacement vector can be smoothed using this detection length. The length of the moving average window during smoothing is the preset detection length, which can be set to nm. Since the interval between any two adjacent components in the first heartbeat displacement vector is fixed (tc), and the length of the first heartbeat displacement vector is also fixed (a first preset number), the acquisition duration can be determined, which can be T. By calculating the average value of adjacent components in the first heartbeat displacement vector that satisfy the preset detection length, the first heartbeat displacement vector is updated. T / tc / nm represents the number of components preserved in the smoothed first heartbeat displacement vector, which is also the dimension of the smoothed first heartbeat displacement vector.

[0091] For example, the first heartbeat displacement vector can be (2, 4, 12, 18, 22, 26, 30, 34, 38, 42, 46, 50, 53, 59, 63, 71, 67, 59, 52, 44, 38, 34, 29, 25, 22, 18, 15, 11, 9, 5, 4, 2). The detection length is preset to 2. The first heartbeat displacement vector is smoothed to update the first heartbeat displacement vector. The updated first heartbeat displacement vector is (3, 15, 24, 32, 40, 48, 56, 67, 63, 48, 36, 27, 20, 13, 7, 3).

[0092] Specifically, the process of determining the vector generated based on each average value according to the average value calculation formula has been described in the above embodiments and will not be repeated here.

[0093] To accurately segment the corresponding second heartbeat displacement vector, based on the above embodiments, in this embodiment, obtaining a second predetermined number of components from the first heartbeat displacement vector to generate each second heartbeat displacement vector includes:

[0094] According to the preset second set number, the adjacent components in the first heartbeat displacement vector that satisfy the second set number are divided into corresponding second heartbeat displacement vectors.

[0095] In this embodiment, to accurately segment the first heartbeat displacement vector into corresponding second heartbeat displacement vectors, adjacent components that satisfy the second preset number can be segmented into corresponding second heartbeat displacement vectors according to a preset second set number. Specifically, a slicing window is preset, and the first heartbeat displacement vector is segmented through the slicing window. The length of the slicing window when segmenting the first heartbeat displacement vector is a preset second set number. By segmenting adjacent components in the first heartbeat displacement vector that satisfy the length of the slicing window, the corresponding second heartbeat displacement vector is obtained.

[0096] The obtained second heartbeat displacement vector is subjected to Fast Fourier Transform (FFT) to obtain a heartbeat displacement vector of the same dimension, FFT_Sh. According to the properties of FFT, the vector FFT_Sh is symmetrical from left to right. Therefore, to save computation, only the first half of the heartbeat displacement vector is taken, which can be denoted as fft_Sh, fft_Sh=FFT_sh(1:np / 2). This vector is used to update the second heartbeat displacement vector. After the first second heartbeat displacement vector is updated, the slice window tp can be moved to the right by one step tf, where tf is the slice movement step, and the slice movement step length is half the length of the slice window, i.e., tf=tp / 2. The above operation is repeated for Sh(np / 2 : 3np / 2) to generate a new vector fft_Sh.

[0097] For ease of explanation, Figure 6 A schematic diagram illustrating the process of generating each second heartbeat displacement vector by obtaining a second predetermined number of components from the first heartbeat displacement vector, as provided in some embodiments of this application, is shown below. Figure 6 As shown, the process includes the following steps:

[0098] S601: Based on the first set number of heartbeat displacements collected by millimeter-wave radar within a set time period, generate a first heartbeat displacement vector according to the first set number of heartbeat displacements;

[0099] S602: Based on the preset detection length, calculate the average value of adjacent components in the first heartbeat displacement vector that satisfy the detection length, and update the first heartbeat displacement vector based on the heartbeat displacement vector generated by each average value;

[0100] S603: According to the preset second set number, the adjacent components in the first heartbeat displacement vector that satisfy the second set number are divided into corresponding second heartbeat displacement vectors.

[0101] In order to accurately determine the state value output by the neural network model, based on the above embodiments, the training process of the neural network model in this embodiment includes:

[0102] Obtain any sample data from the sample set, wherein the sample data includes a sample three-dimensional matrix generated based on a first set number of sample chest displacement, sample respiratory displacement and sample heartbeat displacement, and a label value corresponding to the sample three-dimensional matrix indicating whether it is a detection state;

[0103] The sample's three-dimensional matrix and its corresponding label values ​​are input into the original neural network model to obtain the predicted state values ​​output by the original neural network model.

[0104] The parameters in the original neural network model are adjusted based on the label value and the predicted state value.

[0105] In this embodiment of the application, any sample data in the sample set is obtained, and the obtained sample data in the sample set is trained based on a CNN network. The sample data includes a sample three-dimensional matrix generated according to a first set number of sample chest displacement, sample respiratory displacement and sample heartbeat displacement, and a label value corresponding to whether the sample three-dimensional matrix is ​​a detection state. The sample three-dimensional matrix and its corresponding label value are input into the original neural network model to obtain the predicted state value output by the original neural network model. The parameters in the original neural network model are adjusted according to the label value and the predicted state value.

[0106] Specifically, a minimum threshold δ for the cross-entropy loss function can be preset. The sample 3D matrix and its corresponding label values ​​are input into the original neural network model. The predicted state value output by the original neural network model is obtained, and the difference between the predicted state value and the label value is calculated to determine the loss value of the cross-entropy loss function. The predicted state value can be a value between 0 and 1, for example, 1 indicates that the user is fully in the detection state, and 0 indicates that the user is fully in the non-detection state. The smaller the loss value of the cross-entropy loss function, the closer the predicted state value is to the label value of the detected state.

[0107] Specifically, the loss value is determined based on the label value and the predicted state value. The smaller the loss value, the closer the predicted state value is to the label value of the detected state. For example, the vector corresponding to the label value of the detected state as 1 can be set to (0,1) in advance. If the predicted state value is 0.5, the corresponding predicted vector can be (0,0.5). The loss value can be determined based on the distance between the vectors. If the loss value is greater than the preset minimum threshold δ, the corresponding gradient can be calculated based on the label value and the predicted state value, and the parameters in the original neural network model can be adjusted. The calculation can be performed using the following formula:

[0108] W = W – I G

[0109] Where W represents the parameters in the original neural network model, I represents the iteration step size or training step size, and G represents the gradient of the loss function with respect to the parameters in the original neural network model.

[0110] After adjusting the parameters in the original neural network model, the model is trained again. Based on the cross-entropy loss function, the loss value is determined, and the parameters in the original neural network model are adjusted until the convergence condition is met.

[0111] After the neural network model is trained, it is determined whether the user is in a detection state based on the predicted state value; if the state value is greater than a set threshold, it is determined that the user is in a detection state.

[0112] Figure 7 A schematic diagram of the CNN network model structure provided for some embodiments of this application, such as... Figure 7 As shown, the CNN network model consists of an input layer, a convolutional layer, pooling layer 1, pooling layer 2, pooling layer 3, pooling layer 4, fully connected layer 1, fully connected layer 2, and an output layer. The input image (RGB) is an RGB image of chest displacement, respiratory displacement, or heartbeat displacement, and the size of this RGB image is 224. 224 3, where 3 represents the depth of the input image, and the filter size is 11. 11, stride of 4 represents a stride of 4. A convolutional layer contains multiple neurons, each connected only to a local region of the input image. The size of this connection space is called the neuron's filter. The filter's depth must be the same as the input image's depth; therefore, each neuron in the convolutional layer will have 11... 11 3 = 363 weights, which can be used to obtain a 55 by convolving the filter with the input image. 55 The feature map is 96, and the pooling layer compresses the feature map using max pooling. The filter size connected to the convolutional layer and the pooling layer is 5. 5, where Max Pooling selects the maximum value of an image region as the pooled value for that region, resulting in a value of 27 after compression. 27 The feature map is 256, and the filter size connecting pooling layer 1 and pooling layer 2 is 3. 3. The feature map is compressed using a Max Pooling layer, resulting in a 13-bit version. 13 The feature map is 384, and the filter size connecting pooling layer 2 and pooling layer 3 is 3. 3. Utilizing the spatial invariance of pooling, the compressed feature map is input into the third pooling layer to obtain 13... 13 The feature map is 384, and the filter size connecting pooling layer 3 and pooling layer 4 is 3. 3. The feature map is compressed using a Max Pooling layer, resulting in a 13-bit version. 13 The feature map of 256 is input into the first fully connected layer, and the 13 feature map is used to generate the feature map. 13 256 converted to 1 4096, all nodes of fully connected layer 1 and fully connected layer 2 are connected, and the final feature is input into the output layer, and the output layer outputs the state value.

[0113] For ease of explanation, Figure 8 A schematic diagram illustrating the training process of a neural network model provided for some embodiments of this application, such as... Figure 8 As shown, the process includes the following steps:

[0114] S801: Obtain any sample data in the sample set. The sample data includes a sample three-dimensional matrix generated based on a first set number of sample chest displacement, sample respiratory displacement and sample heartbeat displacement, and a label value corresponding to the sample three-dimensional matrix indicating whether it is a detection state.

[0115] S802: Input the three-dimensional matrix of the sample and its corresponding label values ​​into the original neural network model to obtain the predicted state value output by the original neural network model;

[0116] S803: Adjust the parameters in the original neural network model based on the label value and the predicted state value.

[0117] Figure 9 Flowcharts for determining behavioral states provided for some embodiments of this application, such as Figure 9 As shown, the process includes the following steps:

[0118] (1) Dimension reduction of the three elements sequence of chest displacement, respiration and heartbeat, that is, after collecting a first set number of chest displacement, respiration and heartbeat displacements within a set time period based on millimeter-wave radar, smoothing the first chest displacement vector, first respiration vector and first heartbeat displacement vector generated based on the first set number of chest displacement, respiration and heartbeat displacements. When smoothing, the detection length can be preset, and the length of the sliding average window is the preset detection length. By calculating the average value of the adjacent components of the first chest displacement vector, first respiration vector and first heartbeat vector that satisfy the preset detection length, the first chest displacement vector, first respiration vector and first heartbeat vector are updated.

[0119] (2) Set the detection window duration, slice duration, and slice movement step duration, that is, set a window with a length of tw to detect the user's status, and set a slice window for the updated first chest displacement vector, first respiratory displacement vector, and first heartbeat displacement vector. The length of the slice window can be tp. The first chest displacement vector, first respiratory displacement vector, and first heartbeat displacement vector are segmented through the slice window. The adjacent components of the first chest displacement vector, first respiratory displacement vector, and first heartbeat displacement vector that satisfy the second set number are respectively segmented into the corresponding second chest displacement vector, second respiratory displacement vector, and second heartbeat displacement vector.

[0120] (3) Perform Fast Fourier Transform (FFT) on the second chest displacement vector, second respiratory displacement vector, and second heartbeat displacement vector of each slice window to generate chest displacement vector, respiratory displacement vector, and heartbeat displacement vector of the same dimension. According to the properties of FFT, the chest displacement vector, respiratory displacement vector, and heartbeat displacement vector are all symmetrical. We only need to take the first half of the chest displacement vector FFT_Sc, respiratory displacement vector FFT_Sb, and heartbeat displacement vector FFT_Sh, which can be denoted as fft_Sc, fft_Sb, and fft_Sb, respectively. b. `fft_Sh`, where `fft_Sc = FFT_sc(1:np / 2)`, `fft_Sb = FFT_sb(1:np / 2)`, and `fft_Sh = FFT_sh(1:np / 2)`, is used to update the second respiratory displacement vector. After the first second respiratory displacement vector is updated, the slice window `tp` can be moved to the right by one step `tf`, where `tf` is the slice movement step size, and the slice movement step size is half the length of the slice window, i.e., `tf = tp / 2`. This is then applied to `Sc(np / 2)`. Repeat the above operations (3np / 2), Sb(np / 2 : 3np / 2), Sh(np / 2 : 3np / 2) to generate new vectors fft_Sc, fft_Sb, and fft_Sh. Concatenate each new vector fft_Sc to generate a two-dimensional matrix of chest displacement, concatenate each new vector fft_Sb to generate a two-dimensional matrix of respiratory displacement, and concatenate each new vector fft_Sh to generate a two-dimensional matrix of heartbeat displacement. Superimpose the two-dimensional matrices of chest displacement, respiratory displacement, and heartbeat displacement in sequence to generate a three-dimensional matrix.

[0121] (4) Create samples in the detection state and the non-detection state, obtain any sample data in the sample set, the sample data includes a sample three-dimensional matrix generated according to the first set number of sample chest displacement, sample breathing displacement and sample heartbeat displacement, and the label value corresponding to the sample three-dimensional matrix as whether it is in the detection state, input the sample three-dimensional matrix and its corresponding label value into the original neural network model, and obtain the predicted state value output by the original neural network model.

[0122] (5) Generate a three-dimensional matrix of the chest displacement, respiratory displacement and heartbeat displacement within a set time period based on millimeter-wave radar acquisition, which need to be distinguished, according to the above steps, input it into the pre-trained neural network model, obtain the state value output by the neural network model, and determine whether the user is in the detection state based on the state value; if the state value is greater than the set threshold, it is determined that the user is in the detection state.

[0123] Based on the same technical concept, this application provides a behavior state determination device. Figure 10A schematic diagram of a behavior state determination device is provided for some embodiments of this application, such as... Figure 10 As shown, the device includes:

[0124] The generation module 1001 is used to generate a first set number of chest displacement, respiratory displacement, and heartbeat displacement data within a set time period based on millimeter-wave radar data acquisition; generate a first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector based on the first set number of chest displacement, respiratory displacement, and heartbeat displacement data, respectively; obtain a second set number of components from the first chest displacement vector to generate each second chest displacement vector; obtain a second set number of components from the first respiratory displacement vector to generate each second respiratory displacement vector; obtain a second set number of components from the first heartbeat displacement vector to generate each second heartbeat displacement vector; and superimpose each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector to generate a three-dimensional matrix.

[0125] The determination module 1002 is used to input the three-dimensional matrix into a pre-trained neural network model, obtain the state value output by the neural network model, and determine whether the user is in a detection state based on the state value; if the state value is greater than a set threshold, then it is determined that the user is in a detection state.

[0126] In one possible implementation, the device further includes:

[0127] The update module is used to calculate the average value of adjacent components in the first chest displacement vector that satisfy the preset detection length, and update the first chest displacement vector based on the chest displacement vector generated by each average value.

[0128] In one possible implementation, the device further includes:

[0129] The segmentation module is used to segment adjacent components in the first chest displacement vector that satisfy the second preset number into corresponding second chest displacement vectors according to the preset second preset number.

[0130] In one possible implementation, the updating module is further configured to calculate the average value of adjacent components in the first respiratory displacement vector that satisfy the detection length according to a preset detection length, and update the first respiratory displacement vector according to the respiratory displacement vector generated by obtaining each average value.

[0131] In one possible implementation, the segmentation module is further configured to segment adjacent components in the first respiratory displacement vector that satisfy the second preset number into corresponding second respiratory displacement vectors according to a preset second preset number.

[0132] In one possible implementation, the updating module is further configured to calculate the average value of adjacent components in the first heartbeat displacement vector that satisfy the detection length according to a preset detection length, and update the first heartbeat displacement vector according to the heartbeat displacement vector generated by obtaining each average value.

[0133] In one possible implementation, the segmentation module is further configured to segment adjacent components in the first heartbeat displacement vector that satisfy the second preset number into corresponding second heartbeat displacement vectors according to a preset second preset number.

[0134] In one possible implementation, the device further includes:

[0135] The training module is used to acquire any sample data in the sample set. The sample data includes a sample three-dimensional matrix generated based on a first set number of sample chest displacement, sample respiratory displacement and sample heartbeat displacement, and a label value corresponding to whether the sample three-dimensional matrix is ​​a detection state.

[0136] The sample's three-dimensional matrix and its corresponding label values ​​are input into the original neural network model to obtain the predicted state values ​​output by the original neural network model.

[0137] The parameters in the original neural network model are adjusted based on the label value and the predicted state value.

[0138] In this embodiment, based on millimeter-wave radar data collection within a set time period, a first set number of chest displacement, respiratory displacement, and heartbeat displacement are collected to generate a first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector. Then, based on the first chest displacement vector, the first respiratory displacement vector, and the first heartbeat displacement vector, a second set number of components are obtained to generate each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector. These vectors are then superimposed to generate a three-dimensional matrix. The three-dimensional matrix is ​​input into a neural network model to obtain state values, thereby determining whether the user is in a detection state. This is because measuring breathing and heartbeat during movement also affects the detected breathing and heartbeat; therefore, based on the measured breathing and heartbeat, the user's current state can be accurately determined, thus improving the accuracy of the measurement.

[0139] Based on the same technical concept, this application also provides an electronic device. Figure 11 A schematic diagram of an electronic device structure is provided for some embodiments of this application, such as... Figure 11As shown, it includes: processor 1101, communication interface 1102, memory 1103 and communication bus 1104, wherein processor 1101, communication interface 1102 and memory 1103 communicate with each other through communication bus 1104.

[0140] The memory 1103 stores a computer program, which, when executed by the processor 1101, causes the processor 1101 to perform the following steps:

[0141] Based on millimeter-wave radar data collection within a set time period, a first predetermined number of chest displacement, respiratory displacement, and heartbeat displacement are collected. A first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector are generated based on these measurements. A second predetermined number of components from the first chest displacement vector are then extracted to generate each second chest displacement vector. Similarly, a second predetermined number of components from the first respiratory displacement vector and a second predetermined number of components from the first heartbeat displacement vector are extracted to generate each second heartbeat displacement vector. Finally, each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector are superimposed to generate a three-dimensional matrix.

[0142] The three-dimensional matrix is ​​input into a pre-trained neural network model to obtain the state value output by the neural network model. Based on the state value, it is determined whether the user is in a detection state.

[0143] If the status value is greater than the set threshold, it is determined that the user is in a detection state.

[0144] In one possible implementation, the processor 1101 is further configured to, before generating each second chest displacement vector by acquiring a second predetermined number of components in the first chest displacement vector, further include:

[0145] Based on the preset detection length, the average value of adjacent components in the first chest displacement vector that satisfy the detection length is calculated, and the first chest displacement vector is updated based on the chest displacement vector generated by each average value.

[0146] In one possible implementation, the processor 1101 is further configured to generate each second chest displacement vector by obtaining a second predetermined number of components in the first chest displacement vector, including:

[0147] According to the preset second set number, the adjacent components in the first chest displacement vector that satisfy the second set number are divided into corresponding second chest displacement vectors.

[0148] In one possible implementation, the processor 1101 is further configured to, before generating each second respiratory displacement vector by acquiring a second predetermined number of components in the first respiratory displacement vector, further include:

[0149] Based on the preset detection length, the average value of adjacent components in the first respiratory displacement vector that satisfy the detection length is calculated, and the first respiratory displacement vector is updated based on the respiratory displacement vector generated by each average value.

[0150] In one possible implementation, the processor 1101 is further configured to generate each second respiratory displacement vector by obtaining a second predetermined number of components in the first respiratory displacement vector, including:

[0151] According to the preset second set number, the adjacent components in the first respiratory displacement vector that satisfy the second set number are divided into corresponding second respiratory displacement vectors.

[0152] In one possible implementation, the processor 1101 is further configured to, before generating each second heartbeat displacement vector by acquiring a second predetermined number of components in the first heartbeat displacement vector, further include:

[0153] Based on the preset detection length, the average value of adjacent components in the first heartbeat displacement vector that satisfy the detection length is calculated, and the first heartbeat displacement vector is updated based on the heartbeat displacement vector generated by each average value.

[0154] In one possible implementation, the processor 1101 is further configured to generate each second heartbeat displacement vector by obtaining a second predetermined number of components in the first heartbeat displacement vector, including:

[0155] According to the preset second set number, the adjacent components in the first heartbeat displacement vector that satisfy the second set number are divided into corresponding second heartbeat displacement vectors.

[0156] In one possible implementation, the processor 1101 is further configured to perform the training process of the neural network model, including:

[0157] Obtain any sample data from the sample set, wherein the sample data includes a sample three-dimensional matrix generated based on a first set number of sample chest displacement, sample respiratory displacement and sample heartbeat displacement, and a label value corresponding to the sample three-dimensional matrix indicating whether it is a detection state;

[0158] The sample's three-dimensional matrix and its corresponding label values ​​are input into the original neural network model to obtain the predicted state values ​​output by the original neural network model.

[0159] The parameters in the original neural network model are adjusted based on the label value and the predicted state value.

[0160] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0161] Communication interface 1102 is used for communication between the above-mentioned electronic device and other devices.

[0162] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0163] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0164] Based on the same technical concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by an electronic device. When the program is run on the electronic device, the electronic device performs the following steps:

[0165] Based on millimeter-wave radar data collection within a set time period, a first predetermined number of chest displacement, respiratory displacement, and heartbeat displacement are collected. A first chest displacement vector, a first respiratory displacement vector, and a first heartbeat displacement vector are generated based on these measurements. A second predetermined number of components from the first chest displacement vector are then extracted to generate each second chest displacement vector. Similarly, a second predetermined number of components from the first respiratory displacement vector and a second predetermined number of components from the first heartbeat displacement vector are extracted to generate each second heartbeat displacement vector. Finally, each second chest displacement vector, each second respiratory displacement vector, and each second heartbeat displacement vector are superimposed to generate a three-dimensional matrix.

[0166] The three-dimensional matrix is ​​input into a pre-trained neural network model to obtain the state value output by the neural network model. Based on the state value, it is determined whether the user is in a detection state.

[0167] If the status value is greater than the set threshold, it is determined that the user is in a detection state.

[0168] In one possible implementation, before obtaining a second predetermined number of components from the first chest displacement vector to generate each second chest displacement vector, the method further includes:

[0169] Based on the preset detection length, the average value of adjacent components in the first chest displacement vector that satisfy the detection length is calculated, and the first chest displacement vector is updated based on the chest displacement vector generated by each average value.

[0170] In one possible implementation, obtaining a second predetermined number of components in the first chest displacement vector to generate each second chest displacement vector includes:

[0171] According to the preset second set number, the adjacent components in the first chest displacement vector that satisfy the second set number are divided into corresponding second chest displacement vectors.

[0172] In one possible implementation, before generating each second respiratory displacement vector by obtaining a second predetermined number of components in the first respiratory displacement vector, the method further includes:

[0173] Based on the preset detection length, the average value of adjacent components in the first respiratory displacement vector that satisfy the detection length is calculated, and the first respiratory displacement vector is updated based on the respiratory displacement vector generated by each average value.

[0174] In one possible implementation, obtaining a second predetermined number of components in the first respiratory displacement vector to generate each second respiratory displacement vector includes:

[0175] According to the preset second set number, the adjacent components in the first respiratory displacement vector that satisfy the second set number are divided into corresponding second respiratory displacement vectors.

[0176] In one possible implementation, before generating each second heartbeat displacement vector by obtaining a second predetermined number of components from the first heartbeat displacement vector, the method further includes:

[0177] Based on the preset detection length, the average value of adjacent components in the first heartbeat displacement vector that satisfy the detection length is calculated, and the first heartbeat displacement vector is updated based on the heartbeat displacement vector generated by each average value.

[0178] In one possible implementation, obtaining a second predetermined number of components in the first heartbeat displacement vector to generate each second heartbeat displacement vector includes:

[0179] According to the preset second set number, the adjacent components in the first heartbeat displacement vector that satisfy the second set number are divided into corresponding second heartbeat displacement vectors.

[0180] In one possible implementation, the training process of the neural network model includes:

[0181] Obtain any sample data from the sample set, wherein the sample data includes a sample three-dimensional matrix generated based on a first set number of sample chest displacement, sample respiratory displacement and sample heartbeat displacement, and a label value corresponding to the sample three-dimensional matrix indicating whether it is a detection state;

[0182] The sample's three-dimensional matrix and its corresponding label values ​​are input into the original neural network model to obtain the predicted state values ​​output by the original neural network model.

[0183] The parameters in the original neural network model are adjusted based on the label value and the predicted state value.

[0184] The aforementioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in an electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), optical storage such as CDs, DVDs, BDs, HVDs, etc., and semiconductor storage such as ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.

[0185] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0186] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0187] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0188] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0189] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining behavioral states, characterized in that, The method includes: Based on the first set number of chest displacement, respiratory displacement and heartbeat displacement collected by millimeter-wave radar within a set time period, a first chest displacement vector, a first respiratory displacement vector and a first heartbeat displacement vector are generated according to the first set number of chest displacement, respiratory displacement and heartbeat displacement. According to the preset detection length, the average value of adjacent components in the first chest displacement vector that satisfy the detection length is calculated, and the first chest displacement vector is updated based on the chest displacement vector generated by each average value; a second set number of components in the first chest displacement vector are obtained to generate each second chest displacement vector. According to the preset detection length, the average value of adjacent components in the first respiratory displacement vector that satisfy the detection length is calculated, and the first respiratory displacement vector is updated based on the respiratory displacement vector generated by each average value; a second set number of components in the first respiratory displacement vector are obtained to generate each second respiratory displacement vector; Based on a preset detection length, the average value of adjacent components in the first heartbeat displacement vector that satisfy the detection length is calculated, and the first heartbeat displacement vector is updated based on the heartbeat displacement vector generated by each average value; a second set number of components in the first heartbeat displacement vector are obtained to generate each second heartbeat displacement vector; Each second chest displacement vector is concatenated into a two-dimensional chest displacement matrix, each second respiratory displacement vector is concatenated into a two-dimensional respiratory displacement matrix, and each second heartbeat displacement vector is concatenated into a two-dimensional heartbeat displacement matrix. The two-dimensional chest displacement matrix, the two-dimensional respiratory displacement matrix, and the two-dimensional heartbeat displacement matrix are then superimposed in sequence to generate a three-dimensional matrix. The three-dimensional matrix is ​​input into a pre-trained neural network model to obtain the state value output by the neural network model. Based on the state value, it is determined whether the user is in a detection state. If the status value is greater than the set threshold, it is determined that the user is in a detection state.

2. The method according to claim 1, characterized in that, The step of obtaining a second predetermined number of components from the first chest displacement vector to generate each second chest displacement vector includes: According to the preset second set number, the adjacent components in the first chest displacement vector that satisfy the second set number are divided into corresponding second chest displacement vectors.

3. The method according to claim 1, characterized in that, The step of obtaining a second predetermined number of components in the first respiratory displacement vector to generate each second respiratory displacement vector includes: According to the preset second set number, the adjacent components in the first respiratory displacement vector that satisfy the second set number are divided into corresponding second respiratory displacement vectors.

4. The method according to claim 1, characterized in that, The step of obtaining a second predetermined number of components in the first heartbeat displacement vector to generate each second heartbeat displacement vector includes: According to the preset second set number, the adjacent components in the first heartbeat displacement vector that satisfy the second set number are divided into corresponding second heartbeat displacement vectors.

5. The method according to claim 1, characterized in that, The training process of the neural network model includes: Obtain any sample data from the sample set, wherein the sample data includes a sample three-dimensional matrix generated based on a first set number of sample chest displacement, sample respiratory displacement and sample heartbeat displacement, and a label value corresponding to the sample three-dimensional matrix indicating whether it is a detection state; The sample's three-dimensional matrix and its corresponding label values ​​are input into the original neural network model to obtain the predicted state values ​​output by the original neural network model. The parameters in the original neural network model are adjusted based on the label value and the predicted state value.

6. An electronic device, characterized in that, The electronic device includes at least a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of a behavior state determination method as described in any one of claims 1-5.

7. A computer storage medium, characterized in that, It stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a behavior state determination method according to any one of claims 1-5.

Images