Daily behavior detection method, device, equipment and computer storage medium

Abstract

The application relates to the computer technical field and discloses a daily behavior detection method, device and equipment. The method comprises the following steps: obtaining daily behavior data of a user to be detected; obtaining a local feature signal corresponding to the daily behavior data according to the daily behavior data, and obtaining a global feature signal corresponding to the daily behavior data according to the local feature signal; obtaining a corresponding fusion feature according to the local feature signal and the global feature signal; inputting the fusion feature into a preset daily behavior detection model; and outputting the daily behavior of the user to be detected. The application can better fuse the continuity of the local feature signal and the globality of the global feature signal, so that the daily behavior of the user can be more accurately recognized. In addition, the process of recognizing the daily behavior of the user does not require a specific sensor to be installed at a specific position, so the application has a wider application range and stronger applicability.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and computer storage medium for detecting everyday behaviors. Background Technology

[0002] With the aging population becoming a widespread phenomenon, how to better care for the health of the elderly has become a key issue that society needs to focus on. Understanding the daily behavior of the elderly plays an important role in caring for their health.

[0003] Currently, the identification of people's daily behaviors can be broadly categorized into video-based behavior recognition and behavior recognition based on interactive environmental sensors. Video-based behavior recognition primarily uses thermal imaging and depth cameras to analyze behavioral characteristics in videos or images for behavior identification. While video-based behavior recognition is generally very effective and plays a role in monitoring and protecting the elderly, it raises concerns about privacy and its effectiveness is often less than ideal in environments with changing lighting conditions. Interactive environmental sensor-based behavior recognition involves sensors embedded in the ceiling, floor, and objects of smart homes to track and locate the user's position, interact with the user, and collect information about their behavior. This method is widely used in smart home environments and behavior recognition projects, but its applicability is limited by geographical constraints. Therefore, improving the accuracy and applicability of daily behavior detection has become an important research topic for researchers in this field. Summary of the Invention

[0004] The main technical problem addressed by the embodiments of this application is how to improve the accuracy and applicability of daily behavior detection.

[0005] To address the aforementioned technical problems, one technical solution adopted in this application is: providing a daily behavior detection method, comprising: acquiring daily behavior data of a user to be detected; obtaining local feature signals corresponding to the daily behavior data based on the daily behavior data, and obtaining global feature signals corresponding to the daily behavior data based on the local feature signals; obtaining corresponding fused features based on the local feature signals and the global feature signals; inputting the fused features into a preset daily behavior detection model, and outputting the daily behavior of the user to be detected.

[0006] Optionally, after performing the step of acquiring the daily behavior data of the user to be detected, the method further includes: preprocessing the daily behavior data; the preprocessing of the daily behavior data includes: performing frequency synchronization processing on the daily behavior data so that the data acquisition frequency of the daily behavior data is within a preset frequency range; and performing synchronization processing on the daily behavior data so that the timestamp difference of the daily behavior data collected by different sensors is within a preset time range.

[0007] Optionally, obtaining the local feature signal corresponding to the daily behavior data based on the daily behavior data includes: obtaining the angular velocity and acceleration of the user to be detected based on the daily behavior data; obtaining the detection time corresponding to the angular velocity and the acceleration, and constructing a time series signal, wherein the time series signal is the local feature signal.

[0008] Optionally, obtaining the global feature signal corresponding to the daily behavior data based on the local feature signal includes: normalizing the data corresponding to the time series signal; converting the normalized time series signal data into polar coordinate data; and converting the polar coordinate data into a two-dimensional image that retains the time features.

[0009] Optionally, obtaining the corresponding fused features based on the local feature signals and the global feature signals includes: extracting features from the local feature signals based on the principle of a multilayer perceptron to obtain features corresponding to the local feature signals; extracting features from the global feature signals based on the principle of a convolutional neural network to obtain features corresponding to the global feature signals; and fusing the features corresponding to the local feature signals and the features corresponding to the global feature signals to obtain fused features.

[0010] Optionally, the step of inputting the fused features into a preset daily behavior detection model and outputting the daily behavior of the user to be detected includes: obtaining the predicted probability of the user to be detected belonging to each daily behavior category based on the fused features and the preset daily behavior detection model, and taking the daily behavior category corresponding to the highest predicted probability as the daily behavior detection result of the user to be detected.

[0011] Optionally, before performing the step of inputting the fused features into a preset daily behavior detection model, the method further includes: obtaining the preset daily behavior detection model; obtaining the preset daily behavior detection model includes: sampling users' daily behavior data; preprocessing the daily behavior data to obtain preprocessed sample data; obtaining local feature signals of the sample data, and obtaining global feature signals of the sample data based on the local feature signals; extracting a first feature of the sample data based on the local feature signals, and extracting a second feature of the sample data based on the global feature signals; obtaining fused features corresponding to the sample data based on the first feature and the second feature; inputting the first feature, the second feature, and the fused feature into a preset classifier, and outputting corresponding daily behavior recognition results; constructing a multi-level loss function based on the daily behavior recognition results; optimizing and training the daily behavior detection model based on the results of the multi-level loss function to obtain model parameters corresponding to minimizing the loss function, and determining the daily behavior detection model based on the model parameters.

[0012] Optionally, constructing a multi-level loss function based on the daily behavior recognition results includes:

[0013] Construct the loss function corresponding to the following formula, where the formula is:

[0014]

[0015] L represents the loss function, and M represents the total number of samples. T represents the probability that the multilayer perceptron predicts the j-th sample as belonging to the i-th class. i j This indicates that the j-th sample is the true label corresponding to the i-th class. This represents the probability that the convolutional neural network predicts the j-th sample as belonging to the i-th class. This indicates the probability that the j-th sample belongs to the i-th class based on the fusion features after the fusion processing.

[0016] To address the aforementioned technical problems, another technical solution adopted in this application is: providing a daily behavior detection device, comprising: a first data acquisition module for acquiring daily behavior data of a user to be detected; a second data acquisition module for obtaining local feature signals corresponding to the daily behavior data based on the daily behavior data, and obtaining global feature signals corresponding to the daily behavior data based on the local feature signals; a fusion feature acquisition module for obtaining corresponding fusion features based on the local feature signals and the global feature signals; and a daily behavior detection module for inputting the fusion features into a preset daily behavior detection model and outputting the daily behavior of the user to be detected.

[0017] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a daily behavior detection device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0018] To solve the above-mentioned technical problems, another technical solution adopted in the embodiments of this application is: providing a computer storage medium that stores computer-executable instructions, which, when executed by a daily behavior detection device, cause the daily behavior detection device to perform the method described above.

[0019] This application provides a method, apparatus, device, and computer storage medium for detecting daily behavior. It acquires daily behavior data of a user to be detected, obtains local feature signals corresponding to the daily behavior data, obtains global feature signals corresponding to the daily behavior data based on the local feature signals, acquires fused features corresponding to the local and global feature signals, and finally detects the user's daily behavior based on the fused features and a preset daily behavior detection model. The solution provided in this application, by extracting and fusing local and global feature signals, can better integrate the continuity of local feature signals and the globality of global feature signals, thereby more accurately identifying the user's daily behavior. Furthermore, the process of identifying user daily behavior does not require installing specific sensors in specific locations; therefore, the implementation method provided in this application has a wider range of applications and strong applicability. Attached Figure Description

[0020] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0021] Figure 1 This is a flowchart of a daily behavior detection method provided in an embodiment of this application;

[0022] Figure 2 This is a flowchart of a method for obtaining the fused features corresponding to the local feature signals and the global feature signals provided in an embodiment of this application;

[0023] Figure 3 This is a flowchart of a method for obtaining the preset daily behavior detection model provided in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram illustrating the detection of users' daily behaviors according to an embodiment of this application;

[0025] Figure 5 This is a flowchart of a method for preprocessing daily behavioral data provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of a daily behavior detection device provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the hardware structure of a daily behavior detection device provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] It should be noted that, unless otherwise specified, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device schematic diagram or the order in the flowchart.

[0030] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0031] The daily behavior detection method, apparatus, and device provided in this application can be applied to the detection of daily behaviors of the elderly. Currently, population aging is widespread, and how to better care for the elderly is an urgent problem to be solved. By detecting the daily behaviors of the elderly, caregivers can keep abreast of their activities, thus better caring for them and preventing accidents. Daily behaviors include, but are not limited to, running, sitting, standing, walking, going up and down stairs, eating, falling, lying down, sleeping, smoking, watching TV, leaning against a wall, sneezing, etc. This daily behavior detection method, apparatus, and device mainly detect and identify the above-mentioned daily behaviors. It acquires user daily behavior data through sensors, processes the data to obtain local feature signals, converts these signals into global feature signals (specifically image signals) using relevant image processing methods, extracts features from the local feature signals using a multilayer perceptron neural network, extracts features from the global feature signals using a convolutional neural network, and then fuses the extracted features to obtain fused features. These fused features are then input into a preset daily behavior detection model to output the user's daily behavior detection results. In determining the preset daily behavior detection model, a constructed multi-layer regression loss function guides the training of the model. This embodiment effectively combines the global nature of global feature signals with the local nature of local feature signals, thereby further improving the accuracy of user daily behavior detection.

[0032] The described daily behavior detection method and device can be executed in any suitable type of electronic device with computing power. That is, the daily behavior detection device can be a smartwatch, smart bracelet, smartphone, server, desktop computer, tablet computer, or other electronic products. The server can be a single physical server or a logical server virtualized from multiple physical servers. Alternatively, the server can be a server cluster consisting of multiple interconnected servers, with each functional module distributed across different servers within the server cluster.

[0033] The detailed process of detecting users' daily behavior will now be described with reference to the accompanying drawings in the embodiments of this application.

[0034] Please see Figure 1 , Figure 1 This is a flowchart of a daily behavior detection method provided in an embodiment of this application. Figure 1 As shown, the method includes:

[0035] S11. Obtain the daily behavior data of the user to be tested.

[0036] The users to be tested can be the aforementioned elderly individuals, or other groups, such as patients requiring medical care, patients undergoing rehabilitation training, children, or university students conducting experiments. The daily behavior data includes acceleration, which is the rate of change of velocity with respect to time, representing how quickly the velocity changes. The daily behavior data also includes angular velocity, which is the radius of the circle connecting the moving mass and the center of the circle rotated in radians per unit time. The daily behavior data corresponds to the daily behaviors described.

[0037] To acquire daily behavioral data of the user being monitored, miniature sensors such as accelerometers and gyroscopes can be used. These sensors are easy to install, small in size, and can be attached to multiple body parts, making them suitable for long-term activity monitoring.

[0038] In this embodiment, when detecting the daily behavior of the elderly, the daily behavior data can be sampled using two sensors: the accelerometer and the gyroscope of a mobile phone and a watch. The acceleration and angular velocity collected by the mobile phone and the watch, respectively, are recorded. When acquiring daily behavior data, the user's acceleration and angular velocity can be collected within a preset time period, such as one hour, one day, or one week. The daily behavior data within the preset time period can be used as the recognition length of a daily behavior action; for example, 3 seconds of daily behavior data can be used as the recognition length of a daily behavior action.

[0039] In some embodiments, such as Figure 1 As shown, after performing step S11, obtaining the daily behavior data of the user to be detected, the method further includes: S12, preprocessing the daily behavior data.

[0040] The preprocessing of the daily behavior data includes: performing frequency synchronization processing on the daily behavior data so that the data collection frequency of the daily behavior data is within a preset frequency range; and performing synchronization processing on the daily behavior data so that the timestamp difference of the daily behavior data collected by different sensors is within a preset time range.

[0041] Different sensor devices collect data at different frequencies, and the timestamps used for data collection may also vary significantly. Typically, the acquired daily behavior data can be converted into a suitable format to reduce data noise interference, resulting in more accurate detection of subsequent daily behaviors.

[0042] Frequency matching is a technique that uses filters to eliminate noise at specific frequencies. Specifically, frequency matching can be achieved by setting the lowest sampling frequency as a standard frequency (e.g., 50Hz), and then reducing the amount of data at higher frequencies through equidistant sampling, while keeping the time interval constant. This effectively reduces the impact of noise at specific frequencies. Synchronization processing, on the other hand, is a technique that reduces noise by averaging data over multiple sampling periods. For example, synchronization processing can be performed as follows: Figure 5 As shown, the timestamps of daily behavior data collected by the accelerometer and gyroscope can be obtained respectively. The difference between the two timestamps is calculated, and the absolute value is taken. If the absolute value is less than a threshold, the data collected by the accelerometer and gyroscope respectively are merged as synchronous data. If the difference is not less than the threshold, it is determined whether the timestamp of the accelerometer data detection is less than the timestamp of the gyroscope data detection. If the timestamp of the accelerometer data detection is less than the timestamp of the gyroscope data detection, the timestamp corresponding to the next frame of data from the accelerometer is taken, and the difference between the timestamp of the next frame of data and the timestamp of the gyroscope data detection is calculated, and the absolute value is taken. If the absolute value is less than the threshold, it is considered as synchronous data and merged. Otherwise, the process of comparing the timestamps of the accelerometer data detection and the timestamps of the gyroscope data detection is repeated. If the timestamp of the accelerometer's data is not less than the timestamp of the gyroscope's data, then the timestamp corresponding to the next frame of data from the gyroscope is taken, and the difference between this timestamp and the timestamp of the accelerometer's data is calculated, with the absolute value taken. If this absolute value is less than a threshold, it is considered synchronized data and merged; otherwise, the process of comparing the timestamps of the accelerometer and gyroscope's data is repeated. Since noise is random, its impact varies across different sampling periods; therefore, using the timestamps of synchronized sampling data can eliminate the influence of noise.

[0043] After the above-mentioned same-frequency processing and synchronization processing, the data frames of the obtained daily behavior data can be the same or within a preset range, such as all being 150 frames, with a sampling frame rate of 50Hz and a sampling duration of 3 seconds corresponding to a daily behavior action, thereby determining that the data frame corresponding to a daily behavior action is 150 frames.

[0044] S13. Based on the daily behavior data, obtain the local feature signal corresponding to the daily behavior data, and based on the local feature signal, obtain the global feature signal corresponding to the daily behavior data.

[0045] Specifically, obtaining the local feature signal corresponding to the daily behavior data includes: acquiring the angular velocity and acceleration of the user to be detected based on the daily behavior data; acquiring the detection time corresponding to the angular velocity and acceleration, and constructing a time series signal, wherein the time series signal is the local feature signal. It can be understood that in step S13, in order to acquire the user's daily behavior within a preset time period, the preset time period (i.e., the detection time) corresponding to the daily behavior data, along with the angular velocity and acceleration of the user to be detected, are combined to form a local feature signal.

[0046] For example, the acceleration and angular velocity of the mobile phone used by the target elderly person and the acceleration and angular velocity of the watch used by the elderly person are obtained within a preset time period. Then, a corresponding time series signal is generated based on the timestamp, corresponding acceleration and corresponding angular velocity within the time period. The time series signal is used as a local feature signal that can characterize the daily behavior characteristics of the target elderly person within the time period (e.g., running, sitting, standing, walking, going upstairs and downstairs). For example, the local feature signal characterizing the daily behavior of running is different from the local feature signal characterizing the daily behavior of sitting (e.g., the acceleration during running changes with time, while the acceleration during sitting is always close to 0. Since the body will have obvious acceleration and deceleration during running, the change in acceleration can be observed. When sitting, the body remains relatively still and there is no obvious change in acceleration).

[0047] The step of obtaining the global feature signal corresponding to the daily behavior data based on the local feature signal includes: normalizing the data corresponding to the time series signal; converting the normalized time series signal data into polar coordinate data; and converting the polar coordinate data into a two-dimensional image that retains the time features, i.e., the two-dimensional image that retains the time features is the global feature signal.

[0048] In some embodiments, the local feature signal is transformed into the global feature signal using the Gramian Angular Field (GAF) visualization method. The basic probabilities of Gramian Angular Fields and the fundamental principles of Gramian Angular Field visualization are not elaborated here. Specifically, Gramian Angular Fields are used to represent time-series data in polar coordinates, rather than traditional Cartesian coordinates. In the Gram matrix, each element is the cosine of the sum or difference of a pair of time values, which are composed of the timestamps corresponding to the angular velocity and acceleration of the user being detected.

[0049] In some embodiments, time series data X = x1, x2, ..., xn of n actual observations are obtained. nThe X is processed using a normalization method so that all values ​​fall within the interval [-1, 1]. The feature quantization formula used in the normalization process is as follows:

[0050]

[0051] Where min(X) represents the minimum value in the time series data; max(X) represents the maximum value in the time series data. This represents the data in the normalized time series; X represents the time series.

[0052] Furthermore, since the interval [-1, 1] has the same range as the cosine function, the normalized time series can be expressed using the following formula. Treat it as the cosine of the angle, and the time step as the radius:

[0053]

[0054] in, This represents data in a normalized time series; Represents the normalized time series; t i The time step is represented by ; N represents a constant factor used to normalize the span of the polar coordinate system. Further, after converting the normalized time series signal data into polar coordinate form, the temporal correlations within different time intervals can be identified by utilizing angular perspective and considering the trigonometric sum between each point. The Gram angle field is defined as follows:

[0055]

[0056] Where G represents the local feature signal sequence converted into two-dimensional image information.

[0057] S14. Obtain the corresponding fused features based on the local feature signals and the global feature signals.

[0058] like Figure 2 As shown, obtaining the corresponding fused features based on the local feature signals and the global feature signals includes:

[0059] S141. Based on the principle of multilayer perceptron, feature extraction is performed on the local feature signal to obtain the features corresponding to the local feature signal.

[0060] In some embodiments, the multilayer perceptron structure includes an input layer, multiple hidden layers, and an output layer. The input layer has n (e.g., n = 150) neurons, the hidden layers have q neurons, and the output layer has k (e.g., k = 6) neurons. First, if the input layer is represented by vector X, the output of the hidden layer is f(W1X + b1), where W1 is the first weight coefficient (also called the connection coefficient), b1 is the first bias coefficient, and the function f can be a commonly used sigmoid function or tanh function. Then, the process from the hidden layer to the output layer can be viewed as a multi-class logistic regression, i.e., softmax regression. Therefore, the output of the output layer is softmax(W2X1 + b2), where X1 represents the output of the hidden layer f(W1X + b1), W2 is the second weight coefficient, and b2 is the second bias coefficient. Finally, the network structure formula of the multilayer perceptron is expressed as follows:

[0061]

[0062] Where G represents the softmax activation function; h represents the number of hidden layers; W i and b i represents the weights and biases of the i-th hidden layer; x represents the input feature vector; W1 and b1 represent the weights and biases of the input layer; S represents the activation function.

[0063] S142. Based on the principle of convolutional neural networks, feature extraction is performed on the global feature signal to obtain the features corresponding to the global feature signal.

[0064] In some embodiments, the convolutional neural network has a kernel size of 3*3, a stride of 2, and an activation function of ReLU. After multiple convolutional operations, a feature map of size 5*5*256 is obtained. Finally, two fully connected layers are connected. It should be noted that in the embodiments of this application, the network structure of the feature extraction network is not limited and can be replaced according to actual needs. The ultimate goal of the feature extraction network is to obtain the feature map of the input image. Therefore, the feature extraction network used to extract features from the global feature signal consists of convolutional layers, activation function layers, and instance normalization layers, with the corresponding mathematical formulas as follows:

[0065]

[0066] in, This represents the m-th feature map of the l-th layer; Let represent the m-th feature map of layer l+1; W represents the convolution kernel, with a size of 3*3; B represents the bias term; σ represents the ReLU activation function; and IN represents instance normalization. It should be noted that, due to information loss caused by downsampling and upsampling, uniform stride convolution is used instead of traditional convolution, with the number of convolution kernels in each layer set to 16, 32, 64, 128, and 256 respectively.

[0067] S143. The features corresponding to the local feature signals and the features corresponding to the global feature signals are fused together to obtain fused features.

[0068] Specifically, a feature fusion layer is constructed to fuse the features corresponding to the local feature signals and the features corresponding to the global feature signals. First, after step S141, a first feature vector F1 of size 1*1024 corresponding to the local feature signals is obtained. After step S142, a second feature vector F2 of size 1*1024 corresponding to the global feature signals is obtained. Then, the corresponding fused feature F is obtained using the following formula:

[0069] F = α*F1 + (1-α)*F2;

[0070] Where α represents the adaptive weight value; F represents the fusion feature. The fusion feature obtained after fusion processing includes both the features corresponding to the local feature signals and the features corresponding to the global feature signals, thus more comprehensively encompassing the data features of the daily behavior data of the user to be detected, making the final identification of the user's daily behavior based on the fusion feature more accurate.

[0071] S15. Input the fused features into a preset daily behavior detection model and output the daily behavior of the user to be detected.

[0072] Based on the fusion features and the preset daily behavior detection model, the predicted probability of the user to be detected belonging to each daily behavior category is obtained, and the daily behavior category corresponding to the highest predicted probability is taken as the daily behavior detection result of the user to be detected.

[0073] For example, if the predicted probabilities of a target elderly person's daily behavior data belonging to various daily behavior categories are: running 80%, sitting 5%, standing 5%, walking 10%, going upstairs 0%, and going downstairs 0%, then running, which has the highest probability of 80%, is taken as the daily behavior category corresponding to the target elderly person's daily behavior data.

[0074] In some embodiments, the method further includes: obtaining the preset daily behavior detection model.

[0075] like Figure 3As shown, obtaining the preset daily behavior detection model includes:

[0076] S201. Sample users' daily behavior data;

[0077] S202. Preprocess the daily behavior data to obtain preprocessed sample data;

[0078] S203. Obtain the local feature signals of the sample data, and obtain the global feature signals of the sample data based on the local feature signals;

[0079] S204. Extract the first feature of the sample data based on the local feature signal, and extract the second feature of the sample data based on the global feature signal;

[0080] S205. Obtain the fusion feature corresponding to the sample data based on the first feature and the second feature;

[0081] S206. Input the first feature, the second feature, and the fused feature into a preset classifier, and output the corresponding daily behavior recognition result;

[0082] In some embodiments, the classifier is a softmax layer. The softmax layer is capable of multi-class classification, its main principle being to map the input vector onto a probability distribution, thereby classifying the input. Specifically, the first feature and the second feature are sent to their respective softmax layers, and the fused feature is also sent to the corresponding softmax layer. Classification prediction is performed through the softmax layer to obtain the daily behavior recognition result (e.g., running, sitting, standing, walking, going upstairs, going downstairs, etc.) corresponding to the sample data. It should be noted that the softmax layer outputs a probability distribution predicting the sample data as various daily behaviors. During the model training phase of the daily behavior detection model, the model can be trained and optimized based on this probability distribution. During the model usage phase, the daily behavior category with the highest probability value in the probability distribution is used as the daily behavior recognition result.

[0083] S207. Construct a multi-level loss function based on the daily behavior recognition results;

[0084] Specifically, constructing the multi-level loss function includes: first, obtaining the output result P of the multilayer perceptron processing the local feature signal. g (P g0 P g1 P g2 P g3 P g4 Pg5 The true labels of the local feature signals are obtained using one-hot encoding, resulting in T(T0, T1, T2, T3, T4, T5). Similarly, the output P of the convolutional neural network on the global feature signals is obtained. r (P r0 P r1 P r2 P r3 P r4 P r5 Then, the feature fusion layer obtains the fused feature result P of the features corresponding to the local feature signals and the features corresponding to the global feature signals. b (P b0 P b1 P b2 P b3 P b4 P b5 Construct the loss function corresponding to the following formula, which is:

[0085]

[0086] Where L represents the loss function, and M represents the total number of samples. T represents the probability that the multilayer perceptron predicts the j-th sample as belonging to the i-th class. i j This indicates that the j-th sample is the true label corresponding to the i-th class. This represents the probability that the convolutional neural network predicts the j-th sample as belonging to the i-th class. This indicates the probability that the j-th sample belongs to the i-th class based on the fusion features after the fusion processing.

[0087] S208. Optimize and train the daily behavior detection model based on the results of the multi-level loss function to obtain the model parameters that minimize the loss function, and determine the daily behavior detection model based on the model parameters.

[0088] In some embodiments, optimizing the daily behavior detection model based on the results of the multi-level loss function includes multiple optimization training components:

[0089] The first optimization training content is to optimize the parameters of the classifier to improve the accuracy of the classifier in obtaining the daily behavior recognition result based on the first feature, the second feature and the fused feature;

[0090] The second optimization training content is to optimize the parameters of the feature fusion layer to improve the comprehensiveness of the fused features in representing the first feature and the second feature;

[0091] The third optimization training content is to optimize the parameters of the convolutional neural network so that the convolutional neural network can extract features from the global feature signal more comprehensively;

[0092] The fourth optimization training content is to optimize the parameters of the multilayer perceptron so that the multilayer perceptron can extract features from the local feature signals more comprehensively.

[0093] The fifth optimization training content is to optimize the parameters used by the GAF image processing method in processing the local feature signal sequence, so as to improve the accuracy of the global feature signal in representing the daily behavior data;

[0094] The sixth optimization training content is to optimize other parameters, such as optimizing the frequency of collecting the daily behavior data and optimizing the number of data frames used when processing the daily behavior data at the same frequency.

[0095] The following example illustrates the process of detecting the above-mentioned daily behaviors. For example, ... Figure 4 As shown, daily behavior data (mainly acceleration and angular velocity) of the target elderly person is acquired through sensing devices. Then, the daily behavior data is preprocessed to obtain corresponding local feature signals. Next, image data processing methods are used to process the local feature signals to obtain corresponding global feature signals. The first feature of the local feature signal is extracted by a multilayer perceptron, and the second feature of the global feature signal is extracted by a convolutional neural network. Then, the first feature and the second feature are respectively input into the corresponding fully connected layers and then feature fusion is performed to obtain the fused feature corresponding to the first feature and the second feature. Finally, the first feature, the second feature and the fused feature are input into a Softmax layer for classification prediction to obtain the daily behavior recognition result corresponding to the daily behavior data of the target elderly person.

[0096] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a daily behavior detection device provided in an embodiment of this application. The daily behavior detection device 30 includes:

[0097] The first data acquisition module 301 is used to acquire the daily behavior data of the user to be detected;

[0098] The second data acquisition module 302 is used to obtain a local feature signal corresponding to the daily behavior data based on the daily behavior data, and to obtain a global feature signal corresponding to the daily behavior data based on the local feature signal.

[0099] The fusion feature acquisition module 303 is used to obtain the corresponding fusion features based on the local feature signals and the global feature signals;

[0100] The daily behavior detection module 304 is used to input the fused features into a preset daily behavior detection model and output the daily behavior of the user to be detected.

[0101] In some embodiments, the daily behavior detection device 30 further includes a preprocessing module, which is used to preprocess the daily behavior data. Specifically, the preprocessing module is used to perform synchronous processing on the daily behavior data so that the data acquisition frequency of the daily behavior data is within a preset frequency range; and to perform synchronous processing on the daily behavior data so that the timestamp difference of the daily behavior data collected by different sensors is within a preset time range.

[0102] In some embodiments, the daily behavior detection device 30 further includes a daily behavior detection model determination module, which is specifically used for: sampling users' daily behavior data; preprocessing the daily behavior data to obtain preprocessed sample data; acquiring local feature signals of the sample data, and acquiring global feature signals of the sample data based on the local feature signals; extracting a first feature of the sample data based on the local feature signals, and extracting a second feature of the sample data based on the global feature signals; obtaining fusion features corresponding to the sample data based on the first feature and the second feature; inputting the first feature, the second feature, and the fusion feature to a preset classifier, and outputting corresponding daily behavior recognition results; constructing a multi-level loss function based on the daily behavior recognition results; optimizing and training the daily behavior detection model based on the results of the multi-level loss function to obtain model parameters that minimize the loss function, and determining the daily behavior detection model based on the model parameters.

[0103] It should be noted that the above-mentioned daily behavior detection device can execute the daily behavior detection method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the daily behavior detection device can be found in the daily behavior detection method provided in the embodiments of this application.

[0104] Please see Figure 7 , Figure 7 This is a schematic diagram of the hardware structure of a daily behavior detection device provided in an embodiment of this application. This daily behavior detection device can be used to execute the daily behavior detection method described above. Figure 7 As shown, the daily behavior detection device 40 includes:

[0105] One or more processors 401 and memory 402, Figure 7 Take a processor 401 as an example.

[0106] The processor 401 and the memory 402 can be connected via a bus or other means. Figure 7Taking the example of a connection between China and Israel via a bus.

[0107] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the daily behavior detection method in the embodiments of this application (e.g., attached...). Figure 6 (The various modules shown). The processor 401 executes various functional applications and data processing of the daily behavior detection device by running non-volatile software programs, instructions, and modules stored in the memory 402, thereby implementing the daily behavior detection method of the above method embodiment.

[0108] Memory 402 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the daily behavior detection device. Furthermore, memory 402 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and this remote memory may be connected to the daily behavior detection device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0109] The one or more modules are stored in the memory 402. When executed by the one or more processors 401, they perform the daily behavior detection method in any of the above method embodiments, for example, the method described above. Figure 1 , Figure 2 and Figure 3 The method steps in the text are to achieve the following: Figure 6 The functionality of the modules within.

[0110] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0111] This application provides a computer storage medium storing computer-executable instructions. These computer-executable instructions are executed by a daily behavior detection device using the daily behavior detection method described above, for example, executing the method described above. Figure 1 , Figure 2 and Figure 3 The method steps in the text are to achieve the following: Figure 6 The functionality of the modules within. Specifically, the computer storage medium may be a non-volatile computer-readable storage medium.

[0112] This application provides a computer program product, including a computing program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the daily behavior detection method in any of the above method embodiments, for example, to perform the above-described... Figure 1 , Figure 2 and Figure 3 The method steps in the text are to achieve the following: Figure 6 The functionality of the modules within.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or it can be implemented using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting daily behavior, characterized in that, include: Obtain daily behavioral data of the user to be tested; Based on the daily behavior data, a local feature signal corresponding to the daily behavior data is obtained, and a global feature signal corresponding to the daily behavior data is obtained based on the local feature signal. Based on the local feature signals and the global feature signals, the corresponding fused features are obtained; The fused features are input into a preset daily behavior detection model, which outputs the daily behavior of the user to be detected. The step of obtaining the local feature signal corresponding to the daily behavior data based on the daily behavior data includes: The angular velocity and acceleration of the user to be detected are obtained based on the daily behavior data. The detection times corresponding to the angular velocity and the acceleration are obtained and a time series signal is formed, wherein the time series signal is the local feature signal; The step of obtaining the global feature signal corresponding to the daily behavior data based on the local feature signal includes: The data corresponding to the time series signal is normalized. The data of the normalized time series signal is converted into polar coordinate form; The polar coordinate data is converted into a two-dimensional image that retains the time features.

2. The method according to claim 1, characterized in that, After performing the step of acquiring the daily behavior data of the user to be detected, the method further includes: preprocessing the daily behavior data; The preprocessing of the daily behavior data includes: The daily behavior data is processed in the same frequency so that the data collection frequency of the daily behavior data is within a preset frequency range; The daily behavior data is synchronized so that the timestamp difference between the daily behavior data collected by different sensors is within a preset time range.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining the corresponding fused features based on the local feature signals and the global feature signals includes: Based on the principle of multilayer perceptron, feature extraction is performed on the local feature signal to obtain the features corresponding to the local feature signal; Based on the principle of convolutional neural networks, feature extraction is performed on the global feature signal to obtain the features corresponding to the global feature signal; The features corresponding to the local feature signals and the features corresponding to the global feature signals are fused together to obtain fused features.

4. The method according to claim 3, characterized in that, The step of inputting the fused features into a preset daily behavior detection model and outputting the daily behavior of the user to be detected includes: Based on the fusion features and the preset daily behavior detection model, the predicted probability of the user to be detected belonging to each daily behavior category is obtained, and the daily behavior category corresponding to the highest predicted probability is taken as the daily behavior detection result of the user to be detected.

5. The method according to claim 4, characterized in that, Before performing the step of inputting the fused features into a preset daily behavior detection model, the method further includes: obtaining the preset daily behavior detection model; The process of obtaining the preset daily behavior detection model includes: Sample users' daily behavior data; The daily behavior data is preprocessed to obtain preprocessed sample data; Obtain the local feature signals of the sample data, and obtain the global feature signals of the sample data based on the local feature signals; The first feature of the sample data is extracted based on the local feature signal, and the second feature of the sample data is extracted based on the global feature signal; Based on the first feature and the second feature, the fusion feature corresponding to the sample data is obtained; Input the first feature, the second feature, and the fused feature into a preset classifier, and output the corresponding daily behavior recognition result; A multi-level loss function is constructed based on the daily behavior recognition results; The daily behavior detection model is optimized and trained based on the results of the multi-level loss function to obtain the model parameters that minimize the loss function, and the daily behavior detection model is determined based on the model parameters.

6. The method according to claim 5, characterized in that, The construction of the multi-level loss function based on the daily behavior recognition results includes: Construct the loss function corresponding to the following formula, where the formula is: ； L represents the loss function, and M represents the total number of samples. This represents the probability that the multilayer perceptron predicts the j-th sample as belonging to the i-th class. This indicates that the j-th sample is the true label corresponding to the i-th class. This represents the probability that the convolutional neural network predicts the j-th sample as belonging to the i-th class. This indicates the probability that the j-th sample belongs to the i-th class based on the fusion features after the fusion processing.

7. A daily behavior detection device, characterized in that, include: The first data acquisition module is used to acquire the daily behavior data of the user to be tested. The second data acquisition module is used to obtain a local feature signal corresponding to the daily behavior data based on the daily behavior data, and to obtain a global feature signal corresponding to the daily behavior data based on the local feature signal. The fusion feature acquisition module is used to obtain the corresponding fusion features based on the local feature signals and the global feature signals; The daily behavior detection module is used to input the fused features into a preset daily behavior detection model and output the daily behavior of the user to be detected. The daily behavior detection device is used to perform the daily behavior detection method according to any one of claims 1 to 6.

8. A daily behavior detection device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which, when executed by the daily behavior detection device, cause the daily behavior detection device to perform the method described in any one of claims 1 to 6.

Images