Nursing behavior recognition method and device, nursing behavior training method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a nursing behavior recognition method and device, a training method and device, electronic equipment and a medium, and relates to the technical field of action recognition. Determining screened behavior characteristics according to each piece of behavior data and the corresponding importance degree; fusing the screened behavior characteristics based on a motion dependency relationship among actions reflected by the screened behavior characteristics to obtain related behavior characteristics; for each time step, determining a time sequence change characteristic according to a time sequence change rule between the time step and the correlation behavior characteristic corresponding to the correlation time step; according to each time sequence change feature and the corresponding contribution degree thereof, determining an aggregated behavior feature; and splicing the related behavior characteristics, the aggregated behavior characteristics and the time sequence change characteristics, and determining a nursing behavior recognition result. In this way, the nursing behavior executed by the nurse can be reflected more accurately, and the accuracy of nursing behavior recognition can be improved.

Description

Technical Field

[0001] This application relates to the field of motion recognition technology, and in particular to a nursing behavior recognition method, training method, device, electronic device and medium. Background Technology

[0002] With the increasing aging population and rising prevalence of chronic diseases, the patient load requiring care in medical institutions is constantly increasing. Nursing work involves a series of complex and frequent patient care tasks, such as medication administration, vital sign monitoring, and assisted mobility. These tasks require continuous and meticulous record-keeping, which is of irreplaceable importance for assessing the quality of care, optimizing human resource allocation, reducing medical errors, and achieving refined hospital management.

[0003] For a long time, the identification and documentation of nursing behaviors have relied primarily on manual observation and recording. Manual recording is highly subjective; different nurses may describe the same nursing behavior differently, leading to poor data consistency. In high-intensity clinical work environments, nurses may resort to retrospective recording, which easily results in omissions and human errors, leading to poor accuracy in identifying nursing behaviors. Summary of the Invention

[0004] The purpose of this application is to provide a nursing behavior recognition method, training method, device, electronic device, and medium to improve the accuracy of nursing behavior recognition. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a nursing behavior recognition method, the method comprising:

[0006] Acquire behavioral data to be identified, wherein the behavioral data to be identified includes behavioral data collected by multiple motion sensors, and the multiple motion sensors are worn on different parts of the caregiver's body;

[0007] For each behavioral data collected by a motion sensor, the importance of the behavioral data is determined based on the noise contained in the data, and the filtered behavioral features are determined based on each behavioral data and its importance.

[0008] Based on the motion dependency between actions reflected by the filtered behavioral features corresponding to each motion sensor, the filtered behavioral features are fused to obtain relevant behavioral features.

[0009] For each time step, the temporal change characteristics are determined based on the temporal change pattern between the relevant behavioral characteristics of the time step and the associated time step, wherein the associated time step and the current time step satisfy a temporal correlation relationship in terms of time.

[0010] Based on each temporal change feature and its corresponding contribution level, the aggregated behavioral features are determined, wherein the contribution level characterizes the degree of influence of the corresponding temporal change feature on nursing behavior identification;

[0011] The relevant behavioral features, the aggregated behavioral features, and the temporal change features are concatenated, and the nursing behavior identification result is determined based on the concatenated behavioral features.

[0012] Secondly, embodiments of this application provide a method for training a behavior recognition model, the method comprising:

[0013] Acquire sample behavior data and corresponding identification result labels, wherein the sample behavior data includes behavior data collected by multiple motion sensors, and the multiple motion sensors are worn on different parts of the caregiver's body;

[0014] The sample behavior data is input into the initial behavior recognition model, so that the feature selection module included in the initial behavior recognition model determines the importance of each sample behavior data collected by the motion sensor based on the noise contained in the sample behavior data, and determines the filtered sample behavior features based on each sample behavior data and its corresponding importance, and inputs the filtered sample behavior features into the correlation perception module.

[0015] The correlation perception module uses the current model parameters to fuse the filtered sample behavior features based on the motion dependency between actions reflected by the filtered sample behavior features corresponding to each motion sensor, to obtain sample correlation behavior features, and then inputs the sample correlation behavior features into the time series modeling module.

[0016] The temporal modeling module, using the current model parameters, determines the temporal change features of samples for each time step based on the temporal change pattern between the sample correlation behavior features corresponding to the time step and the associated time step. Based on each sample temporal change feature and its corresponding contribution level, the module determines the aggregated sample behavior features and inputs the aggregated sample behavior features into the feature fusion and classification module. The associated time step and the current time step satisfy a temporal correlation relationship, and the contribution level characterizes the degree of influence of the corresponding sample temporal change features on nursing behavior recognition.

[0017] The feature fusion and classification module uses the current model parameters to concatenate the sample correlation behavior features, the aggregated sample behavior features, and the sample temporal change features, and determines the prediction and recognition result based on the concatenated sample behavior features.

[0018] Based on the difference between the predicted recognition result and the recognition result label, the model parameters of the initial behavior recognition model are adjusted until the initial behavior recognition model meets the convergence condition, thus obtaining the trained behavior recognition model.

[0019] Thirdly, embodiments of this application provide a nursing behavior recognition device, the device comprising:

[0020] The first data acquisition module is used to acquire behavior data to be identified, wherein the behavior data to be identified includes behavior data collected by multiple motion sensors, and the multiple motion sensors are worn on different parts of the caregiver's body;

[0021] The first feature filtering module is used to determine the importance of each behavioral data collected by each motion sensor based on the noise contained in the behavioral data, and to determine the filtered behavioral features based on each behavioral data and its corresponding importance.

[0022] The first feature fusion module is used to fuse the filtered behavioral features based on the motion dependency relationship between actions reflected by the filtered behavioral features corresponding to each motion sensor to obtain relevant behavioral features.

[0023] The first temporal feature extraction module is used to determine the temporal change features for each time step based on the temporal change pattern between the relevant behavioral features of the time step and the associated time step, wherein the associated time step and the current time step satisfy a temporal correlation relationship in time.

[0024] The first feature aggregation module is used to determine the aggregated behavioral features based on each temporal change feature and its corresponding contribution level, wherein the contribution level characterizes the degree of influence of the corresponding temporal change feature on nursing behavior recognition.

[0025] The first result output module is used to concatenate the relevant behavioral features, the aggregated behavioral features, and the temporal change features, and determine the nursing behavior recognition result based on the concatenated behavioral features.

[0026] Fourthly, embodiments of this application provide a training apparatus for a behavior recognition model, the apparatus comprising:

[0027] The second data acquisition module is used to acquire sample behavior data and corresponding recognition result labels. The sample behavior data includes behavior data collected by multiple motion sensors, which are worn on different parts of the caregiver's body.

[0028] The second feature filtering module is used to input the sample behavior data into the initial behavior recognition model, so that the feature selection module included in the initial behavior recognition model can determine the importance of each sample behavior data collected by the motion sensor based on the noise contained in the sample behavior data through the current model parameters, and determine the filtered sample behavior features according to each sample behavior data and its corresponding importance, and input the filtered sample behavior features into the correlation perception module.

[0029] The second feature fusion module is used by the correlation perception module to fuse the filtered sample behavior features based on the motion dependency relationship between actions reflected by the filtered sample behavior features corresponding to each motion sensor, using the current model parameters, to obtain sample correlation behavior features, and input the sample correlation behavior features into the time series modeling module.

[0030] The second feature aggregation module is used by the temporal modeling module to determine the sample temporal change features for each time step based on the temporal change pattern between the sample correlation behavior features corresponding to the time step and the associated time step, using the current model parameters. Based on the temporal change features of each sample and its corresponding contribution, the aggregated sample behavior features are determined, and the aggregated sample behavior features are input into the feature fusion and classification module. The associated time step and the current time step satisfy a temporal correlation relationship, and the contribution level characterizes the degree of influence of the corresponding sample temporal change features on nursing behavior recognition.

[0031] The second result output module is used by the feature fusion and classification module to concatenate the sample correlation behavior features, the aggregated sample behavior features, and the sample temporal change features using the current model parameters, and to determine the prediction and recognition result based on the concatenated sample behavior features.

[0032] The parameter adjustment module is used to adjust the model parameters of the initial behavior recognition model according to the difference between the predicted recognition result and the recognition result label, until the initial behavior recognition model meets the convergence condition and the trained behavior recognition model is obtained.

[0033] Fifthly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0034] Memory, used to store computer programs;

[0035] When a processor executes a program stored in memory, it implements the steps of the method described in either the first or second aspect above.

[0036] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in either the first or second aspect above.

[0037] Beneficial effects of the embodiments in this application:

[0038] In the solution provided in this application embodiment, since noise can interfere with the accuracy of nursing behavior recognition, in order to filter out behavioral data containing significant noise from the behavioral data collected by each motion sensor, it can be determined that the importance of behavioral data containing significant noise is low. Furthermore, based on each behavioral data point and its corresponding importance, the filtered behavioral features are determined, which can reduce the interference of behavioral data containing significant noise on nursing behavior recognition. When nursing staff perform nursing behaviors, multiple body parts may cooperate with each other, such as the coordination of hands, ankles, and thighs. At this time, there is a certain correlation between the behavioral features corresponding to the above body parts. Based on the motion dependency relationship between the actions reflected by the filtered behavioral features corresponding to each motion sensor, the filtered behavioral features can be fused to obtain correlated behavioral features, thereby reflecting the correlation of motion features when different body parts cooperate. The actions of nursing staff performing nursing behaviors are usually continuously changing. Therefore, determining the temporal change features based on the temporal change pattern between the correlated behavioral features corresponding to multiple time steps that satisfy temporal correlation can more accurately reflect the nursing behaviors performed by nursing staff. The contribution of each temporal variation feature is related to its influence on nursing behavior recognition; that is, the greater the influence, the higher the contribution of the temporal variation feature. Based on each temporal variation feature and its corresponding contribution, aggregated behavioral features are determined. Therefore, in the aggregated behavioral features, temporal variation features with a greater influence on nursing behavior recognition occupy a higher proportion, allowing the aggregated behavioral features to more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. The relevant behavioral features, aggregated behavioral features, and temporal variation features are concatenated to obtain the concatenated behavioral features. Since relevant behavioral features can reflect the correlation of movement features when different body parts coordinate, the aggregated behavioral features can more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. Temporal variation features can more accurately reflect the nursing behaviors performed by nurses and can reduce the interference of noisy behavioral data on nursing behavior recognition. Therefore, determining the nursing behavior recognition result based on the concatenated behavioral features can improve the accuracy of nursing behavior recognition. Of course, implementing any product or method of this application does not necessarily require achieving all of the above advantages simultaneously. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0040] Figure 1 A flowchart illustrating a nursing behavior recognition method provided in this application embodiment;

[0041] Figure 2(a) shows the results based on Figure 1 A schematic diagram of the behavior recognition model in the embodiment shown;

[0042] Figure 2(b) is based on Figure 1 Another structural diagram of the behavior recognition model in the embodiment shown;

[0043] Figure 3 A flowchart illustrating a training method for a behavior recognition model provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the structure of a nursing behavior recognition device provided in an embodiment of this application;

[0045] Figure 5 A schematic diagram of the structure of a training device for a behavior recognition model provided in an embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0048] To improve the accuracy of nursing behavior recognition, embodiments of this application provide a nursing behavior recognition method, a training method for a behavior recognition model, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product. The nursing behavior recognition method provided in this application embodiment is described below.

[0049] The nursing behavior recognition method provided in this application can be applied to any electronic device that needs to recognize nursing behaviors, such as a desktop computer, laptop computer, tablet computer, etc., without specific limitation. For clarity, it will be referred to as an electronic device herein.

[0050] like Figure 1 As shown, a nursing behavior recognition method includes:

[0051] S101, Obtain the behavior data to be identified.

[0052] The behavioral data to be identified includes behavioral data collected by multiple motion sensors, which are worn on different parts of the caregiver's body.

[0053] S102, for each behavioral data collected by the motion sensor, based on the noise contained in the behavioral data, determine the importance of the behavioral data, and determine the filtered behavioral features according to each behavioral data and its corresponding importance.

[0054] S103, based on the motion dependency relationship between actions reflected by the filtered behavioral features corresponding to each motion sensor, the filtered behavioral features are fused to obtain relevant behavioral features.

[0055] S104. For each time step, determine the time-series change characteristics based on the temporal change pattern between the relevant behavioral characteristics of that time step and the associated time steps.

[0056] The associated time step and the current time step satisfy a temporal association relationship in terms of timing.

[0057] S105, determine the aggregated behavioral characteristics based on each temporal change feature and its corresponding contribution level.

[0058] The degree of contribution represents the degree of influence of the corresponding temporal change characteristics on the identification of nursing behavior.

[0059] S106, the relevant behavioral features, the aggregated behavioral features, and the temporal change features are spliced ​​together, and the nursing behavior identification result is determined based on the spliced ​​behavioral features.

[0060] It is evident that noise interferes with the accuracy of nursing behavior recognition. Therefore, to filter out behavioral data containing significant noise from the behavioral data collected by each motion sensor, it can be determined that behavioral data with significant noise corresponds to lower importance. Furthermore, by determining the filtered behavioral features based on each behavioral data point and its corresponding importance, the interference of noisy behavioral data on nursing behavior recognition can be reduced. When nursing staff perform nursing actions, multiple body parts may coordinate with each other, such as the coordination of hands, ankles, and thighs. In this case, there is a certain correlation between the behavioral features corresponding to these body parts. Based on the motion dependencies between the actions reflected by the filtered behavioral features corresponding to each motion sensor, the filtered behavioral features can be fused to obtain correlated behavioral features, thus reflecting the correlation of motion features when different body parts coordinate. The actions of nursing staff performing nursing actions are usually continuously changing. Therefore, determining the temporal change features based on the temporal change patterns between correlated behavioral features corresponding to multiple time steps that satisfy temporal correlation can more accurately reflect the nursing behaviors performed by nursing staff. The contribution of each temporal variation feature is related to its influence on nursing behavior recognition; that is, the greater the influence, the higher the contribution of the temporal variation feature. Based on each temporal variation feature and its corresponding contribution, aggregated behavioral features are determined. In the aggregated behavioral features, temporal variation features with a greater influence on nursing behavior recognition occupy a higher proportion, allowing the aggregated behavioral features to more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. The relevant behavioral features, aggregated behavioral features, and temporal variation features are concatenated to obtain the concatenated behavioral features. Since relevant behavioral features can reflect the correlation of movement features when different body parts coordinate, the aggregated behavioral features can more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. Temporal variation features can more accurately reflect the nursing behaviors performed by nurses and reduce the interference of noisy behavioral data on nursing behavior recognition. Therefore, determining the nursing behavior recognition results based on the concatenated behavioral features can improve the accuracy of nursing behavior recognition.

[0061] In step S101, the electronic device can acquire the behavioral data to be identified. This behavioral data may include data collected by multiple motion sensors, which can be worn on different parts of the caregiver's body. The number of motion sensors worn by the same caregiver can be 2, 3, 4, 5, etc., and the body parts on which the motion sensors are worn can be the waist, thigh, ankle, wrist, etc., without specific limitations. For example, if a caregiver wears 5 motion sensors, then the motion sensors can be worn on the waist, left thigh, right thigh, left ankle, and right ankle.

[0062] In one implementation, the motion sensor can be an IMU (Inertial Measurement Unit). In this case, the behavioral data can include the following four physical meaning groups: quaternion orientation data, quaternion derivatives, velocity and acceleration, and magnetic field strength data.

[0063] Among these, quaternion orientation data represents the IMU's attitude in three-dimensional space, typically with 12 dimensions; quaternion derivatives can be used to capture the rate of orientation change, which is related to angular velocity, and also typically has 12 dimensions; velocity and acceleration include linear velocity and angular velocity components, i.e., linear velocity, linear acceleration, angular velocity, and angular acceleration, typically with 24 dimensions. Magnetic field strength data consists of the IMU's magnetometer readings, typically with 22 dimensions. Thus, the behavioral data collected by multiple IMUs at each time step can form a 70-dimensional feature vector, used for the identification of nursing behaviors.

[0064] For example, the aforementioned behavior data to be identified can be obtained by standardizing the raw data collected by multiple motion sensors. Specifically, the raw data collected by multiple motion sensors can be normalized to eliminate numerical differences between different physical dimensions (such as angle, velocity, and magnetic field strength). Next, a non-overlapping sliding window strategy can be used to divide the time steps according to a preset time window length, thereby segmenting the continuous time series and obtaining the behavior data to be identified that is within the preset time window length after standardization.

[0065] The preset time window length can be set according to actual needs, such as 20, 25, 30, etc., without specific limitations here. This segmentation method can ensure that each window contains sufficient temporal context for nursing behavior identification, prevent data leakage, and ensure the consistency of nursing behavior within the window.

[0066] The behavioral data collected by motion sensors may contain noise, which can interfere with the recognition of nursing behaviors. To reduce the interference of noise, for each behavioral data collected by motion sensors, the importance of the behavioral data can be determined based on the noise contained in the behavioral data, and the filtered behavioral features can be determined according to each behavioral data and its corresponding importance, i.e., step S102 is executed.

[0067] Specifically, the importance of behavioral data can be negatively correlated with the amount of noise it contains; that is, the more noise a behavioral data point contains, the lower its importance. Consequently, this behavioral data will account for a smaller proportion of the filtered behavioral features. This reduces the interference of the behavioral data in the identification of nursing behaviors.

[0068] In one implementation, for each behavioral data point collected by a motion sensor, an importance weight can be determined based on the noise contained in the behavioral data. A weighted sum is then performed on each behavioral data point and its corresponding importance weight to obtain the filtered behavioral features. The importance weight of each behavioral data point can be negatively correlated with the amount of noise contained in the behavioral data.

[0069] When performing nursing care, multiple body parts may move in coordination. For example, when assisting a patient to change clothes, the left and right hands move together; when making a bed, the hands and waist move together; when pushing a wheelchair, the thighs and ankles move together, and so on.

[0070] In this case, the selected behavioral features corresponding to motion sensors worn on multiple body parts that coordinate movements can reflect the motion dependencies between actions, and can more accurately reflect the characteristics of nursing behaviors. Therefore, based on the motion dependencies between actions reflected by the selected behavioral features corresponding to each motion sensor, the selected behavioral features can be fused to obtain relevant behavioral features, i.e., step S103 is executed.

[0071] Nursing behaviors typically consist of a series of continuous actions. For example, when making a bed, a nurse usually lays the sheets first and then arranges the pillows; when assisting a patient to change clothes, a nurse usually helps the patient take off their clothes first and then put them on. Therefore, there is usually a certain temporal variation pattern among the relevant behavioral characteristics corresponding to multiple time steps. For each time step, the temporal variation characteristics can be determined based on the temporal variation pattern between the relevant behavioral characteristics corresponding to that time step and the associated time steps, i.e., step S104 can be executed.

[0072] Specifically, the associated time steps and the current time step satisfy a temporal correlation relationship. Specifically, the associated time steps can be multiple consecutive or discontinuous time steps preceding the current time step, or multiple consecutive or discontinuous time steps following the current time step. For example, assuming the current time step is time step 5, then the associated time steps could be time step 1-time step 4, time step 6-time step 10, or time step 2, time step 3, time step 6, and time step 8, which is also reasonable.

[0073] To identify the temporal variation features that have a significant impact on the recognition of nursing behaviors, such as the temporal variation features corresponding to the moment when a nurse exerts force to assist a patient in getting up, which are relatively discriminative for the recognition of this nursing behavior, we can determine the contribution of each temporal variation feature based on its degree of influence on the recognition of nursing behaviors.

[0074] In one implementation, when determining the contribution level corresponding to temporal variation features, an attention mechanism can be introduced, using the following formula, through a fully connected layer and an activation function. (Hyperbolic tangent function), calculate time step Weight scores of corresponding time-series variation features : .

[0075] in, For the transpose of a pre-determined attention vector, The weight matrix of the attention layer is predetermined. For time steps The corresponding time-series variation characteristics, The bias term for the pre-determined attention layer.

[0076] Next, the weight scores can be adjusted using the Softmax (normalization exponent) function as follows. Normalization is performed to obtain the time step. Corresponding attention weights : .

[0077] in, It is a natural exponential function. The total number of time steps included in the behavioral data to be identified. This is a time step identifier, specifically 1 to... , For time steps The corresponding weight score. That is, for each time step, the ratio between the weight score and the total weight score is calculated to obtain the attention weight for that time step. In this way, the sum of the attention weights for each time step is 1, and the attention weights can reflect the degree of contribution mentioned above.

[0078] After determining the contribution level of each time-series variation feature, the aggregated behavioral features can be determined based on each time-series variation feature and its corresponding contribution level, i.e., step S105 is executed. Specifically, the following formula can be used for each time-series variation feature. and their corresponding attention weights We perform a weighted summation to obtain the aggregated behavioral characteristics. : .

[0079] Among them, the aggregated behavioral features Also known as context vectors, they aggregate the time-series information most relevant to nursing behavior recognition throughout the entire time series, effectively addressing the problem of long-series information being easily forgotten.

[0080] Next, the relevant behavioral features, aggregated behavioral features, and temporal change features determined in the above steps can be spliced ​​together, and the nursing behavior identification result can be determined based on the spliced ​​behavioral features, i.e., step S106 is executed.

[0081] The nursing behavior recognition results can include the probability distribution corresponding to each preset nursing behavior category, that is, the probability that the nursing behavior performed by the caregiver belongs to each preset nursing behavior category. For example, preset nursing behaviors may include changing clothes, making the bed, assisting with toileting, etc., and can be set according to actual needs, without specific limitations here.

[0082] As one implementation of this application, the step of determining the importance of the behavioral data based on the noise contained in the behavioral data may include:

[0083] The behavioral data is input into a pre-trained behavior recognition model, so that the feature selection module included in the behavior recognition model extracts the nonlinear features of the behavioral data; and the importance of the behavioral data is determined based on the nonlinear features.

[0084] To identify nursing behaviors, a behavior recognition model can be pre-trained. After acquiring the behavior data to be identified, the electronic device can input this data into the behavior recognition model. The training method for the behavior recognition model will be described in detail in subsequent embodiments and will not be elaborated here.

[0085] Since the nonlinear characteristics of behavioral data reflect the amount of noise contained in the data, the feature selection module included in the behavior recognition model can extract the nonlinear characteristics of each behavioral data point collected by a motion sensor and determine the importance of that behavioral data based on these characteristics. The importance is negatively correlated with the noise level.

[0086] As can be seen, in this embodiment, the feature selection module can extract nonlinear features from the behavioral data, determine the noise contained in the behavioral data through these nonlinear features, and thus determine the importance of the behavioral data. This allows for quick and accurate determination of the importance of the behavioral data, thereby suppressing the interference of noisy behavioral data on nursing behavior recognition.

[0087] As one embodiment of this application, the feature selection module may include a first fully connected layer and a second fully connected layer. In this case, the step of the feature selection module extracting the nonlinear features of the behavioral data and determining the importance of the behavioral data based on the nonlinear features may include:

[0088] The first fully connected layer extracts the nonlinear features of the behavioral data and inputs the nonlinear features into the second fully connected layer; the second fully connected layer generates a feature importance mask based on the nonlinear features.

[0089] To address the issue of high-dimensional and noisy behavioral data acquired by motion sensors, this application proposes an adaptive gating network to dynamically filter key feature channels, which is the feature selection module in the behavior recognition model. The feature selection module learns the importance weights of behavioral data through two neural network layers: a first fully connected layer and a second fully connected layer. The structure and function of the first and second fully connected layers are described below:

[0090] After acquiring the behavioral data, the first fully connected layer can map the behavioral data to the hidden space and use an activation function. Nonlinearity is introduced to obtain nonlinear features corresponding to the behavioral data, and these features are then input into the second fully connected layer. The second fully connected layer maps the nonlinear features back to the original feature dimensions of the behavioral data and activates them through an activation function. (S-shaped growth curve) generates a feature importance mask with values ​​in the range [0,1]. Each element in the feature importance mask can be used to reflect the importance of the behavioral data.

[0091] The above process can be expressed by the following formula: .

[0092] in, It is a feature importance mask, also known as a feature selection gating vector, used to measure the importance of each feature channel, that is, the importance of the behavioral data collected by each motion sensor. , which is the behavioral data to be identified, with dimensions of , Indicates the total number of time steps. This represents the feature dimension of each behavioral data point. This represents the behavioral data at the first time step. This represents the behavioral data at the second time step. This represents the behavioral data at the last time step. and These represent the weight matrices of the second fully connected layer and the second fully connected layer, respectively, used to learn the non-linear relationships between features. and These represent the bias vectors of the second fully connected layer and the second fully connected layer, respectively. , , as well as All of these are determined during model training. Assuming there are C predefined categories of nursing behaviors, training the behavior recognition model is to allow the model to learn... The optimal mapping.

[0093] Next, feature importance masks can be applied. With the behavioral data to be identified Element-by-element multiplication yields the filtered behavioral characteristics, i.e. .in, This is the weighted feature representation after adaptive filtering, which is the filtered behavioral feature. This indicates element-wise multiplication.

[0094] With the development of Internet of Things (IoT) and Micro-Electro-Mechanical Systems (MEMS) technologies, automated activity recognition using wearable sensors (such as IMUs) combined with machine learning algorithms has become a research hotspot. Currently, while general human activity recognition (HAR) technology has achieved some success in recognizing everyday behaviors (such as walking and running), it still falls short when applied to complex clinical nursing scenarios.

[0095] One reason for the current poor accuracy of nursing behavior recognition is the high-dimensional noise and heterogeneity of multimodal data. Specifically, sensor data in nursing scenarios typically have extremely high dimensionality (e.g., 70 dimensions) and significant noise interference, and there are huge individual differences in actions among different nurses (inter-subject variability). Existing methods mostly adopt a globally uniform feature processing strategy and lack an adaptive feature selection mechanism. They cannot dynamically select key sensor channels according to the current activity type, causing the model to be easily overwhelmed by irrelevant sensor noise, affecting the accuracy of nursing behavior recognition.

[0096] In this embodiment, motion sensor channels that contribute little to the recognition of the current behavior activity or contain a lot of noise can be automatically identified and suppressed. For example, when a caregiver performs caregiving actions with their upper limbs, the weight of the behavior data collected by the motion sensor worn on the lower limbs is automatically reduced, thereby suppressing noise interference from irrelevant channels and generating an enhanced feature representation.

[0097] As one embodiment of this application, the behavior recognition model may further include a correlation perception module. In this case, the step of fusing the filtered behavior features based on the motion dependency relationship between actions reflected by the filtered behavior features corresponding to each motion sensor to obtain correlated behavior features may include:

[0098] The correlation perception module uses a linear correlation calculation function to calculate the linear correlation matrix between the filtered behavioral features corresponding to each motion sensor; the filtered behavioral features are multiplied by the linear correlation matrix, and the multiplication result is concatenated with the filtered behavioral features to obtain the concatenated behavioral features; the temporal features of the concatenated behavioral features are extracted to obtain the correlated behavioral features.

[0099] To capture the coordinated motion relationships between motion sensors worn on different parts of the body, this application introduces a correlation perception mechanism, implemented through the correlation perception module in the behavior recognition model, to clarify the dependencies between different motion sensors. Specifically:

[0100] Since linear correlation can reflect the dependency between the selected behavioral features corresponding to each motion sensor, in order to quantify the strength of the linear dependency between the selected behavioral features corresponding to each motion sensor, the correlation perception module can use a linear correlation calculation function to calculate the linear correlation matrix between the selected behavioral features corresponding to each motion sensor within the feature batch according to the following formula: .

[0101] in, Indicates time step Alternatively, the cross-correlation matrix between behavioral features selected within the same batch, i.e., the linear correlation matrix, can be used to characterize motion dependencies and quantify the strength of linear dependencies between feature channels of different motion sensors. This represents the linear correlation calculation function, which can be Pearson (Pearson correlation coefficient function), Spearman (Spearman correlation coefficient function), Kendall (Kendall correlation coefficient function), etc., without being specifically limited here.

[0102] Next, the correlation perception module multiplies the filtered behavioral features with a linear correlation matrix, thus explicitly introducing interaction information between motion sensors. Then, the correlation perception module concatenates the multiplication result with the filtered behavioral features to obtain concatenated behavioral features, forming a combined feature set containing both the original information and inter-channel interaction information. The correlation perception module can then use a one-dimensional convolutional layer to extract local temporal features from the concatenated behavioral features, and fuse and reduce the dimensionality of the concatenated behavioral features to obtain the correlated behavioral features.

[0103] The above process can be expressed by the following formula: .

[0104] in, The relevant behavioral features are the hidden layer features after the relevant perception processing. This represents a one-dimensional convolution operation used to extract local temporal features. This indicates a feature concatenation operation. Indicates to and The behavioral features obtained by performing feature splicing.

[0105] Current technologies for nursing behavior recognition lack consideration for domain-specific physical constraints. Existing deep learning models, such as standard CNNs (Convolutional Neural Networks) or LSTMs (Long Short-Term Memory), typically treat motion sensor data as ordinary numerical matrices, ignoring the physical structure and semantic constraints of human motion data. For example, motion sensor data contains various physical quantities such as quaternion direction, angular velocity, linear velocity, and magnetic field strength, representing different aspects of human movement. Simply performing feature concatenation or general convolution operations fails to effectively distinguish and utilize the inherent relationships between these physical quantities, making it difficult for models to capture subtle kinematic differences when handling similar actions (such as wiping or assisted transfer).

[0106] In this embodiment, a linear correlation matrix can be used to mine the cooperative motion patterns between motion sensors worn on different parts of the body, introducing correlation perception processing. This ensures that the behavior recognition model can not only focus on the readings of individual motion sensors, but also understand the relative motion relationships between multiple motion sensors.

[0107] As one embodiment of this application, the behavior recognition model may further include a temporal modeling module, which may include a bidirectional long short-term memory (BiLSTM) network, which may include a forward layer, a backward layer and an activation layer.

[0108] The forward layer and the backward layer are two independent long short-term memory network layers. The forward layer can process data in the forward order of time steps, from the beginning to the end of the sequence, while the backward layer can process data in the reverse order of time steps, from the end to the beginning of the sequence.

[0109] In this context, the step of determining the temporal change characteristics based on the temporal change pattern between the relevant behavioral characteristics corresponding to the time step and the associated time steps may include:

[0110] The forward layer extracts the temporal variation pattern between the relevant behavioral features corresponding to the current time step and the forward time step, determines the forward temporal variation feature, and inputs the forward temporal variation feature into the activation layer; the backward layer extracts the temporal variation pattern between the relevant behavioral features corresponding to the current time step and the backward time step, determines the backward temporal variation feature, and inputs the backward temporal variation feature into the activation layer; the activation layer fuses the forward temporal variation feature and the backward temporal variation feature to obtain the temporal variation feature.

[0111] To address the varying durations and complex temporal dependencies inherent in nursing behaviors, this application employs a bidirectional long short-term memory network, combined with a temporal attention mechanism, to capture contextual information. Specifically:

[0112] Relevant behavioral features are input into a bidirectional long short-term memory (LSSM) network. The forward layer of the LSM network can extract the temporal variation pattern between the relevant behavioral features at the current time step and those at the previous time step, thus determining the forward temporal variation features. Similarly, the backward layer of the LSM network can extract the temporal variation pattern between the relevant behavioral features at the current time step and those at the next time step, thus determining the backward temporal variation features.

[0113] In this context, the forward time step can be a related time step earlier than the current time step, and the backward time step can be a related time step later than the current time step. In this way, the temporal evolution pattern of related behavioral features can be captured from both the forward and backward directions through the forward and backward layers, respectively.

[0114] The activation layer can fuse forward and backward temporal variation features to obtain temporal variation features. The formula for this process is as follows: .in, This represents the hidden state sequence output by a bidirectional long short-term memory network, i.e., the time sequence change characteristics, which includes contextual information of the input sequence at each time step.

[0115] Nursing behaviors exhibit complex temporal dependencies, intricate temporal structures, activity patterns of varying lengths, and extremely wide durations. Real-world data shows a significant difference in duration between different nursing behaviors. For example, oral care may only take a few minutes, while prolonged physical therapy or companionship can last for hours, a difference of tens of thousands of seconds. Current technologies, particularly traditional fixed-window recognition methods, face a dilemma when dealing with such variable-length sequences: excessively long windows lead to the "diluting" or temporal dilution of short activities, while excessively short windows fragment long activities, resulting in the loss of the complete semantics and boundary dynamics of the nursing behavior. Furthermore, existing single-task recognition models often neglect the sequential nature of nursing tasks, making it difficult to capture the logical connections between preceding and subsequent nursing behaviors.

[0116] In this embodiment of the application, an advanced temporal model (a bidirectional long short-term memory network combined with a time attention mechanism) can be used to generate temporal change features that simultaneously contain past and future contextual information, thereby accurately capturing long-distance temporal dependencies and achieving high-precision, automated, and robust recognition of complex nursing behaviors.

[0117] As one embodiment of this application, the above-mentioned behavior recognition model may further include a feature fusion and classification module, which may include a third fully connected layer and a fourth fully connected layer. In this case, the step of concatenating the relevant behavior features, the aggregated behavior features, and the temporal change features, and determining the nursing behavior recognition result based on the concatenated behavior features, may include:

[0118] The third fully connected layer concatenates the relevant behavioral features, the aggregated behavioral features, and the temporal change features to obtain concatenated behavioral features, and inputs the concatenated behavioral features into the fourth fully connected layer; the fourth fully connected layer maps the concatenated behavioral features to the category space to obtain the probability distribution corresponding to each preset nursing behavior category.

[0119] To comprehensively utilize local features, temporal features, and global context to improve the robustness of nursing behavior recognition, this application employs a multi-scale fusion strategy. Specifically, the third fully connected layer can concatenate the relevant behavioral features, aggregated behavioral features, and temporal change features obtained in the aforementioned embodiments to obtain concatenated behavioral features. The above process is expressed by the following formula: .

[0120] in, This represents the fused feature vector used for final classification, which is the concatenated behavioral features. This represents the weight matrix of the fusion layer included in the third fully connected layer. Indicates the last time step The corresponding temporal variation features captured the temporal characteristics at the end of the sequence. Indicates the behavioral characteristics of relevance. The vector obtained by global average pooling provides global statistical information. This indicates the bias term of the fusion layer included in the third fully connected layer. and This is determined during model training.

[0121] Next, the fourth fully connected layer maps the concatenated behavioral features to a category space, obtaining the probability distribution corresponding to each preset nursing behavior category. Specifically, the fourth fully connected layer can process the concatenated behavioral features through a Softmax classification layer to obtain the probability distribution of the current time window belonging to each preset nursing behavior category. The preset nursing behavior categories can be set according to actual needs and are not specifically limited here.

[0122] The formula for the above process is expressed as follows: The Softmax function is used to convert the output into probability values. This represents the probability distribution vector of the prediction. This represents the weight matrix of the classification layer. This represents the bias term for the classification layer.

[0123] As can be seen, in this embodiment, the feature fusion and classification module can concatenate feature vectors from three different sources and identify nursing behaviors based on the concatenated behavioral features. Since the concatenated behavioral features incorporate multi-scale features, the accuracy and robustness of nursing behavior identification can be improved.

[0124] As one embodiment of this application, a schematic diagram of the above-mentioned behavior recognition model can be shown in Figures 2(a) and 2(b), and the processing flow in Figures 2(a) and 2(b) is continuous. In Figure 2(a), the feature selection module 201 of the behavior recognition model acquires the input behavior data to be recognized. The sequence length of the behavior data to be recognized can be 20, and the number of features can be 70.

[0125] First, one-dimensional global average pooling is performed on the behavior data to be identified, resulting in pooled behavior data 202. The first fully connected layer 203 extracts 35-dimensional nonlinear features 204 from the 70-dimensional pooled behavior data 202 using a linear rectified function, and inputs the nonlinear features 204 into the second fully connected layer 205. The deactivation probability of the first fully connected layer 203 is 0.3. The second fully connected layer 205 outputs a feature importance mask 206, which is 70-dimensional, with each element ranging from [0,1]. Next, element-wise multiplication is performed between the behavior data to be identified and the feature importance mask 206 to obtain the filtered behavior features 207.

[0126] The filtered behavioral features 207 have a dimension of 70 and are divided into four groups: 1-12 dimensions are quaternion orientation data 208, 13-24 dimensions are quaternion derivatives 209, 25-48 dimensions are velocity and acceleration 210, and 49-70 dimensions are magnetic field strength data 211. The correlation perception module 212 uses a linear rectified function to perform a one-dimensional convolution operation on the above four groups of data to obtain h1-h4.

[0127] The correlation sensing module 212 uses the following formula to calculate the linear correlation matrix between each group of data: Since i ≠ j, c can be calculated. 12 c 13 c 14 c 23 c 24 and c 34 With c 12 For example, c 12This represents the linear correlation matrix between h1 and h2. The correlation perception module 212 fuses the linear correlation matrix with the filtered behavioral features 207 to obtain the concatenated behavioral features. It then extracts the temporal features of the concatenated behavioral features to obtain the correlated behavioral features 213. The correlated behavioral features 213 are then input into the bidirectional long short-term memory network 215 and the multi-head attention mechanism network of the temporal modeling module 214 in Figure 2(b).

[0128] In Figure 2(b), on the one hand, the input layer 216 of the bidirectional long short-term memory network 215 inputs the relevant behavioral features 213 into the forward and backward layers. The forward layer extracts the forward temporal change features corresponding to the relevant behavioral features 213, and the backward layer extracts the backward temporal change features corresponding to the relevant behavioral features 213. The activation layer 217 fuses the forward and backward temporal change features to obtain the temporal change features, and inputs the temporal change features into the output layer 218.

[0129] On the other hand, the multi-head attention mechanism network extracts the Q (query) matrix, K (key) matrix, and V (value) matrix corresponding to the relevance behavioral features 213, all three matrices being 128-dimensional. These three matrices are then processed by linear layers, resulting in 128-dimensional matrices. Scaling dot product attention is then calculated using these processed matrices to obtain... The calculation results are concatenated to obtain the matrix corresponding to the attention weights.

[0130] The temporal modeling module 214 performs residual connection and layer normalization on the matrices corresponding to the temporal change features and attention weights to obtain 128-dimensional aggregated behavioral features, and inputs the aggregated behavioral features into the feature fusion and classification module 219.

[0131] The feature fusion and classification module 219 performs global average pooling on the aggregated behavioral features and inputs the pooled behavioral features into the third fully connected layer 220. The third fully connected layer 220 concatenates the relevant behavioral features 213, the aggregated behavioral features, and the temporal variation features through a linear rectified function to obtain a 64-dimensional concatenated behavioral feature, which is then input into the fourth fully connected layer 221. The deactivation probability of the third fully connected layer 220 during random deactivation is 0.3.

[0132] The fourth fully connected layer 221 maps the concatenated behavioral features to the category space, obtaining the probability corresponding to each preset nursing behavior category, with the total number being the number of categories. The probabilities are then processed using a normalized exponential function to output the nursing behavior recognition result.

[0133] Corresponding to the above-described nursing behavior recognition method, this application also provides a method for training a behavior recognition model. The following describes the method for training a behavior recognition model provided in this application.

[0134] like Figure 3 As shown, a method for training a behavior recognition model includes:

[0135] S301, Obtain sample behavior data and corresponding recognition result labels.

[0136] The sample behavior data includes behavior data collected by multiple motion sensors, which are worn on different parts of the caregiver's body.

[0137] S302, the sample behavior data is input into the initial behavior recognition model, so that the feature selection module included in the initial behavior recognition model determines the importance of each sample behavior data collected by the motion sensor based on the noise contained in the sample behavior data, and determines the filtered sample behavior features according to each sample behavior data and its corresponding importance, and inputs the filtered sample behavior features into the correlation perception module.

[0138] S303, the correlation perception module uses the current model parameters to fuse the filtered sample behavior features based on the motion dependency between actions reflected by the filtered sample behavior features corresponding to each motion sensor, to obtain sample correlation behavior features, and inputs the sample correlation behavior features into the time series modeling module.

[0139] S304, the time series modeling module, using the current model parameters, determines the sample time series change features for each time step based on the time series change pattern between the sample correlation behavior features corresponding to the time step and the associated time step. Based on the time series change features of each sample and its corresponding contribution, the module determines the aggregated sample behavior features and inputs the aggregated sample behavior features into the feature fusion and classification module.

[0140] Wherein, the associated time step and the current time step satisfy a temporal correlation relationship in terms of time sequence, and the degree of contribution characterizes the degree of influence of the corresponding sample temporal change characteristics on nursing behavior recognition.

[0141] S305, the feature fusion and classification module uses the current model parameters to concatenate the sample correlation behavior features, the aggregated sample behavior features, and the sample temporal change features, and determines the prediction and recognition result based on the concatenated sample behavior features.

[0142] S306, Based on the difference between the predicted recognition result and the recognition result label, adjust the model parameters of the initial behavior recognition model until the initial behavior recognition model meets the convergence condition, and obtain the trained behavior recognition model.

[0143] As can be seen, the solution provided in this application uses sample behavioral data to train a behavior recognition model for identifying nursing behaviors. This model utilizes a deep neural network architecture that combines adaptive feature selection, correlation perception processing, and a bidirectional long short-term memory network with a temporal attention mechanism. It performs in-depth analysis and fusion of behavioral data collected by motion sensors across multiple dimensions, including quaternion direction, angular velocity, linear velocity, and magnetic field strength, thereby achieving high-precision automatic classification and recognition of complex nursing behavior sequences. This solution can be widely applied in medical and nursing scenarios such as hospitals, nursing homes, and community rehabilitation centers. It assists in the automated recording of nursing workload, real-time monitoring of nursing quality, and optimized management of medical processes. It aims to solve the problems of strong subjectivity, time-consuming and labor-intensive methods, and the tendency to generate recording errors inherent in traditional manual observation and recording of nursing activities, providing data support and decision-making basis for intelligent nursing management.

[0144] In step S301, the electronic device can acquire sample behavior data and corresponding recognition result tags. The sample behavior data may include behavior data collected by multiple motion sensors in a real clinical nursing environment. The recognition result tags can be categories of nursing behaviors recorded by the nursing staff themselves.

[0145] To accommodate the input requirements of deep learning models and ensure the stability of model training, the aforementioned sample behavior data can be obtained by standardizing the raw data collected by motion sensors. The specific method of standardization has been explained above and will not be repeated here. Next, a non-overlapping sliding window strategy can be used to divide the time steps according to a preset time window length, thereby segmenting the continuous time series and obtaining standardized sample behavior data within the preset time window length. This segmentation method ensures that each window contains sufficient temporal context for nursing behavior recognition, prevents data leakage, and ensures the consistency of the recognition result labels within the window.

[0146] In one implementation, a stratified sampling strategy can be used to divide the dataset to which the sample behavioral data belongs into a training set, a validation set, and a test set, so as to ensure that the distribution ratio of various nursing behaviors is consistent in each set.

[0147] Next, the electronic device can input the sample behavior data into the initial behavior recognition model. The initial behavior recognition model can process the sample behavior data based on the current model parameters of each module and output the predicted recognition result, i.e., execute steps S302 to S305.

[0148] Since the data processing procedures of the model are corresponding in the training and application phases, and the data processing procedures of the model in the application phase have been explained above, the data processing procedures performed by the model in the training phase will not be described in detail here.

[0149] After obtaining the predicted recognition result, the electronic device can adjust the model parameters of the initial behavior recognition model according to the difference between the predicted recognition result and the recognition result label, until the initial behavior recognition model meets the convergence condition and the trained behavior recognition model is obtained, i.e., step S306 is executed.

[0150] Specifically, the electronic device can calculate the value of the loss function based on the difference between the predicted recognition result and the recognized label. Following the direction that reduces the value of the loss function, the AdamW (decoupling weight decay) optimizer is used to update the parameters of the initial behavior recognition model until the loss function converges, or the number of iterations reaches a preset number, i.e., the initial behavior recognition model meets the convergence condition. At this point, the trained behavior recognition model can be obtained.

[0151] In the above process, the ReduceLROnPlateau scheduler can be used to dynamically adjust the learning rate. Specifically, when the validation set loss corresponding to the initial behavior recognition model does not decrease within a certain number of rounds, the learning rate can be halved (the decay factor can be 0.5). For example, the initial learning rate of the AdamW optimizer can be... Weight decay can be... .

[0152] To ensure the robustness of the behavior recognition model under noisy data, label smoothing can be used. The cross-entropy loss function is used to prevent the behavior recognition model from overfitting to noisy labels.

[0153] One implementation approach is to employ a regularization strategy, introducing an early stopping mechanism by setting the patience value to 100 rounds, which means setting the preset number of rounds to 100. Based on this, gradient clipping is applied, setting the maximum norm to 1.0 to prevent gradient explosion.

[0154] When the categories of sample behavior data are extremely imbalanced (ratio reaching 156:1), an inverse frequency weighting strategy can be adopted to increase the penalty weight of the behavior recognition model for the minority category, thereby enhancing the behavior recognition model's attention to the minority category.

[0155] Through systematic validation on a dataset containing 7,631,843 sample behavioral data, the nursing behavior recognition system (CareAttenNet) proposed in this application achieves significantly better results than existing technologies, as explained in the following three aspects:

[0156] Firstly, as shown in Tables 1-3, the experimental results demonstrate that the nursing behavior recognition system proposed in this application achieves an accuracy of 77.36% on the validation set and 60.00% on the test set. In comparison, the traditional CNN-LSTM (Convolutional Neural Network-Long Short-Term Memory) achieves an accuracy of only 57.92% on the test set, Correlation CNN (Correlation-AwareConvolutional Neural Network) achieves only 54.62%, Attention LSTM (Attention Long Short-Term Memory) achieves only 58%, and Feature-Selective Network achieves only 52.86%. This application comprehensively surpasses the baseline models compared with CareAttenNet in terms of accuracy, demonstrating excellent overall performance.

[0157]

[0158]

[0159]

[0160] Secondly, in terms of accuracy, the nursing behavior recognition system achieved a high precision rate of 79.39% on the validation set, demonstrating conservative and reliable predictive behavior, which is particularly important in medical scenarios with low tolerance for error.

[0161] Thirdly, ablation experiments further confirmed the effectiveness of each module in the behavior recognition model proposed in this application. As shown in Tables 4-6, the temporal modeling module alone achieved an accuracy of 78.33% on the test set, demonstrating the core role of temporal modeling in nursing behavior recognition. Combining the feature selection module with the temporal modeling module yielded a high accuracy of 77.40%, validating the potential for collaborative work between feature selection and temporal analysis.

[0162]

[0163]

[0164]

[0165] In summary, this application effectively solves the challenges of high-dimensional noise, lack of physical constraints, and complex temporal dependencies in nursing activity identification by integrating adaptive feature selection, relevance perception, and temporal attention mechanisms, providing strong technical support for realizing intelligent nursing workload monitoring and quality assessment.

[0166] Corresponding to the above-described nursing behavior recognition method, this application also provides a nursing behavior recognition device. The nursing behavior recognition device provided in this application embodiment is described below.

[0167] like Figure 4 As shown, a nursing behavior recognition device includes:

[0168] The first data acquisition module 401 is used to acquire behavior data to be identified, wherein the behavior data to be identified includes behavior data collected by multiple motion sensors, and the multiple motion sensors are worn on different parts of the caregiver's body.

[0169] The first feature filtering module 402 is used to determine the importance of each behavioral data collected by each motion sensor based on the noise contained in the behavioral data, and to determine the filtered behavioral features based on each behavioral data and its corresponding importance.

[0170] The first feature fusion module 403 is used to fuse the filtered behavioral features based on the motion dependency relationship between actions reflected by the filtered behavioral features corresponding to each motion sensor to obtain relevant behavioral features.

[0171] The first temporal feature extraction module 404 is used to determine the temporal change features for each time step based on the temporal change pattern between the relevant behavioral features of the time step and the associated time step, wherein the associated time step and the current time step satisfy a temporal correlation relationship in time.

[0172] The first feature aggregation module 405 is used to determine the aggregated behavioral features based on each temporal change feature and its corresponding contribution degree, wherein the contribution degree characterizes the degree of influence of the corresponding temporal change feature on nursing behavior recognition.

[0173] The first result output module 406 is used to splice the relevant behavioral features, the aggregated behavioral features, and the temporal change features, and determine the nursing behavior recognition result based on the spliced ​​behavioral features.

[0174] As can be seen, in the solution provided in this application embodiment, since noise can interfere with the accuracy of nursing behavior recognition, in order to filter out behavioral data containing significant noise from the behavioral data collected by each motion sensor, it can be determined that the importance of behavioral data containing significant noise is low. Furthermore, by determining the filtered behavioral features based on each behavioral data point and its corresponding importance, the interference of behavioral data containing significant noise on nursing behavior recognition can be reduced. When nursing staff perform nursing behaviors, multiple body parts may cooperate with each other, such as the cooperation of both hands, or the cooperation of the ankles and thighs. At this time, there is a certain correlation between the behavioral features corresponding to the above body parts. Based on the motion dependency relationship between the actions reflected by the filtered behavioral features corresponding to each motion sensor, the filtered behavioral features can be fused to obtain correlated behavioral features, thereby reflecting the correlation of motion features when different body parts cooperate. The actions of nursing staff performing nursing behaviors are usually continuously changing. Therefore, determining the temporal change features based on the temporal change pattern between the correlated behavioral features corresponding to multiple time steps that satisfy temporal correlation can more accurately reflect the nursing behaviors performed by nursing staff. The contribution of each temporal variation feature is related to its influence on nursing behavior recognition; that is, the greater the influence, the higher the contribution of the temporal variation feature. Based on each temporal variation feature and its corresponding contribution, aggregated behavioral features are determined. In the aggregated behavioral features, temporal variation features with a greater influence on nursing behavior recognition occupy a higher proportion, allowing the aggregated behavioral features to more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. The relevant behavioral features, aggregated behavioral features, and temporal variation features are concatenated to obtain the concatenated behavioral features. Since relevant behavioral features can reflect the correlation of movement features when different body parts coordinate, the aggregated behavioral features can more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. Temporal variation features can more accurately reflect the nursing behaviors performed by nurses and reduce the interference of noisy behavioral data on nursing behavior recognition. Therefore, determining the nursing behavior recognition results based on the concatenated behavioral features can improve the accuracy of nursing behavior recognition.

[0175] As one embodiment of this application, the first feature filtering module 402 described above may include:

[0176] The feature selection submodule is used to input the behavior data into a pre-trained behavior recognition model, so that the feature selection module included in the behavior recognition model can extract the nonlinear features of the behavior data and determine the importance of the behavior data based on the nonlinear features. The nonlinear features are used to reflect the noise level, and the importance is negatively correlated with the noise level.

[0177] As one embodiment of this application, the feature selection module may include a first fully connected layer and a second fully connected layer. In this case, the feature filtering submodule may include:

[0178] The feature extraction unit is used to extract the nonlinear features of the behavioral data from the first fully connected layer and input the nonlinear features into the second fully connected layer.

[0179] A mask generation unit is used by the second fully connected layer to generate a feature importance mask based on the nonlinear features, wherein each element of the feature importance mask is used to reflect the importance of the behavioral data.

[0180] As one embodiment of this application, the behavior recognition model may further include a relevance perception module. In this case, the first feature fusion module 403 may include:

[0181] The matrix calculation submodule is used by the correlation perception module to calculate the linear correlation matrix between the filtered behavioral features corresponding to each motion sensor using a linear correlation calculation function, wherein the elements in the linear correlation matrix are used to characterize the motion dependency relationship.

[0182] The first feature splicing submodule is used to multiply the filtered behavioral features with the linear correlation matrix, and splice the multiplication result with the filtered behavioral features to obtain the spliced ​​behavioral features;

[0183] The temporal feature extraction submodule is used to extract the temporal features of the concatenated behavioral features to obtain the relevant behavioral features.

[0184] As one embodiment of this application, the behavior recognition model may further include a temporal modeling module. The temporal modeling module may include a bidirectional long short-term memory network, which may include a forward layer, a backward layer, and an activation layer. In this case, the first temporal feature extraction module 404 may include:

[0185] The forward feature extraction submodule is used to extract the temporal change pattern between the relevant behavioral features corresponding to the current time step and the forward time step, determine the forward temporal change features, and input the forward temporal change features into the activation layer, wherein the forward time step is an associated time step earlier than the current time step;

[0186] The backward feature extraction submodule is used to extract the temporal change pattern between the relevant behavioral features corresponding to the current time step and the backward time step, determine the backward temporal change features, and input the backward temporal change features into the activation layer, wherein the backward time step is the associated time step that is later than the current time step;

[0187] The temporal feature fusion submodule is used by the activation layer to fuse the forward temporal change features and the backward temporal change features to obtain temporal change features.

[0188] As one embodiment of this application, the behavior recognition model may further include a feature fusion and classification module, which may include a third fully connected layer and a fourth fully connected layer. In this case, the first result output module 406 may include:

[0189] The second feature splicing submodule is used by the third fully connected layer to splice the relevant behavioral features, the aggregated behavioral features, and the temporal change features to obtain the spliced ​​behavioral features, and input the spliced ​​behavioral features into the fourth fully connected layer.

[0190] The feature mapping submodule is used by the fourth fully connected layer to map the concatenated behavioral features to the category space to obtain the probability distribution corresponding to each preset nursing behavior category.

[0191] Corresponding to the above-described method for training a behavior recognition model, this application also provides a training apparatus for a behavior recognition model. The following describes the training apparatus for a behavior recognition model provided in this application.

[0192] like Figure 5 As shown, a training device for a behavior recognition model includes:

[0193] The second data acquisition module 501 is used to acquire sample behavior data and corresponding recognition result labels. The sample behavior data includes behavior data collected by multiple motion sensors, which are worn on different parts of the caregiver's body.

[0194] The second feature filtering module 502 is used to input the sample behavior data into the initial behavior recognition model, so that the feature selection module included in the initial behavior recognition model can determine the importance of each sample behavior data collected by the motion sensor based on the noise contained in the sample behavior data, and determine the filtered sample behavior features according to each sample behavior data and its corresponding importance, and input the filtered sample behavior features into the correlation perception module.

[0195] The second feature fusion module 503 is used by the correlation perception module to fuse the filtered sample behavior features based on the motion dependency relationship between actions reflected by the filtered sample behavior features corresponding to each motion sensor through the current model parameters, to obtain sample correlation behavior features, and input the sample correlation behavior features into the time series modeling module.

[0196] The second feature aggregation module 504 is used by the temporal modeling module to determine the sample temporal change features for each time step based on the temporal change pattern between the sample correlation behavior features corresponding to the time step and the associated time step, using the current model parameters. Based on the temporal change features of each sample and its corresponding contribution, the aggregated sample behavior features are determined, and the aggregated sample behavior features are input into the feature fusion and classification module. The associated time step and the current time step satisfy a temporal correlation relationship, and the contribution level characterizes the degree of influence of the corresponding sample temporal change features on nursing behavior recognition.

[0197] The second result output module 505 is used by the feature fusion and classification module to concatenate the sample correlation behavior features, the aggregated sample behavior features, and the sample temporal change features using the current model parameters, and to determine the prediction and recognition result based on the concatenated sample behavior features.

[0198] The parameter adjustment module 506 is used to adjust the model parameters of the initial behavior recognition model according to the difference between the predicted recognition result and the recognition result label, until the initial behavior recognition model meets the convergence condition and the trained behavior recognition model is obtained.

[0199] As can be seen, the solution provided in this application uses sample behavioral data to train a behavior recognition model for identifying nursing behaviors. This model utilizes a deep neural network architecture that combines adaptive feature selection, correlation perception processing, and a bidirectional long short-term memory network with a temporal attention mechanism. It performs in-depth analysis and fusion of behavioral data collected by motion sensors across multiple dimensions, including quaternion direction, angular velocity, linear velocity, and magnetic field strength, thereby achieving high-precision automatic classification and recognition of complex nursing behavior sequences. This solution can be widely applied in medical and nursing scenarios such as hospitals, nursing homes, and community rehabilitation centers. It assists in the automated recording of nursing workload, real-time monitoring of nursing quality, and optimized management of medical processes. It aims to solve the problems of strong subjectivity, time-consuming and labor-intensive methods, and the tendency to generate recording errors inherent in traditional manual observation and recording of nursing activities, providing data support and decision-making basis for intelligent nursing management.

[0200] This application also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0201] Memory 603 is used to store computer programs;

[0202] The processor 601, when executing the program stored in the memory 603, implements the nursing behavior recognition method or behavior recognition model training method described in any of the above embodiments.

[0203] As can be seen, in the solution provided in this application embodiment, since noise can interfere with the accuracy of nursing behavior recognition, in order to filter out behavioral data containing significant noise from the behavioral data collected by each motion sensor, it can be determined that the importance of behavioral data containing significant noise is low. Furthermore, by determining the filtered behavioral features based on each behavioral data point and its corresponding importance, the interference of behavioral data containing significant noise on nursing behavior recognition can be reduced. When nursing staff perform nursing behaviors, multiple body parts may cooperate with each other, such as the cooperation of both hands, or the cooperation of the ankles and thighs. At this time, there is a certain correlation between the behavioral features corresponding to the above body parts. Based on the motion dependency relationship between the actions reflected by the filtered behavioral features corresponding to each motion sensor, the filtered behavioral features can be fused to obtain correlated behavioral features, thereby reflecting the correlation of motion features when different body parts cooperate. The actions of nursing staff performing nursing behaviors are usually continuously changing. Therefore, determining the temporal change features based on the temporal change pattern between the correlated behavioral features corresponding to multiple time steps that satisfy temporal correlation can more accurately reflect the nursing behaviors performed by nursing staff. The contribution of each temporal variation feature is related to its influence on nursing behavior recognition; that is, the greater the influence, the higher the contribution of the temporal variation feature. Based on each temporal variation feature and its corresponding contribution, aggregated behavioral features are determined. In the aggregated behavioral features, temporal variation features with a greater influence on nursing behavior recognition occupy a higher proportion, allowing the aggregated behavioral features to more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. The relevant behavioral features, aggregated behavioral features, and temporal variation features are concatenated to obtain the concatenated behavioral features. Since relevant behavioral features can reflect the correlation of movement features when different body parts coordinate, the aggregated behavioral features can more effectively highlight temporal variation features with a significant impact on nursing behavior recognition. Temporal variation features can more accurately reflect the nursing behaviors performed by nurses and reduce the interference of noisy behavioral data on nursing behavior recognition. Therefore, determining the nursing behavior recognition results based on the concatenated behavioral features can improve the accuracy of nursing behavior recognition.

[0204] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0205] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0206] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0207] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0208] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described nursing behavior recognition methods or behavior recognition model training methods.

[0209] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the nursing behavior recognition methods or behavior recognition model training methods in the above embodiments.

[0210] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0211] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0212] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0213] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for identifying nursing behaviors, characterized in that, The method includes: Acquire behavioral data to be identified, wherein the behavioral data to be identified includes behavioral data collected by multiple motion sensors, and the multiple motion sensors are worn on different parts of the caregiver's body; For each behavioral data collected by a motion sensor, the importance of the behavioral data is determined based on the noise contained in the data, and the filtered behavioral features are determined based on each behavioral data and its importance. Based on the motion dependency between actions reflected by the filtered behavioral features corresponding to each motion sensor, the filtered behavioral features are fused to obtain relevant behavioral features. For each time step, the temporal change characteristics are determined based on the temporal change pattern between the relevant behavioral characteristics of the time step and the associated time step, wherein the associated time step and the current time step satisfy a temporal correlation relationship in terms of time. Based on each temporal change feature and its corresponding contribution level, the aggregated behavioral features are determined, wherein the contribution level characterizes the degree of influence of the corresponding temporal change feature on nursing behavior identification; The relevant behavioral features, the aggregated behavioral features, and the temporal change features are concatenated, and the nursing behavior identification result is determined based on the concatenated behavioral features.

2. The method according to claim 1, characterized in that, The step of determining the importance of the behavioral data based on the noise contained in the behavioral data includes: The behavior data is input into a pre-trained behavior recognition model so that the feature selection module included in the behavior recognition model can extract the nonlinear features of the behavior data. The importance of the behavioral data is determined based on the nonlinear feature, wherein the nonlinear feature is used to reflect the noise level, and the importance is negatively correlated with the noise level.

3. The method according to claim 2, characterized in that, The feature selection module includes a first fully connected layer and a second fully connected layer; The step of the feature selection module extracting nonlinear features from the behavioral data and determining the importance of the behavioral data based on the nonlinear features includes: The first fully connected layer extracts the nonlinear features of the behavioral data and inputs the nonlinear features into the second fully connected layer; The second fully connected layer generates a feature importance mask based on the nonlinear features, wherein each element of the feature importance mask is used to reflect the importance of the behavioral data.

4. The method according to claim 2, characterized in that, The behavior recognition model also includes a correlation perception module; The step of fusing the filtered behavioral features based on the motion dependencies between actions reflected by the behavioral features corresponding to each motion sensor to obtain correlated behavioral features includes: The correlation perception module uses a linear correlation calculation function to calculate the linear correlation matrix between the filtered behavioral features corresponding to each motion sensor, wherein the elements in the linear correlation matrix are used to characterize the motion dependency relationship. The filtered behavioral features are multiplied by the linear correlation matrix, and the multiplication result is concatenated with the filtered behavioral features to obtain the concatenated behavioral features. The temporal features of the concatenated behavioral features are extracted to obtain the relevant behavioral features.

5. The method according to claim 2, characterized in that, The behavior recognition model also includes a temporal modeling module, which includes a bidirectional long short-term memory network, comprising a forward layer, a backward layer, and an activation layer. The step of determining the temporal change characteristics based on the temporal change pattern between the relevant behavioral characteristics corresponding to the time step and the associated time step includes: The forward layer extracts the temporal change pattern between the relevant behavioral features corresponding to the current time step and the forward time step, determines the forward temporal change features, and inputs the forward temporal change features into the activation layer, wherein the forward time step is an associated time step earlier than the current time step; The backward layer extracts the temporal change pattern between the relevant behavioral features corresponding to the current time step and the backward time step, determines the backward temporal change features, and inputs the backward temporal change features into the activation layer, wherein the backward time step is the associated time step that is later than the current time step; The activation layer fuses the forward temporal variation features and the backward temporal variation features to obtain the temporal variation features.

6. The method according to claim 2, characterized in that, The behavior recognition model also includes a feature fusion and classification module, which includes a third fully connected layer and a fourth fully connected layer; The step of concatenating the relevant behavioral features, the aggregated behavioral features, and the temporal change features, and determining the nursing behavior identification result based on the concatenated behavioral features, includes: The third fully connected layer concatenates the relevant behavioral features, the aggregated behavioral features, and the temporal change features to obtain concatenated behavioral features, and then inputs the concatenated behavioral features into the fourth fully connected layer. The fourth fully connected layer maps the spliced ​​behavioral features to the category space to obtain the probability distribution corresponding to each preset nursing behavior category.

7. A training method for a behavior recognition model, characterized in that, The method includes: Acquire sample behavior data and corresponding identification result labels, wherein the sample behavior data includes behavior data collected by multiple motion sensors, and the multiple motion sensors are worn on different parts of the caregiver's body; The sample behavior data is input into the initial behavior recognition model, so that the feature selection module included in the initial behavior recognition model determines the importance of each sample behavior data collected by the motion sensor based on the noise contained in the sample behavior data, and determines the filtered sample behavior features based on each sample behavior data and its corresponding importance, and inputs the filtered sample behavior features into the correlation perception module. The correlation perception module uses the current model parameters to fuse the filtered sample behavior features based on the motion dependency between actions reflected by the filtered sample behavior features corresponding to each motion sensor, to obtain sample correlation behavior features, and then inputs the sample correlation behavior features into the time series modeling module. The temporal modeling module, using the current model parameters, determines the temporal change features of samples for each time step based on the temporal change pattern between the sample correlation behavior features corresponding to the time step and the associated time step. Based on each sample temporal change feature and its corresponding contribution level, the module determines the aggregated sample behavior features and inputs the aggregated sample behavior features into the feature fusion and classification module. The associated time step and the current time step satisfy a temporal correlation relationship, and the contribution level characterizes the degree of influence of the corresponding sample temporal change features on nursing behavior recognition. The feature fusion and classification module uses the current model parameters to concatenate the sample correlation behavior features, the aggregated sample behavior features, and the sample temporal change features, and determines the prediction and recognition result based on the concatenated sample behavior features. Based on the difference between the predicted recognition result and the recognition result label, the model parameters of the initial behavior recognition model are adjusted until the initial behavior recognition model meets the convergence condition, thus obtaining the trained behavior recognition model.

8. A nursing behavior recognition device, characterized in that, The device includes: The first data acquisition module is used to acquire behavior data to be identified, wherein the behavior data to be identified includes behavior data collected by multiple motion sensors, and the multiple motion sensors are worn on different parts of the caregiver's body; The first feature filtering module is used to determine the importance of each behavioral data collected by each motion sensor based on the noise contained in the behavioral data, and to determine the filtered behavioral features based on each behavioral data and its corresponding importance. The first feature fusion module is used to fuse the filtered behavioral features based on the motion dependency relationship between actions reflected by the filtered behavioral features corresponding to each motion sensor to obtain relevant behavioral features. The first temporal feature extraction module is used to determine the temporal change features for each time step based on the temporal change pattern between the relevant behavioral features of the time step and the associated time step, wherein the associated time step and the current time step satisfy a temporal correlation relationship in time. The first feature aggregation module is used to determine the aggregated behavioral features based on each temporal change feature and its corresponding contribution level, wherein the contribution level characterizes the degree of influence of the corresponding temporal change feature on nursing behavior recognition. The first result output module is used to concatenate the relevant behavioral features, the aggregated behavioral features, and the temporal change features, and determine the nursing behavior recognition result based on the concatenated behavioral features.

9. A training device for a behavior recognition model, characterized in that, The device includes: The second data acquisition module is used to acquire sample behavior data and corresponding recognition result labels. The sample behavior data includes behavior data collected by multiple motion sensors, which are worn on different parts of the caregiver's body. The second feature filtering module is used to input the sample behavior data into the initial behavior recognition model, so that the feature selection module included in the initial behavior recognition model can determine the importance of each sample behavior data collected by the motion sensor based on the noise contained in the sample behavior data through the current model parameters, and determine the filtered sample behavior features according to each sample behavior data and its corresponding importance, and input the filtered sample behavior features into the correlation perception module. The second feature fusion module is used by the correlation perception module to fuse the filtered sample behavior features based on the motion dependency relationship between actions reflected by the filtered sample behavior features corresponding to each motion sensor, using the current model parameters, to obtain sample correlation behavior features, and input the sample correlation behavior features into the time series modeling module. The second feature aggregation module is used by the temporal modeling module to determine the sample temporal change features for each time step based on the temporal change pattern between the sample correlation behavior features corresponding to the time step and the associated time step, using the current model parameters. Based on the temporal change features of each sample and its corresponding contribution, the aggregated sample behavior features are determined, and the aggregated sample behavior features are input into the feature fusion and classification module. The associated time step and the current time step satisfy a temporal correlation relationship, and the contribution level characterizes the degree of influence of the corresponding sample temporal change features on nursing behavior recognition. The second result output module is used by the feature fusion and classification module to concatenate the sample correlation behavior features, the aggregated sample behavior features, and the sample temporal change features using the current model parameters, and to determine the prediction and recognition result based on the concatenated sample behavior features. The parameter adjustment module is used to adjust the model parameters of the initial behavior recognition model according to the difference between the predicted recognition result and the recognition result label, until the initial behavior recognition model meets the convergence condition and the trained behavior recognition model is obtained.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images