Intelligent management system for home safety and health of old people

Abstract

The invention relates to an intelligent management system for home safety and health of old people. The method comprises the following steps: carrying out key frame sampling on an obtained behavior state monitoring video in a home safety health management server; performing feature extraction on the time sequence of the behavior state monitoring key frame through a behavior state feature extractor based on a deep neural network; inputting the time sequence of the behavior state semantic feature vector into a behavior state semantic time sequence propagation fusion network to obtain a behavior state time sequence propagation aggregation semantic representation vector as the behavior state time sequence propagation aggregation semantics of the old person object; and determining whether the monitored old person falls down or not, and determining whether to generate an early warning prompt and send the early warning prompt to remote terminal equipment or not. Thus, the behavior pattern and state of the old at home can be monitored in real time, early warning of falling of the old is achieved, falling and accidental injury are prevented, a safe and healthy home living environment is provided for the old, and meanwhile the care burden of family members is relieved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to an intelligent management system for elderly people's home safety and health. Background Art

[0002] As the global population ages, the safety and health of the elderly at home are receiving increasing attention from society. As the elderly age, their physical functions gradually decline, making them more susceptible to safety issues such as falls and sudden illnesses. At the same time, many elderly people suffer from chronic diseases and require long-term health care and management.

[0003] However, traditional home safety and health management for the elderly mainly relies on the observation and care of family members or caregivers. With the changes in social structure, family members may not be able to monitor the elderly around the clock, and the cost of professional caregivers is high, resulting in insufficient human resources. Some families can install cameras at home to monitor the safety and health of the elderly, but this monitoring method is usually intermittent and cannot achieve real-time monitoring and management of the elderly's behavior status. It is easy to miss emergencies, resulting in untimely rescue and affecting the safety of the elderly. In addition, these existing home safety monitoring methods for the elderly will lead to delayed reminders of safety issues for the elderly, that is, alarms and rescue are usually issued when the elderly fall or encounter other safety problems, affecting the safety and health of the elderly.

[0004] Therefore, an intelligent management system for the safety and health of the elderly at home is desired. Summary of the invention

[0005] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] In a first aspect, the present disclosure provides an intelligent management system for elderly people's home safety and health, the system comprising:

[0007] The elderly behavior status video acquisition module is used to collect the behavior status monitoring video of the monitored elderly object through the smart camera;

[0008] A monitoring video data transmission module, used to transmit the behavior status monitoring video to a backend home safety and health management server via a wireless communication network;

[0009] A behavior state monitoring video key frame sampling module is used to perform key frame sampling on the behavior state monitoring video on the home safety and health management server to obtain a time series of behavior state monitoring key frames;

[0010] The module for extracting semantic features of the behavior state of the elderly is used to extract features of each behavior state monitoring key frame in the time series of the behavior state monitoring key frames through a behavior state feature extractor based on a deep neural network to obtain a time series of behavior state semantic feature vectors;

[0011] The elderly object behavior state semantic temporal propagation aggregation module is used to input the time series of the behavior state semantic feature vector into the behavior state semantic temporal propagation fusion network to obtain the behavior state temporal propagation aggregation semantic representation vector as the elderly object behavior state temporal propagation aggregation semantics;

[0012] The elderly fall warning and warning information transmission module is used to determine whether the monitored elderly object has fallen based on the time-series propagation aggregation semantics of the elderly object's behavior state, and determine whether to generate a warning prompt and send it to a remote terminal device.

[0013] Optionally, the behavior state feature extractor based on deep neural network is a behavior state feature extractor based on DenseNet.

[0014] Optionally, the semantic temporal propagation aggregation module for the behavior state of the elderly object includes: a vector fusion unit, which is used to fuse the first s behavior state semantic feature vectors in the time series of the behavior state semantic feature vectors to obtain a front-end node semantic information fusion feature vector; a weighted optimization unit, which is used to perform weighted optimization on the s-th behavior state semantic feature vector based on the importance of sequence nodes to obtain a weighted optimized behavior state semantic feature vector; a multi-layer perceptron processing unit, which is used to sum the front-end node semantic information fusion feature vector and the weighted optimized behavior state semantic feature vector by position and then process them through a multi-layer perceptron to obtain a front-end node semantic temporal propagation aggregation feature vector corresponding to the s-th behavior state semantic feature vector and fused with current node information; a positional summation calculation unit, which is used to calculate the positional sum between the front-end node semantic temporal propagation aggregation feature vectors fused with current node information corresponding to each behavior state semantic feature vector in the sequence of the behavior state semantic feature vectors to obtain the behavior state temporal propagation aggregation semantic representation vector.

[0015] Optionally, the vector fusion unit is used to calculate the positional sum of first s behavior state semantic feature vectors in the time series of the behavior state semantic feature vector to obtain the front-end node semantic information fusion feature vector.

[0016] Optionally, the weighted optimization unit is used to: perform weighted processing on each position feature value in the sth behavior state semantic feature vector using the sum of a trainable preset hyperparameter and one as a weighting coefficient to obtain a weighted optimized behavior state semantic feature vector.

[0017] Optionally, the elderly fall warning and warning information transmission module includes: an elderly object fall detection unit, which is used to input the behavior state time-series propagation aggregation semantic representation vector into a classifier-based behavior monitor to obtain a monitoring result, and the monitoring result is used to indicate whether the monitored elderly object has fallen; a fall warning prompt information generation and transmission unit, which is used to generate a warning prompt in response to the monitoring result that the monitored elderly object has fallen and send it to a remote terminal device.

[0018] Optionally, the elderly object fall detection unit includes: a behavior monitoring subunit, used to input the behavior state temporal propagation aggregation semantic representation vector into the classifier-based behavior monitor to obtain a monitoring result, and the monitoring result is used to indicate whether the monitored elderly object falls.

[0019] Optionally, the behavior monitoring subunit includes: a fully connected encoding secondary subunit, used to use multiple fully connected layers of the classifier to fully connect the behavior state temporal propagation aggregate semantic representation vector to obtain an encoded classification feature vector; and a classification secondary subunit, used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the monitoring result.

[0020] The above technical solution is adopted to sample key frames of the acquired behavior status monitoring video on the home safety and health management server; extract features of the time series of the behavior status monitoring key frames respectively through the behavior status feature extractor based on the deep neural network; input the time series of the behavior status semantic feature vector into the behavior status semantic time series propagation fusion network to obtain the behavior status time series propagation aggregation semantic representation vector as the behavior status time series propagation aggregation semantics of the elderly object; and then determine whether the monitored elderly object falls, and determine whether to generate an early warning prompt and send it to the remote terminal device. In this way, the behavior pattern and state of the elderly at home can be monitored in real time, and the fall warning of the elderly can be realized, and falls and accidental injuries can be prevented, providing a safe and healthy home living environment for the elderly, while also reducing the care burden of family members.

[0021] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0023] Figure 1 The present invention is a block diagram of an intelligent management system for elderly people's home safety and health, according to an exemplary embodiment.

[0024] Figure 2 The present invention is a flowchart of a method for intelligently managing the safety and health of the elderly at home according to an exemplary embodiment.

[0025] Figure 3 It is a block diagram of an electronic device according to an exemplary embodiment.

[0026] Figure 4 This is an application scenario diagram of an intelligent management system for elderly people's home safety and health, according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] In order to solve the above problems, the present disclosure provides an intelligent management system for the safety and health of the elderly at home. By sampling the key frames of the acquired behavior status monitoring video on the home safety and health management server; extracting features from the time series of the behavior status monitoring key frames respectively through the behavior status feature extractor based on the deep neural network; inputting the time series of the behavior status semantic feature vector into the behavior status semantic time series propagation fusion network to obtain the behavior status time series propagation aggregation semantic representation vector as the behavior status time series propagation aggregation semantics of the elderly object; and then determining whether the monitored elderly object falls, and determining whether to generate an early warning prompt and send it to the remote terminal device. In this way, the behavior pattern and state of the elderly at home can be monitored in real time, and the fall warning of the elderly can be realized, and falls and accidental injuries can be prevented, so as to provide a safe and healthy home living environment for the elderly, and at the same time, the care burden of family members can be reduced.

[0034] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0035] Figure 1 The present invention is a block diagram of an intelligent management system for elderly people's home safety and health, according to an exemplary embodiment.

[0036] like Figure 1 As shown, the system 100 includes:

[0037] The elderly behavior status video acquisition module 101 is used to acquire the behavior status monitoring video of the monitored elderly object through a smart camera;

[0038] A monitoring video data transmission module 102, for transmitting the behavior status monitoring video to a backend home safety and health management server via a wireless communication network;

[0039] The behavior state monitoring video key frame sampling module 103 is used to perform key frame sampling on the behavior state monitoring video on the home safety and health management server to obtain a time series of behavior state monitoring key frames;

[0040] The elderly behavior state semantic feature extraction module 104 is used to extract features from each behavior state monitoring key frame in the time series of the behavior state monitoring key frames through a behavior state feature extractor based on a deep neural network to obtain a time series of behavior state semantic feature vectors;

[0041] The elderly object behavior state semantic temporal propagation aggregation module 105 is used to input the time series of the behavior state semantic feature vector into the behavior state semantic temporal propagation fusion network to obtain the behavior state temporal propagation aggregation semantic representation vector as the elderly object behavior state temporal propagation aggregation semantics;

[0042] The elderly fall warning and warning information transmission module 106 is used to determine whether the monitored elderly object falls based on the time series propagation aggregation semantics of the elderly object's behavior state, and determine whether to generate a warning prompt and send it to the remote terminal device.

[0043] Among them, the behavior state feature extractor based on deep neural network is a behavior state feature extractor based on DenseNet.

[0044] With the development of computer vision and image processing technology, intelligent monitoring technology can intelligently analyze the monitoring video collected by the camera to realize real-time monitoring and abnormal warning of the behavior patterns of the elderly, and help prevent the occurrence of accidents. Based on this, in the technical solution of this application, a smart management system for the safety and health of the elderly at home is proposed, which can collect the behavior status monitoring video of the elderly object through an intelligent camera, and introduce image processing and analysis algorithms based on artificial intelligence and machine vision technology at the back end to analyze the behavior status monitoring video, so as to capture and identify the behavior pattern and state of the elderly object, and at the same time characterize the propagation aggregation information and correlation characteristics of this behavior state in time series, so as to perform preventive detection and warning of falls for the elderly object, and send the warning information to the remote terminal device to remind family members. In this way, the behavior pattern and state of the elderly at home can be monitored in real time, the fall warning of the elderly can be realized, the falls and accidental injuries can be prevented, and a safe and healthy home living environment can be provided for the elderly, while also reducing the care burden of family members.

[0045] Specifically, in the technical solution of the present application, first, the behavior status monitoring video of the monitored elderly object is collected by an intelligent camera, and the behavior status monitoring video is transmitted to the background home safety and health management server through a wireless communication network. Next, considering that the behavior status monitoring video file of the elderly object usually has a large amount of data, if each frame is processed, it will take up a lot of computing resources and storage space. At the same time, since there are usually some video frames with the same semantics or little change in semantic information in the behavior status monitoring video, if these video frames are feature extracted and characterized, it will cause redundancy of the monitoring video feature information of the elderly behavior status, which will cause a burden for subsequent processing. Based on this, in the technical solution of the present application, it is necessary to sample the key frames of the behavior status monitoring video on the home safety and health management server to obtain the time series of the behavior status monitoring key frames. It should be understood that the key frames usually contain the most representative and information-rich content in the video. By sampling the key frames in the behavior status monitoring video, it can be beneficial to extract the key behavior status features and timing information about the elderly object in the future, avoid processing redundant and irrelevant information, and help the system to analyze the behavior status of the elderly more accurately. In addition, by performing key frame sampling on the behavior status monitoring video, the amount of data that needs to be processed can be effectively reduced, the burden of processing and transmission can be reduced, and the processing efficiency of the system can be higher.

[0046] Then, in order to capture the semantics of the behavior state of the elderly object contained in each behavior state monitoring key frame in the time series of the behavior state monitoring key frame, so as to subsequently use the temporal aggregation information of the semantics of the behavior state of the elderly object under these key frames to identify and predict the behavior patterns of the elderly, such as falling, sitting for a long time, etc., so as to more accurately carry out the fall warning of the elderly, in the technical solution of the present application, each behavior state monitoring key frame in the time series of the behavior state monitoring key frame is further respectively passed through the behavior state feature extractor based on DenseNet to obtain the time series of the behavior state semantic feature vector. It should be understood that DenseNet is a deep neural network structure, which effectively improves the information flow and parameter efficiency of the network through a dense connection mode, which enables DenseNet to extract richer and more effective features from the key frame. Through the processing of the behavior state feature extractor based on DenseNet, feature representations from pixel level to high-level abstraction can be learned, which can capture the behavior state characteristics and key detail semantics of the elderly under each key frame, and provide strong feature support for subsequent elderly behavior analysis and fall warning.

[0047] It should be understood that since each behavior state semantic feature vector in the time series of the behavior state semantic feature vector contains semantic feature information about the behavior state of the elderly object in each key frame, and the behavior semantic features of the elderly object in different key frames have temporal correlation and importance in the time dimension, the temporal correlation features and important semantics between key frames can better integrate and highlight the temporal behavior state and pattern of the elderly, and provide a basis for fall warning for the elderly. Based on this, in order to capture the temporal correlation relationship between the semantic features of the behavior state of the elderly contained in each key frame, and identify the importance of these key frame semantic features in the fall warning task, so as to more accurately and comprehensively understand the behavior patterns and state changes of the elderly, in the technical solution of the present application, the time series of the behavior state semantic feature vector is further input into the behavior state semantic temporal propagation fusion network to obtain the behavior state temporal propagation aggregated semantic representation vector. Through the processing of the behavior state semantic temporal propagation fusion network, the semantic features of the behavior state of the elderly contained in each key frame can be used as a node, so as to transmit and aggregate the information of each node according to the learned behavior semantic relevance and importance between different nodes, which helps to more effectively integrate the behavior semantic features of the elderly objects from different time points and form a more comprehensive and coherent behavior representation. In this way, the system can pay more attention to the key frame semantics of the time points that have an important impact on the analysis of the behavior state of the elderly and the prediction of falls, so as to more effectively capture the temporal evolution process and dynamic changes of the behavior state of the elderly, and provide a more accurate data basis for subsequent behavior monitoring and fall warning.

[0048] In one embodiment of the present disclosure, the semantic temporal propagation aggregation module for the behavior state of the elderly object includes: a vector fusion unit, which is used to fuse the first s behavior state semantic feature vectors in the time series of the behavior state semantic feature vector to obtain a front-end node semantic information fusion feature vector; a weighted optimization unit, which is used to perform weighted optimization on the s-th behavior state semantic feature vector based on the importance of sequence nodes to obtain a weighted optimized behavior state semantic feature vector; a multi-layer perceptron processing unit, which is used to sum the front-end node semantic information fusion feature vector and the weighted optimized behavior state semantic feature vector by position and then process them through a multi-layer perceptron to obtain a front-end node semantic temporal propagation aggregation feature vector corresponding to the s-th behavior state semantic feature vector and fused with current node information; a positional summation calculation unit, which is used to calculate the positional sum between the front-end node semantic temporal propagation aggregation feature vectors fused with current node information corresponding to each behavior state semantic feature vector in the sequence of the behavior state semantic feature vectors to obtain the behavior state temporal propagation aggregation semantic representation vector.

[0049] Furthermore, in one embodiment of the present disclosure, the vector fusion unit is used to: calculate the positional sum of the first s behavior state semantic feature vectors in the time series of the behavior state semantic feature vector to obtain the front-end node semantic information fusion feature vector.

[0050] Furthermore, in one embodiment of the present disclosure, the weighted optimization unit is used to: use the sum of a trainable preset hyperparameter and one as a weighting coefficient to weight each position feature value in the sth behavior state semantic feature vector to obtain a weighted optimized behavior state semantic feature vector.

[0051] Specifically, the time series of the behavior state semantic feature vector is input into the behavior state semantic time series propagation fusion network and processed by the following node message propagation fusion formula to obtain the behavior state time series propagation aggregation semantic representation vector; wherein the node message propagation fusion formula is:

[0052]

[0053] Among them, V s is the sth behavior state semantic feature vector in the time series of the behavior state semantic feature vector, V k is the kth behavior state semantic feature vector in the time series of the behavior state semantic feature vector, ∈ is a trainable preset hyperparameter, MLP(·) represents a multi-layer perceptron, n is the number of feature vectors in the time series of the behavior state semantic feature vector, and V is the behavior state temporal propagation aggregation semantic representation vector.

[0054] Then, the behavior state temporal propagation aggregate semantic representation vector is input into the classifier-based behavior monitor to obtain the monitoring result, and the monitoring result is used to indicate whether the monitored elderly object has fallen. The classification process is performed by transferring the aggregate feature information through the behavior state temporal semantics of the elderly to detect whether the elderly object has a behavior state of falling, so as to perform preventive detection and early warning of falls for the elderly object, and send the early warning information to the remote terminal device to remind family members. Specifically, in response to the monitoring result that the monitored elderly object has fallen, an early warning prompt is generated and sent to the remote terminal device. In this way, the behavior pattern and state of the elderly at home can be monitored in real time, the fall early warning of the elderly can be realized, falls and accidental injuries can be prevented, and a safe and healthy home living environment can be provided for the elderly, while also reducing the care burden of family members.

[0055] In one embodiment of the present disclosure, the elderly fall warning and warning information transmission module includes: an elderly object fall detection unit, which is used to input the behavior state time series propagation aggregation semantic representation vector into a classifier-based behavior monitor to obtain a monitoring result, and the monitoring result is used to indicate whether the monitored elderly object has fallen; a fall warning prompt information generation and transmission unit, which is used to generate a warning prompt in response to the monitoring result that the monitored elderly object has fallen, and send it to a remote terminal device.

[0056] Furthermore, in one embodiment of the present disclosure, the elderly object fall detection unit includes: a behavior monitoring subunit, used to input the behavior state temporal propagation aggregation semantic representation vector into the classifier-based behavior monitor to obtain a monitoring result, and the monitoring result is used to indicate whether the monitored elderly object falls.

[0057] Furthermore, in one embodiment of the present disclosure, the behavior monitoring subunit includes: a fully connected encoding secondary subunit, used to use multiple fully connected layers of the classifier to fully connect the behavior state temporal propagation aggregate semantic representation vector to obtain an encoded classification feature vector; and a classification secondary subunit, used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the monitoring result.

[0058] Here, considering that the time series of the behavior state semantic feature vector represents a set of feature time dimensions of the semantic features of the behavior state image of the monitored elderly object at each predetermined time point, after the node message propagation fusion of the fusion node importance in the global time domain, the behavior state time series propagation aggregated semantic representation vector as a whole will cause the existence of local overflow features caused by the repeated superposition of node features, thereby affecting the accuracy of the behavior state time series propagation aggregated semantic representation vector when classified by the classifier.

[0059] Preferably, when the behavior state temporal propagation aggregate semantic representation vector is passed through a classifier-based behavior monitor to obtain a monitoring result, the behavior state temporal propagation aggregate semantic representation vector is optimized, and the optimization process includes the following steps:

[0060] Calculate the absolute value sum w1 of all eigenvalues ​​of the behavior state temporal propagation aggregate semantic representation vector and the square root w2 of the square sum of all eigenvalues ​​of the behavior state temporal propagation aggregate semantic representation vector, and divide w2 by w1 to obtain the behavior state temporal propagation aggregate semantic representation probability value p, that is:

[0061] w1=Σ i |v i |

[0062]

[0063] Among them, v i Represents the i-th eigenvalue of the behavior state temporal propagation aggregation semantic representation vector;

[0064] Each feature value in the behavior state temporal propagation aggregate semantic representation vector V is normalized to the maximum value to obtain the behavior state temporal propagation aggregate semantic representation probability vector V':

[0065]

[0066] Among them, v max Represents the maximum value in the behavior state temporal propagation aggregation semantic representation vector, v' i The i-th eigenvalue of the probability vector representing the temporal propagation aggregation semantics of the behavior state;

[0067] After the behavior state temporal propagation aggregation semantic representation probability vector is subtracted from the behavior state temporal propagation aggregation semantic representation probability value, it is further multiplied by the reciprocal of the behavior state temporal propagation aggregation semantic representation probability value to obtain the behavior state temporal propagation aggregation semantic representation domain gradient vector ,in, represents point subtraction, ⊙ represents point multiplication;

[0068] After subtracting the behavior state temporal propagation aggregation semantic representation probability vector from the unit feature vector, the behavior state temporal propagation aggregation semantic representation domain gradient vector is divided by the point to obtain the behavior state temporal propagation aggregation semantic representation differential vector, that is:

[0069]

[0070] Among them, V I represents the unit eigenvector, V1 ⊙-1 represents the bit-by-bit inverse of the gradient vector of the behavior state temporal propagation aggregation semantic representation domain, that is, the inverse of each eigenvalue of the gradient vector of the behavior state temporal propagation aggregation semantic representation domain is calculated, and V2 represents the differential vector of the behavior state temporal propagation aggregation semantic representation;

[0071] Calculate the power function V1 of the gradient vector of the behavior state temporal propagation aggregation semantic representation domain with the probability value of the behavior state temporal propagation aggregation semantic representation as an exponent ⊙p , and perform weighted summation with the exponential function exp(V2) of the behavior state temporal propagation aggregation semantic representation differential vector with the natural constant as the base to obtain the optimized behavior state temporal propagation aggregation semantic representation vector, that is:

[0072] V o =(α⊙V1⊙p )⊕[β⊙exp(V2)]

[0073] Among them, α and β represent weighted hyperparameters, ⊕ represents dot addition, V o Represents the optimized behavior state temporal propagation aggregation semantic representation vector.

[0074] That is, the mutually exclusive generalized representation behavior unit of the behavior state time series propagation aggregation semantic representation vector is constructed through the discrete differential of the probability density domain gradient corresponding to the behavior state time series propagation aggregation semantic representation vector, and the different behavior unit organization spaces under the non-uniform topological architecture of the behavior state time series propagation aggregation semantic representation vector are used to enhance the static response characteristics of the short-range associated micro-information configuration of the behavior state time series propagation aggregation semantic representation vector to the generative probabilistic generalized representation behavior, thereby ensuring the steady-state convergence of the optimization process between the generated target and the extracted features in the feature space-generated probability mapping, and improving the accuracy of the monitoring results obtained by the optimized behavior state time series propagation aggregation semantic representation vector through the classifier-based behavior monitor. In this way, it is possible to more accurately monitor and warn of falls for elderly subjects, and send warning information to remote terminal devices to remind family members, provide a safe and healthy home living environment for the elderly, and also reduce the care burden of family members.

[0075] In summary, the above scheme is adopted to collect the behavior status monitoring video of the elderly through the intelligent camera, and introduce the image processing and analysis algorithm based on artificial intelligence and machine vision technology in the back end to analyze the behavior status monitoring video, so as to capture and identify the behavior pattern and state of the elderly, and at the same time depict the propagation aggregation information and correlation characteristics of this behavior state in time series, so as to perform fall preventive detection and early warning for the elderly, and send the early warning information to the remote terminal device to remind family members. In this way, the behavior pattern and state of the elderly at home can be monitored in real time, the fall early warning of the elderly can be realized, the fall and accidental injury can be prevented, and a safe and healthy home living environment can be provided for the elderly, while also reducing the care burden of family members.

[0076] Figure 2 is a flow chart of a method for intelligent management of elderly people's home safety and health according to an exemplary embodiment. Figure 2 As shown, the method includes:

[0077] Step 201: Collecting behavioral status monitoring video of the monitored elderly subject through a smart camera;

[0078] Step 202: transmitting the behavior status monitoring video to a backend home safety and health management server via a wireless communication network;

[0079] Step 203: On the home safety and health management server, key frame sampling is performed on the behavior status monitoring video to obtain a time series of behavior status monitoring key frames;

[0080] Step 204: extracting features from each behavior state monitoring key frame in the time series of the behavior state monitoring key frames by using a behavior state feature extractor based on a deep neural network to obtain a time series of behavior state semantic feature vectors;

[0081] Step 205: input the time series of the behavior state semantic feature vector into the behavior state semantic time series propagation fusion network to obtain the behavior state time series propagation aggregate semantic representation vector as the behavior state time series propagation aggregate semantic of the elderly object;

[0082] Step 206: Based on the time series propagation aggregation semantics of the elderly object's behavior state, determine whether the monitored elderly object has fallen, and determine whether to generate an early warning prompt and send it to the remote terminal device.

[0083] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0084] like Figure 3 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0085] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0086] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0087] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0088] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0089] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0090] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0091] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0092] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, a test parameter acquisition module may also be described as a "module for acquiring device test parameters corresponding to a target device".

[0093] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0095] Figure 4 FIG. 1 is an application scenario diagram of an intelligent management system for elderly people's home safety and health according to an exemplary embodiment. Figure 4 As shown, in this application scenario, first, the behavior status monitoring video of the monitored elderly object is collected by a smart camera (for example, Figure 4 Then, the obtained behavior status monitoring video is input to a server deployed with an intelligent management algorithm for elderly home safety and health (for example, Figure 4 In S) shown in , the server is able to process the behavior status monitoring video based on the intelligent management algorithm for the safety and health of the elderly at home to determine whether the monitored elderly object falls, and determine whether to generate an early warning prompt and send it to the remote terminal device.

[0096] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0097] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0098] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. An intelligent management system for elderly people's home safety and health, characterized in that: include: The elderly behavior status video acquisition module is used to collect the behavior status monitoring video of the monitored elderly object through the smart camera; A monitoring video data transmission module, used to transmit the behavior status monitoring video to a backend home safety and health management server via a wireless communication network; A behavior state monitoring video key frame sampling module is used to perform key frame sampling on the behavior state monitoring video on the home safety and health management server to obtain a time series of behavior state monitoring key frames; The module for extracting semantic features of the behavior state of the elderly is used to extract features of each behavior state monitoring key frame in the time series of the behavior state monitoring key frames through a behavior state feature extractor based on a deep neural network to obtain a time series of behavior state semantic feature vectors; The elderly object behavior state semantic temporal propagation aggregation module is used to input the time series of the behavior state semantic feature vector into the behavior state semantic temporal propagation fusion network to obtain the behavior state temporal propagation aggregation semantic representation vector as the elderly object behavior state temporal propagation aggregation semantics; The elderly fall warning and warning information transmission module is used to determine whether the monitored elderly object has fallen based on the time-series propagation aggregation semantics of the elderly object's behavior state, and determine whether to generate a warning prompt and send it to a remote terminal device.

2. The smart management system for elderly people's home safety and health according to claim 1 is characterized in that: The behavior state feature extractor based on deep neural network is a behavior state feature extractor based on DenseNet.

3. The smart management system for elderly people's home safety and health according to claim 2 is characterized in that: The elderly object behavior state semantic temporal propagation aggregation module includes: A vector fusion unit, used for fusing the first s behavior state semantic feature vectors in the time series of the behavior state semantic feature vector to obtain a front-end node semantic information fusion feature vector; A weighted optimization unit, used for performing weighted optimization on the s-th behavior state semantic feature vector based on the importance of sequence nodes to obtain a weighted optimized behavior state semantic feature vector; A multi-layer perceptron processing unit is used to process the front-end node semantic information fusion feature vector and the weighted optimized behavior state semantic feature vector by position and then obtain a front-end node semantic time series propagation aggregation feature vector corresponding to the s-th behavior state semantic feature vector that fuses the current node information; A positional summation calculation unit is used to calculate the positional sum between the front-end node semantic time series propagation aggregation feature vectors that fuse the current node information and correspond to each behavior state semantic feature vector in the sequence of the behavior state semantic feature vectors to obtain the behavior state time series propagation aggregation semantic representation vector.

4. The smart management system for elderly people's home safety and health according to claim 3 is characterized in that: The vector fusion unit is used for: The position-based sum of the first s behavior state semantic feature vectors in the time series of the behavior state semantic feature vector is calculated to obtain the front-end node semantic information fusion feature vector.

5. The smart management system for elderly people's home safety and health according to claim 4 is characterized in that: The weighted optimization unit is used for: The sum of the trainable preset hyperparameter and one is used as a weighting coefficient to perform weighted processing on each position feature value in the s-th behavior state semantic feature vector to obtain a weighted optimized behavior state semantic feature vector.

6. The smart management system for elderly people's home safety and health according to claim 5 is characterized in that: The elderly fall warning and warning information transmission module includes: An elderly object fall detection unit, used for inputting the behavior state time series propagation aggregation semantic representation vector into a classifier-based behavior monitor to obtain a monitoring result, wherein the monitoring result is used to indicate whether the monitored elderly object falls; A fall warning prompt information generation and transmission unit is used to generate a warning prompt and send it to a remote terminal device in response to the monitoring result that the monitored elderly object has fallen.

7. The smart management system for elderly people's home safety and health according to claim 6 is characterized in that: The elderly subject fall detection unit comprises: The behavior monitoring subunit is used to input the behavior state time series propagation aggregation semantic representation vector into the classifier-based behavior monitor to obtain a monitoring result, and the monitoring result is used to indicate whether the monitored elderly object falls.

8. The smart management system for elderly people's home safety and health according to claim 7 is characterized in that: The behavior monitoring subunit comprises: A fully connected encoding secondary subunit, configured to use a plurality of fully connected layers of the classifier to perform fully connected encoding on the behavior state temporal propagation aggregation semantic representation vector to obtain an encoded classification feature vector; and The classification secondary subunit is used to pass the encoded classification feature vector through the Softmax classification function of the classifier to obtain the monitoring result.

Images