Home safety monitoring and early warning system for old people

Abstract

The invention is suitable for the technical field of safety monitoring and early warning, and provides an old people home safety monitoring and early warning system, which comprises an old people physiological information acquisition end, an old people image information acquisition end, an old people position information acquisition end, a server end, a cloud end and an early warning end, the server side is connected with the elderly physiological information acquisition side, the elderly image information acquisition side, the elderly position information acquisition side, the cloud side and the early warning side. The server side is used for sending the received first physiological information, the second physiological information, the third physiological information, the first image information, the second image information and the position information to the cloud side, so that the cloud side transmits an analysis result to the server side after analyzing and processing, and meanwhile, the server side generates old people safety monitoring information based on the analysis result. And the elderly safety monitoring information is sent to the early warning end, so that the early warning end performs safety risk feedback response. According to the invention, the comprehensiveness and accuracy of home safety monitoring of old people are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring and early warning, and in particular to a home safety monitoring and early warning system for the elderly. Background Art

[0002] At present, the home safety monitoring and early warning technology for the elderly has made some progress. On the one hand, sensor technology has been widely used, such as accelerometers and gyroscopes in wearable devices and pressure sensors in smart mattresses, which can collect the activities and sleep status of the elderly, and environmental sensors can monitor the safety of the home environment. At the same time, the Internet of Things and wireless communication technologies have achieved fast and stable data transmission, and family members can access remotely through mobile phone APP or web pages.

[0003] In the prior art, the home safety monitoring and early warning for the elderly focuses on unilateral monitoring. The Chinese invention patent CN117496663A discloses a home safety management system and method for the elderly, which uses monitoring equipment to monitor the body contour of the elderly in real time to determine whether they are in a falling state; the monitoring equipment detects that the elderly are in a falling state and starts timing information to send to the server; the server responds to different levels of alarms according to the duration of the timing. It can be seen that the patent only performs safety monitoring from the dimension of the body contour of the elderly, and does not consider other factors for monitoring. The Chinese invention patent CN118982900A discloses a home care safety monitoring method for the elderly, which includes: relying on the built Internet of Things TOT platform, collecting sensor data; generating multi-dimensional, multi-period, and multi-comparison trend data reports for user historical data according to real-time, weekly, monthly, seasonal, and annual periodic time periods; extracting multi-dimensional effective change characteristics of users from the long-term data change trend based on historical user real data, performing feature engineering and security risk index intelligent analysis algorithms, and making health warnings based on the analysis results. The multi-dimensional monitoring in this patent does not mention the monitoring of the physiological and emotional changes of the elderly, which makes the monitoring data incomplete and leads to the problem of inaccurate monitoring.

[0004] In view of this, a home safety monitoring and early warning system for the elderly is needed. Summary of the invention

[0005] The embodiment of the present application provides a home safety monitoring and early warning system for the elderly, which is used to solve the problem of inaccurate safety monitoring.

[0006] The first aspect of the embodiment of the present application provides a home safety monitoring and early warning system for the elderly, comprising:

[0007] Elderly physiological information collection terminal, elderly image information collection terminal, elderly location information collection terminal, service terminal, cloud terminal and early warning terminal;

[0008] The service end is respectively connected to the elderly physiological information collection end, the elderly image information collection end, the elderly location information collection end, the cloud and the early warning end;

[0009] The elderly physiological information collection terminal is used to transmit the collected first physiological information, second physiological information and third physiological information to the service terminal;

[0010] The elderly image information acquisition terminal is used to transmit the identified first image information and second image information to the service terminal;

[0011] The elderly location information collection terminal is used to transmit the collected location information to the service terminal;

[0012] The server is used to send the received first physiological information, second physiological information, third physiological information, first image information, second image information and location information to the cloud, so that the cloud analyzes and processes the first physiological information, second physiological information, third physiological information, first image information, second image information and location information and transmits the analysis result to the server. At the same time, the server generates elderly safety monitoring information based on the analysis result, and sends the elderly safety monitoring information to the early warning end, so that the early warning end responds to safety risk feedback.

[0013] Furthermore, the physiological information collection terminal for the elderly includes a wearable device, a non-invasive physiological monitoring device and an environmental sensor; the wearable device is used to collect first physiological information, which includes basic physiological indicator data of the elderly; the non-invasive physiological monitoring device is used to collect second physiological information, which includes body metabolism data of the elderly; the environmental sensor is used to collect third physiological information, which includes physiological change indicator data of the elderly.

[0014] Furthermore, the elderly image information acquisition terminal includes a surveillance camera, which is used to collect first image information and second image information, and the first image information includes the elderly's body behavior image data and the elderly's face image data; the elderly location information acquisition terminal includes a surveillance camera, which is used to collect third image information, and the third image information includes indoor panoramic image data.

[0015] Furthermore, the cloud performs analysis and processing based on the first physiological information, the second physiological information, the third physiological information, the first image information, the second image information and the location information and transmits the analysis result to the server, including:

[0016] The cloud includes a physiological data analysis module, a limb image analysis module, a facial image analysis module and a location analysis module; the physiological data analysis module is used to analyze the physiological health status of the elderly based on the first physiological information, the second physiological information and the third physiological information; the limb image analysis module is used to analyze the limb movements and postures of the elderly; the facial image analysis module is used to identify the emotional state of the elderly; and the location analysis module is used to track the location information of the elderly indoors.

[0017] Furthermore, the physiological data analysis module is used to analyze the physiological health status of the elderly according to the first physiological information, the second physiological information and the third physiological information, including:

[0018] Comparing the various indicator data in the first physiological information with a preset personal physiological indicator baseline, and determining the time change curve of the various physiological indicators of the elderly within a preset time based on the comparison result, and determining the first health condition characteristic according to the time change curve;

[0019] Calculating the basal metabolic rate and nighttime energy expenditure rate of the elderly based on the metabolic index data in the second physiological information, and determining a second health condition characteristic according to the basal metabolic rate and nighttime energy expenditure rate;

[0020] Determine environmental information data according to the third physiological information, calculate a curve of physiological indicators of the elderly changing with the environment based on the environmental information data, and determine a third health condition characteristic according to the environmental change curve;

[0021] Based on the first health condition characteristic, the second health condition characteristic and the third health condition characteristic, the trained comprehensive health assessment model is input to obtain the target health condition result of the elderly.

[0022] Furthermore, the limb image analysis module is used to analyze the limb movements and postures of the elderly, including:

[0023] Performing image preprocessing on the first image information, and extracting key limb node positions and contour information based on the preprocessed image;

[0024] Based on the extracted limb key node positions and contour information, posture recognition and classification as well as motion tracking analysis are performed to determine the movements and postures of the elderly's limbs.

[0025] Furthermore, the facial image analysis module is used to identify the emotional state of the elderly, including:

[0026] Performing face detection and positioning on the second image information using a preset Haar-Cascade feature template to determine a face region image, and performing image normalization processing on the face region image;

[0027] Extracting facial geometric features and texture features based on the processed face image;

[0028] The geometric features and texture features of the face are input into a trained convolutional neural network model to identify the emotional state of the elderly.

[0029] Furthermore, the step of inputting the geometric features and texture features of the face into a trained convolutional neural network model to identify the emotional state of the elderly includes:

[0030] The trained convolutional neural network model includes a filter group that increases the sensitivity of wrinkle features:

[0031]

[0032] Where: y ij is the value of the output feature map at position (i, j) after convolution, x i+m,j+n is the pixel value of the input image at the corresponding position, w mn is the weight of the convolution kernel at position (m,n), and b is the bias term.

[0033] Furthermore, the location analysis module is used to track the location information of the elderly indoors, including:

[0034] Using a target monitoring algorithm in deep learning to detect the position and range of the elderly in the image from the third image information, and extracting the movement state and direction of the elderly;

[0035] According to the transformation relationship obtained by the preset camera calibration and map registration, the position coordinates of the elderly in the third image information are converted into actual coordinates in the indoor map;

[0036] The activity trajectory of the elderly is determined based on tracking their location information at different times.

[0037] Furthermore, the server generates elderly safety monitoring information based on the analysis result, and sends the elderly safety monitoring information to the early warning end, so that the early warning end performs a safety risk feedback response, including:

[0038] The server integrates and correlates the analysis results based on a set timeline, and determines whether the elderly are at potential risk by setting different levels of health and safety thresholds;

[0039] When the risk reaches the warning level based on the multi-dimensional factor analysis, the generated warning information is sent to the warning end for feedback response.

[0040] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0041] The present invention sets a physiological information acquisition terminal for the elderly, an image information acquisition terminal for the elderly, a location information acquisition terminal for the elderly, a service terminal, a cloud end, and an early warning terminal. After the service terminal sends the received first physiological information, second physiological information, third physiological information, first image information, second image information, and location information to the cloud end, the cloud end performs analysis and processing based on the first physiological information, second physiological information, third physiological information, first image information, second image information, and location information to obtain several dimensional factors of the elderly's physiological health status, body dynamics and posture, emotional state, and activity trajectory. The service end comprehensively analyzes the safety monitoring results of the elderly based on multi-dimensional factors, and finally provides feedback based on the early warning terminal. By performing safety monitoring of the elderly at home in multiple dimensions, the accuracy of safety monitoring is effectively improved, providing an accurate and comprehensive data basis for the health management of the elderly.

[0042] Other advantages, objectives, and features of the present invention will be set forth in part in the following description, and in part will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a structural block diagram of a home safety monitoring and early warning system for the elderly in the present invention;

[0044] Figure 2 A schematic diagram of an embodiment of a home safety monitoring and early warning system for the elderly in the present invention;

[0045] Figure 3 It is a flow chart of another embodiment of a home safety monitoring and early warning system for the elderly in the present invention;

[0046] Figure 4 It is a flow chart of another embodiment of a home safety monitoring and early warning system for the elderly in the present invention;

[0047] Figure 5 The present invention is a flowchart of another embodiment of a home safety monitoring and early warning system for the elderly. DETAILED DESCRIPTION

[0048] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] Embodiment 1

[0050] The following will introduce the elderly home safety monitoring and early warning system in this application from the perspective of data information interaction between various terminals. Figure 1 , a home safety monitoring and early warning system for the elderly includes the following:

[0051] The elderly physiological information collection terminal 1, the elderly image information collection terminal 2, the elderly position information collection terminal 3, the server 4, the cloud 5 and the early warning terminal 6, the server 4 is respectively connected to the elderly physiological information collection terminal 1, the elderly image information collection terminal 2, the elderly position information collection terminal 3, the cloud 5 and the early warning terminal 6, the elderly physiological information collection terminal 1 is used to transmit the collected first physiological information, the second physiological information and the third physiological information to the server 4; the elderly image information collection terminal 2 is used to transmit the identified first image information and the second image information to the server 4; the elderly position information collection terminal 3 is used to transmit the collected position information to the server 4; the server 4 is used to send the received first physiological information, the second physiological information, the third physiological information, the first image information, the second image information and the position information to the cloud 5, so that the cloud 5 analyzes and processes the first physiological information, the second physiological information, the third physiological information, the first image information, the second image information and the position information, and then transmits the analysis result to the server 4, and at the same time, the server 4 generates the elderly safety monitoring information based on the analysis result, and sends the elderly safety monitoring information to the early warning terminal 6, so that the early warning terminal 6 performs a safety risk feedback response.

[0052] Furthermore, the elderly physiological information collection terminal 1 includes a wearable device, a non-invasive physiological monitoring device and an environmental sensor; the wearable device is used to collect the first physiological information, and the first physiological information includes the basic physiological index data of the elderly; the non-invasive physiological monitoring device is used to collect the second physiological information, and the second physiological information includes the body metabolism data of the elderly; the environmental sensor is used to collect the third physiological information, and the third physiological information includes the physiological change index data of the elderly.

[0053] Furthermore, the elderly image information collection terminal 2 includes a surveillance camera, which is used to collect first image information and second image information, the first image information includes the elderly's body behavior image data and the elderly's face image data; the elderly location information collection terminal includes a surveillance camera, which is used to collect third image information, and the third image information includes indoor panoramic image data.

[0054] Furthermore, the cloud 5 includes a physiological data analysis module, a limb image analysis module, a facial image analysis module and a location analysis module; the physiological data analysis module is used to analyze the physiological health status of the elderly according to the first physiological information, the second physiological information and the third physiological information; the limb image analysis module is used to analyze the limb movements and postures of the elderly; the facial image analysis module is used to identify the emotional state of the elderly; and the location analysis module is used to track the location information of the elderly indoors.

[0055] The above-mentioned embodiments effectively improve the comprehensiveness and accuracy of home safety monitoring of the elderly by conducting comprehensive safety monitoring of the elderly from the dimensions of their physiological health status, body movements and postures, emotional states, and activity trajectories.

[0056] Embodiment 2

[0057] See also Figure 2 The physiological data analysis module in the present invention is used to analyze the physiological health status of the elderly according to the first physiological information, the second physiological information and the third physiological information, and further includes the following:

[0058] S21. Compare the various indicator data in the first physiological information with the preset personal physiological indicator baseline, and determine the time change curve of the various physiological indicators of the elderly within the preset time based on the comparison result, and determine the first health condition characteristics according to the time change curve;

[0059] Specifically, the first physiological information includes heart rate, blood pressure, blood oxygen saturation and body temperature. Heart rate reflects the basic condition of heart function and cardiovascular system. The normal range is generally 60-100 beats / minute, but it will fluctuate due to individual differences and physical activity status. Blood pressure includes systolic pressure and diastolic pressure, which is an important indicator for measuring the pressure of cardiovascular system. Normal systolic pressure is generally 90-139mmHg, and diastolic pressure is between 60-89mmHg. Blood oxygen saturation indicates the degree of oxygen binding to hemoglobin in the blood, and the normal range is usually 95%-100%. Body temperature is one of the indicators reflecting the overall metabolism and immune status of the body. The normal body temperature range is 36℃-37.2℃. The body temperature regulation function of the elderly is relatively weak, and the body temperature is slightly lower than that of young people.

[0060] Collect basic physiological indicator data of the elderly within a preset time, such as a week, calculate the average value, standard deviation and other statistical parameters of each indicator, determine the baseline range of personal physiological indicators, and build a personal physiological indicator baseline. Monitor the physiological indicator data uploaded by the wearable device in real time and compare it with the personal baseline range. When an indicator exceeds the baseline range by a certain amplitude, such as the heart rate change exceeds 10% of the baseline, it is marked as an abnormal data point, and the time, duration and severity of the abnormality are recorded. Draw the change curve of each physiological indicator over time, and determine the health status characteristics of the elderly based on physiological data based on the time change curve.

[0061] S22. Calculate the basal metabolic rate and nighttime energy expenditure rate of the elderly based on the metabolic index data in the second physiological information, and determine the second health status characteristic according to the basal metabolic rate and the nighttime energy expenditure rate;

[0062] Specifically, the second physiological information includes metabolic index data, calculation of basal metabolic rate and nighttime energy expenditure rate, wherein the basal metabolic rate is estimated by monitoring parameters such as heart rate, breathing, skin temperature during sleep, combined with body fat rate, muscle mass, etc. The nighttime energy expenditure rate is obtained by recording the energy consumption of the elderly during sleep, including energy consumption to maintain basic physiological functions of the body and energy consumption increased by possible nighttime activities. By comparing the calculated basal metabolic rate and nighttime energy expenditure rate with the set threshold, the physiological health status characteristics of the elderly in terms of metabolism are determined based on the comparison results.

[0063] S23. Determine environmental information data according to the third physiological information, calculate the physiological index of the elderly with the environment change curve based on the environmental information data, and determine the third health condition characteristics according to the environment change curve;

[0064] Specifically, the third physiological information includes indoor temperature and humidity, air quality and light intensity. Indoor temperature and humidity have a significant impact on the physiological state of the elderly. Too high or too low temperature may cause changes in the elderly's body; indoor air quality indicators include carbon dioxide concentration, particulate matter concentration, formaldehyde content, etc.; changes in indoor light intensity and light time will affect the sleep quality of the elderly.

[0065] Establish a correlation model between environmental factors, such as temperature and humidity, air quality, light intensity, etc. and basic physiological indicators of the elderly, such as heart rate, blood pressure, respiratory rate, body temperature, etc. By real-time monitoring of changes in environmental data and physiological indicator data, analyze the influence of environmental factors on physiological status, and generate characteristics of environmental impact on health status based on the analysis of physiological changes in the elderly under different environmental conditions.

[0066] S24. Based on the first health condition characteristic, the second health condition characteristic and the third health condition characteristic, the trained comprehensive health assessment model is input to obtain the target health condition result of the elderly.

[0067] The generated first health condition characteristics, second health condition characteristics and third health condition characteristics are input into the trained comprehensive health assessment model, and the model outputs the assessment results of the current health status of the elderly and the prediction of health trends in the future.

[0068] The above-mentioned embodiments can make full use of physiological data from different devices to achieve comprehensive and accurate monitoring and evaluation of the health status of the elderly, and provide an effective data basis for health management and monitoring of the elderly.

[0069] Embodiment 3

[0070] See also Figure 3 The limb image analysis module in the present invention is used to analyze the limb movements and postures of the elderly and also includes the following:

[0071] S31. Preprocessing the first image information, and extracting key limb node positions and contour information based on the preprocessed image;

[0072] Specifically, the limb image analysis module extracts limb image information of the elderly based on the image information captured by the surveillance camera. Image preprocessing includes image graying, image filtering and image enhancement. Image graying converts color images into grayscale images to reduce the amount of data while retaining the basic contour and shape information of the image to improve efficiency for subsequent processing. Image filtering removes noise to make the image clearer and facilitate feature extraction. Image enhancement improves image features such as contrast and brightness, making limb details more obvious, which is helpful for subsequent posture estimation and motion analysis.

[0073] S32. Perform posture recognition and classification as well as motion tracking analysis based on the extracted positions and contour information of key limb nodes to determine the movements and postures of the elderly's limbs.

[0074] Use Gaussian filter to smooth the image, reduce the influence of noise, calculate the gradient amplitude and direction of the image, and through non-maximum suppression, only retain the pixel points with the largest amplitude in the gradient direction as edge candidates. Through dual threshold detection, the final edge pixel points are determined according to the high threshold and low threshold. For example, set the high threshold to 100 and the low threshold to 50. Pixels with gradient amplitude greater than the high threshold are directly determined as edge points, and pixels with gradient amplitude less than the low threshold are excluded. Pixels between the two are also determined as edge points if they are adjacent to the determined edge points.

[0075] Secondly, determine the key joints of the human body, such as shoulders, elbows, wrists, hips, knees and ankles. Using the trained deep neural network model, input the preprocessed image, and the model automatically extracts features from the image through structures such as convolutional layers and pooling layers.

[0076] According to the extracted limb joint position and contour information, the posture of the elderly is compared with the predefined normal and abnormal posture models to determine the category to which the current posture belongs. The position changes of limb joints in continuous image frames are tracked, and the speed, amplitude, trajectory and other characteristics of the movement are analyzed to determine whether there are abnormal movements, such as sudden falls, limb convulsions, etc.

[0077] The above-mentioned embodiment monitors the elderly from the dimension of their body movements. Body movements can intuitively reflect the health status of the elderly, providing an intuitive data basis for subsequent comprehensive testing.

[0078] Embodiment 4

[0079] See also Figure 4 The facial image analysis module of the present invention is used to identify the emotional state of the elderly and includes:

[0080] S41. Perform face detection and positioning on the second image information using a preset Haar-Cascade feature template to determine the face region image, and perform image normalization on the face region image;

[0081] Use a series of predefined Haar-Cascade feature templates and cascade classifiers to slide windows at different positions and scales of the image, calculate the degree of match between the image area in the window and the Haar-Cascade feature template, and screen through the cascade classifier to determine the face area. Perform operations such as rotation, scaling and translation on the detected face to place key feature points such as eyes, nose and mouth in standard positions; use histogram equalization or gamma correction to adjust the image's light intensity.

[0082] S42. Extracting facial geometric features and texture features based on the processed face image;

[0083] Geometric feature extraction: On the normalized face image, the active shape model ASM is used to locate key facial feature points, such as the corners of the eyes, the corners of the mouth, the tip of the nose, etc. Various geometric feature parameters are calculated based on the located feature points. For example, the degree of eye opening and closing is calculated by the distance between the upper and lower eyelid feature points, the width and height ratio of the mouth is calculated by feature points such as the corners of the mouth and the midpoint of the lips, and the tilt angle of the eyebrows is calculated by the feature points of the eyebrows and the beginning of the eyebrows. At the same time, for the wrinkle features unique to the elderly, a morphological image processing method is used. For some static wrinkles, such as fixed wrinkles caused by natural aging, a database is established for comparison and exclusion to distinguish dynamic wrinkle changes caused by emotions.

[0084] Texture feature extraction: For each pixel in the face image, a small neighborhood is taken with it as the center, and the pixel value in the neighborhood is compared with the center pixel value to generate a binary code. If the neighborhood pixel value is greater than the center pixel value, the position is 1, otherwise it is 0. These binary codes are converted into decimal numbers as the LBP value of the pixel. By statistically analyzing the distribution of LBP values ​​of the entire face image, a texture descriptor is obtained to represent the texture features of the face. At the same time, based on the skin texture characteristics of the elderly, an adaptive LBP algorithm is used to automatically adjust the neighborhood size and threshold according to the grayscale changes in the skin area to more accurately extract texture features to distinguish between skin texture changes caused by emotions and texture changes caused by natural aging. Local analysis is performed in the spatial domain and frequency domain using Gabor filters to extract texture features of different directions and scales. Multiple Gabor filters of different directions and scales are convolved with the image to obtain multiple filtered images, which contain texture information of different directions and scales. Then, feature extraction is performed on these filtered images, and the mean and variance are calculated to construct texture feature vectors.

[0085] S43. The emotional state of the elderly is identified by inputting the facial geometric features and texture features into the trained convolutional neural network model.

[0086] 1. Convolutional neural network model construction:

[0087] Set up a convolutional neural network (CNN) architecture, which has good feature extraction capabilities and contains multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer is used to extract local features of the image, and the convolution operation is performed by sliding the convolution kernel on the image.

[0088] The above network architecture is adjusted for emotion analysis of the elderly: In the convolution layer, special filters are designed according to the shape and texture characteristics of wrinkles. For horizontal wrinkles, narrow-band filters are designed in the horizontal direction, and the convolution kernel size can be 1×3 or 1×5 to better capture the line characteristics of wrinkles. These filters are convolved with the image through convolution operations, and the calculation formula is:

[0089]

[0090] Where: y ij is the value of the output feature map at position (i, j) after convolution, x i+m,j+n is the pixel value of the input image at the corresponding position, w mn is the weight of the convolution kernel at position (m,n), and b is the bias term.

[0091] A spatial attention mechanism is added before the fully connected layer to calculate the importance weight of each feature point, so that the network pays more attention to facial areas that are closely related to emotions, such as eyes, mouth, and wrinkle areas. The specific calculation method is to generate an attention weight map through a small convolutional neural network. Assume that the input feature map is F∈R C×H×W , after two convolutional layers, the convolution kernel size is 1×1 and a sigmoid activation function, the attention weight map A∈R is obtained 1×H×W , the calculation formula is: A = σ(f2(δ(f1(F)))), where f1 and f2 are two convolution operations, σ is the ReLU activation function, and δ is the sigmoid activation function. Then the attention weight map is multiplied by the original feature map to obtain the weighted feature map F att =A×F, used for subsequent fully connected layer processing.

[0092] 2. Model training:

[0093] Collect a dataset of elderly face images containing various emotional states, such as happiness, sadness, anger, etc., including different lighting conditions, facial angles, and expression intensity. Label each image with emotional categories, and record special features related to the emotions of the elderly, such as the distribution of wrinkles, shape changes, and other information. Process the data according to the image preprocessing steps mentioned above to make the image data meet the model input requirements. Here, the cross entropy loss function is selected for training in emotional state recognition. Assuming that the output of the model is and the true label is, the calculation formula of the cross entropy loss function is:

[0094]

[0095] Where: N is the number of categories, y i is the probability that the model predicts the i-th type of emotion, y true,iis the probability of the i-th category of emotion in the true label, which is 0 or 1 here, indicating whether it belongs to this category. This loss function is used to measure the difference between the probability distribution predicted by the model and the true label. The model is optimized by minimizing this loss during training. Select stochastic gradient descent. In each iteration, the weight of the model is updated according to the gradient of the loss function, and finally a trained convolutional neural network model is obtained. Finally, the geometric features and texture features are input into the trained convolutional neural network model to identify the emotional state of the elderly.

[0096] The above-mentioned embodiment can indirectly obtain the emotional changes of the elderly by using intelligent algorithms to identify the emotional state of the elderly, thereby providing an effective data basis for subsequent comprehensive monitoring.

[0097] Embodiment 5

[0098] See also Figure 5 The location analysis module in the present invention is used to track the location information of the elderly indoors and includes:

[0099] S51. Detect the position and range of the elderly in the image from the third image information using the target monitoring algorithm in deep learning, and extract the movement state and direction of the elderly;

[0100] First, determine the camera's internal parameters such as focal length, principal point coordinates, and external parameters such as the camera's position and posture. Use the checkerboard calibration tool to shoot checkerboard images at different angles and use computer vision algorithms to calculate the camera's parameters. Secondly, align the coordinate system of the indoor map with the camera's coordinate system by selecting some known feature points on the indoor map, finding the corresponding points in the camera image, and then calculating the transformation relationship between the two.

[0101] The target detection algorithm in deep learning is used to process the images taken by the camera to identify the elderly in the image. After the elderly are detected, their relevant feature information is further extracted, such as body posture, head position, facial orientation, etc. This is used to determine the state and movement direction of the elderly, providing basic data for subsequent position tracking.

[0102] It should be noted that the camera here and the camera for obtaining the first image information and the second image information can be set to the same camera, or can be set to multiple cameras. The number of cameras is not limited here. When it is limited to multiple cameras, the first image information, the second image information, and the third image information are obtained based on different cameras. When it is limited to one camera, the camera has a wide-angle shooting function and can capture the image of the elderly in real time. The first image information, the second image information, and the third image information are obtained based on different dimensions of the camera.

[0103] S52. According to the transformation relationship obtained by the preset camera calibration and map registration, the position coordinates of the elderly in the third image information are converted into actual coordinates in the indoor map;

[0104] According to the transformation relationship obtained by camera calibration and map registration, the position coordinates of the elderly in the image are converted to the actual coordinates in the indoor map. Assuming the image coordinates are (x, y), the corresponding map coordinates (X, Y) can be obtained through the transformation matrix T. The calculation formula is:

[0105] S53. Determine the activity trajectory of the elderly based on tracking their location information at different times.

[0106] As time goes by, new camera images are continuously acquired, and the above process of person detection, feature extraction, and position mapping is repeated to obtain a series of position coordinates of the elderly at different times, thereby tracking their position indoors. The position trajectory of the elderly is analyzed to extract the characteristic parameters of the trajectory, such as moving speed, moving direction, and dwell time.

[0107] In the above embodiment, the location analysis module can assist in monitoring the safety status of the elderly from various dimensions by tracking the activity trajectory of the elderly indoors.

[0108] Embodiment 6

[0109] At the same time, the server generates elderly safety monitoring information based on the analysis results, and sends the elderly safety monitoring information to the early warning end, so that the early warning end can respond to safety risk feedback, including the following:

[0110] 1. The server integrates and correlates the analysis results based on the set timeline, and determines whether the elderly are at potential risk by setting different levels of health and safety thresholds;

[0111] First, each set of received data is given an accurate timestamp, and the accuracy of the timestamp is set according to actual needs. Various types of data with the same timestamp are integrated to build a structured data record containing fields such as health status, body movements, emotional state, and activity trajectory.

[0112] Health status analysis: Based on medical standards and the individual health records of the elderly, different levels of threshold ranges are set for various physiological indicators, and the health status level of the elderly is judged according to the set different levels. For example, when one of the main indicators such as heart rate, blood pressure, and blood oxygen saturation is in the mild abnormal range, and there are no obvious abnormalities in other indicators, the health status is judged to be mild risk; if multiple indicators are in the moderate abnormal range, it is judged to be moderate risk; when there are severely abnormal indicators or multiple indicators seriously deviate from the normal range, it is judged to be severe risk.

[0113] Assessment of limb movement and emotional state: The assessment of abnormal limb movement is based on the preset abnormal movement pattern library. Common abnormal movements include falling, long-term immobility, twitching, abnormal limb tremors, etc. By comparing the current limb movement features with the data in the abnormal movement pattern library, if the match is successful, it is determined that there is abnormal limb movement and the corresponding danger level is marked. For example, a falling action is determined to be a high danger level, while a slight limb tremor may be marked as a low danger level. Abnormal emotional state is based on the emotional state identified by the face image analysis module, combined with the results of psychological research and the daily emotional patterns of the elderly, to determine whether the emotion is abnormal, and divide the danger level according to the severity and duration of the emotion. Based on the association between limb movement and emotional state, the combination of the two is further analyzed. For example, when it is detected that the elderly have tense limb movements and are accompanied by anxious emotional expressions, it may indicate high psychological pressure or potential dangerous situations, and the comprehensive danger level will be increased. By establishing an emotion-limb movement association matrix, corresponding danger weights are assigned according to different combinations of emotions and limb movements, so as to more accurately assess the overall state of the elderly.

[0114] 2. When the risk reaches the warning level based on multi-dimensional factor analysis, the generated warning information will be sent to the warning end for feedback response.

[0115] The server integrates the risk assessment results of health status, body movements and emotional state to calculate the overall risk level. Here, weighted summation is used to assign a weight to each factor, and the weight is determined based on its importance and impact on the safety of the elderly at home. For example, the weight of health status is set to 0.5, the weight of abnormal body movements is set to 0.3, and the weight of abnormal emotional state is set to 0.2. Different levels of warning thresholds are set according to actual needs and the requirements for the safety of the elderly. For example, when the risk level reaches 3, a first-level warning is triggered, and a general attention reminder is sent to the warning end; when the risk level reaches 4, a second-level warning is triggered, and an emergency notification is sent; when the risk level reaches 5, the highest level of the third-level warning is triggered, and a reminder is given to contact the medical emergency agency at the same time to ensure that the elderly can receive timely treatment.

[0116] In addition, the activity trajectory information is correlated with the health status and the risk assessment results. For example, if it is found that the elderly frequently walk back and forth in the room and the activity trajectory is disordered when they are in poor health, or if they have abnormal body movements and emotional tension after entering the dangerous area, the server can infer that the activity trajectory may be a factor leading to the dangerous situation. According to the correlation analysis results of the activity trajectory and other factors, the corresponding feedback information is generated and provided to the early warning end and relevant personnel. The feedback information includes a description of the current activity status of the elderly and possible risk warnings. These feedback information can help relevant personnel understand the elderly's situation more comprehensively, take more targeted measures to intervene and care, and effectively ensure the safety of the elderly at home.

[0117] The above embodiment is described below in conjunction with experimental data. Please refer to Table 1 for the monitoring data of the elderly in one day:

[0118]

[0119]

[0120]

[0121] As can be seen from the above table, from morning to afternoon, heart rate and blood pressure show a gradual upward trend, and after 14:00, they exceed the normal range and continue to deteriorate. By analyzing the health data trend of this day, it can be judged that the elderly may have problems with their cardiovascular system, which requires timely attention and further examination. As time goes by, limb movements gradually become abnormal from normal, such as difficulty getting up, unsteady steps, and falls. Combined with changes in health status, these abnormal limb movements may be caused by physical discomfort, and may further increase health risks. The emotional state gradually changes from calm to anxiety, pain, irritability and panic, which echoes the abnormal changes in health status and limb movements. Emotional instability may affect the psychological and physical health of the elderly, and also reflects the impact of physical discomfort on their mental state. The activity trajectory is mainly concentrated in areas such as bedrooms, living rooms, bathrooms and dining rooms, which is consistent with the daily activity range of the elderly at home. However, at around 18:00, the elderly fell in the living room. This location information combined with other factors indicates that there may be safety hazards in the living room environment, or that the elderly’s physical condition at the time could no longer support normal activities. Taking all the above factors into consideration, during the period of 14:00-15:00, when the health status, body movements and emotional state are obviously abnormal, the server will issue a first-level warning according to the preset warning mechanism, notifying family members or caregivers to pay attention to the elderly, and recommend measuring blood pressure, asking about physical feelings, etc. When the elderly falls at 18:00, the highest level of third-level warning is immediately triggered, providing the family with the specific location and current status of the elderly so that rescue measures can be taken quickly.

[0122] The server end of the above-mentioned embodiment can conduct a comprehensive and in-depth integrated analysis of the home monitoring data of the elderly, timely and accurately discover potential safety risks, and send effective warning information to the warning end, providing strong support and protection for the health and safety of the elderly.

[0123] It is understandable that those skilled in the art can, under the guidance of the above embodiments, combine various implementation methods in the above embodiments to obtain technical solutions of multiple implementation methods.

[0124] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A home safety monitoring and early warning system for the elderly, characterized in that: include: Elderly physiological information collection terminal, elderly image information collection terminal, elderly location information collection terminal, service terminal, cloud terminal and early warning terminal; The service end is respectively connected to the elderly physiological information collection end, the elderly image information collection end, the elderly location information collection end, the cloud and the early warning end; The elderly physiological information collection terminal is used to transmit the collected first physiological information, second physiological information and third physiological information to the service terminal; The elderly image information acquisition terminal is used to transmit the identified first image information and second image information to the service terminal; The elderly location information collection terminal is used to transmit the collected location information to the service terminal; The server is used to send the received first physiological information, second physiological information, third physiological information, first image information, second image information and location information to the cloud, so that the cloud analyzes and processes the first physiological information, second physiological information, third physiological information, first image information, second image information and location information and transmits the analysis result to the server. At the same time, the server generates elderly safety monitoring information based on the analysis result, and sends the elderly safety monitoring information to the early warning end, so that the early warning end responds to safety risk feedback.

2. The elderly home safety monitoring and early warning system according to claim 1 is characterized in that: The elderly physiological information collection terminal includes a wearable device, a non-invasive physiological monitoring device and an environmental sensor; the wearable device is used to collect first physiological information, which includes basic physiological indicator data of the elderly; the non-invasive physiological monitoring device is used to collect second physiological information, which includes body metabolism data of the elderly; the environmental sensor is used to collect third physiological information, which includes physiological change indicator data of the elderly.

3. The elderly home safety monitoring and early warning system according to claim 1 is characterized in that: The elderly image information acquisition terminal includes a surveillance camera, which is used to acquire first image information and second image information, wherein the first image information includes image data of the elderly's limb behavior and image data of the elderly's face; the elderly location information acquisition terminal includes a surveillance camera, which is used to acquire third image information, wherein the third image information includes indoor panoramic image data.

4. The elderly home safety monitoring and early warning system according to any one of claims 1-3, characterized in that: The method enables the cloud end to analyze and process the first physiological information, the second physiological information, the third physiological information, the first image information, the second image information and the location information and transmit the analysis result to the server end, including: The cloud includes a physiological data analysis module, a limb image analysis module, a facial image analysis module and a location analysis module; the physiological data analysis module is used to analyze the physiological health status of the elderly based on the first physiological information, the second physiological information and the third physiological information; the limb image analysis module is used to analyze the limb movements and postures of the elderly; the facial image analysis module is used to identify the emotional state of the elderly; and the location analysis module is used to track the location information of the elderly indoors.

5. The elderly home safety monitoring and early warning system according to claim 4 is characterized in that: The physiological data analysis module is used to analyze the physiological health status of the elderly according to the first physiological information, the second physiological information and the third physiological information, including: Comparing the various indicator data in the first physiological information with a preset personal physiological indicator baseline, and determining the time change curve of the various physiological indicators of the elderly within a preset time based on the comparison result, and determining the first health condition characteristic according to the time change curve; Calculating the basal metabolic rate and nighttime energy expenditure rate of the elderly based on the metabolic index data in the second physiological information, and determining a second health condition characteristic according to the basal metabolic rate and nighttime energy expenditure rate; Determine environmental information data according to the third physiological information, calculate a curve of physiological indicators of the elderly changing with the environment based on the environmental information data, and determine a third health condition characteristic according to the environmental change curve; Based on the first health condition characteristic, the second health condition characteristic and the third health condition characteristic, the trained comprehensive health assessment model is input to obtain the target health condition result of the elderly.

6. The elderly home safety monitoring and early warning system according to claim 4 is characterized in that: The limb image analysis module is used to analyze the limb movements and postures of the elderly, including: Performing image preprocessing on the first image information, and extracting key limb node positions and contour information based on the preprocessed image; Based on the extracted limb key node positions and contour information, posture recognition and classification as well as motion tracking analysis are performed to determine the movements and postures of the elderly's limbs.

7. The elderly home safety monitoring and early warning system according to claim 4 is characterized in that: The facial image analysis module is used to identify the emotional state of the elderly, including: Using a preset Haar-Cascade feature template to perform face detection and positioning on the second image information to determine a face area image, and performing image normalization processing on the face area image; Extracting facial geometric features and texture features based on the processed face image; The geometric features and texture features of the face are input into a trained convolutional neural network model to identify the emotional state of the elderly.

8. The elderly home safety monitoring and early warning system according to claim 7 is characterized in that: The step of inputting the geometric features and texture features of the face into a trained convolutional neural network model to identify the emotional state of the elderly includes: The trained convolutional neural network model includes a filter group that increases the sensitivity of wrinkle features: Where: y ij is the value of the output feature map at position (i, j) after convolution, x i+m,j+n is the pixel value of the input image at the corresponding position, w mn is the weight of the convolution kernel at position (m,n), and b is the bias term.

9. The elderly home safety monitoring and early warning system according to claim 4 is characterized in that: The location analysis module is used to track the location information of the elderly indoors, including: Using a target monitoring algorithm in deep learning to detect the position and range of the elderly in the image from the third image information, and extracting the movement state and direction of the elderly; According to the transformation relationship obtained by the preset camera calibration and map registration, the position coordinates of the elderly in the third image information are converted into actual coordinates in the indoor map; The activity trajectory of the elderly is determined based on tracking their location information at different times.

10. The elderly home safety monitoring and early warning system according to claim 1 is characterized in that: At the same time, the server generates elderly safety monitoring information based on the analysis result, and sends the elderly safety monitoring information to the early warning end, so that the early warning end performs a safety risk feedback response, including: The server integrates and correlates the analysis results based on a set timeline, and determines whether the elderly are at potential risk by setting different levels of health and safety thresholds; When the risk reaches the warning level based on the multi-dimensional factor analysis, the generated warning information is sent to the warning end for feedback response.

Images