Home-based old-age care service system based on smart home

Abstract

The invention discloses a home-based old-age care service system based on smart home, and belongs to the technical field of smart home, and the system specifically comprises the steps: collecting a voice signal of an old person, converting the voice signal into a text, collecting image information of the old person, and recognizing motion features; detecting the semantic integrity of the text by using the model, and if the semantics is incomplete, marking the text as a scattered text; based on the historical medical data, the real-time physiological data and the current action characteristics of the elderly, the health state of the elderly is evaluated, and if the health state is abnormal, an alarm is triggered and a contact person is notified; if yes, keywords of the scattered texts are extracted and matched with historical instructions; if the matching fails, analyzing the directivity action, locking a pointed object, and judging whether to generate a target instruction according to whether the object belongs to the intelligent home service range and the current time scene; according to the invention, the function of deducing the real intention of the elderly from the scattered vocabularies sent by the elderly is realized, the health of the elderly is monitored, and the experience of home-based care for the elderly is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of smart home, and particularly to a home-based elderly care service system based on smart home. Background Art

[0002] At present, with the increasingly prominent trend of population aging, the number of the elderly population continues to rise, and the social elderly care pressure is increasing day by day. Against this background, home-based elderly care has become the first choice for many elderly people due to its unique advantages. And with the booming development of smart home technology, various home-based elderly care service systems based on this have emerged, aiming to improve the quality of life and safety of the elderly.

[0003] These home-based elderly care service systems skillfully integrate cutting-edge technologies such as the Internet of Things, big data, and artificial intelligence to build an intelligent and user-friendly elderly care ecosystem. For example, intelligent devices can be used to accurately monitor key physiological indicators of the elderly such as heart rate, blood pressure, and sleep quality, and once an abnormality is found, an alarm can be issued in a timely manner; intelligent security systems can be used to monitor the situation in the home in real time to effectively prevent potential safety hazards such as fires and thefts; intelligent home appliance control systems can automatically adjust the indoor temperature and humidity according to the living habits of the elderly, so that the elderly can always be in a comfortable living environment.

[0004] However, current smart home systems face many challenges in recognizing the voice commands of the elderly. Currently, the mainstream voice recognition technologies mostly establish acoustic models and language models based on the learning of a large amount of general voice data. But they have limited ability in dealing with the variability of the elderly's voices. For example, the elderly may only be able to utter fragmented words due to illness (such as Parkinson's disease, stroke, etc.), and the algorithm may be difficult to infer the true intentions of the elderly from the fragmented words, resulting in incorrect or no response from the device, thereby affecting the usability of the home service system and the user experience of the elderly. Summary of the Invention

[0005] The purpose of the present invention is to provide a home-based elderly care service system based on smart home to solve the following technical problems:

[0006] Current smart home systems cannot deeply analyze and reason about fragmented words, thus unable to accurately obtain the true needs of the elderly, greatly reducing the usability of the home service system and seriously affecting the user experience of the elderly in home-based elderly care.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A home-based elderly care service system based on smart home, comprising:

[0009] A data acquisition module, which is used to collect the voice signals sent by the elderly, preprocess the voice signals, convert the preprocessed voice signals into text, collect the images of the elderly through a camera, and identify the action features of the elderly from the images;

[0010] A natural language processing module, which is used to construct a semantic analysis model based on grammar rules and lexical semantic associations, use the semantic analysis model to detect the semantics of the text, and if the semantics are incomplete, mark it as fragmented text;

[0011] A health anomaly detection module, which is used to obtain the historical medical data and real-time physiological data of the elderly, judge the health status of the elderly according to the historical medical data, real-time physiological data and current action features, and if the health status is abnormal, trigger a health anomaly alarm and send it to the contacts of the elderly;

[0012] A historical instruction matching module, which is used to, when the health status of the elderly is normal, use a preset instruction keyword library to extract keywords from the fragmented text, match the keywords with the historical instruction data at the current time and scene, and if the match is successful, directly output the corresponding instruction;

[0013] An action feature matching module, which is used to, when the instruction matching fails, judge whether the action feature of the elderly is a pointing action, and judge whether the object corresponding to the pointing action belongs to the service scope of the smart home. If so, judge whether to generate a target instruction for the object according to the current time and scene; if not, generate an anomaly prompt and send it to the contacts of the elderly.

[0014] As a further solution of the present invention: in the data acquisition module, the process of preprocessing the voice signal is as follows:

[0015] The collected voice signal is denoised by spectral subtraction, the denoised voice signal is framed according to a fixed frame length and frame shift, and based on a double-threshold endpoint detection method of short-time energy and zero-crossing rate, by setting a high threshold, the rough starting point of the framed voice signal is detected, and by setting a low threshold, the precise starting point of the framed voice signal is detected, and the silent segments and non-speech segments are removed based on the rough starting point and the precise starting point.

[0016] As a further solution of the present invention: in the data acquisition module, the process of identifying the action features of the elderly from the images is as follows:

[0017] Obtain a continuous image sequence through a camera, use an object detection algorithm to detect the position of the elderly from the images, utilize a lightweight human pose estimation model to extract the bone key point coordinates of the elderly in the images, generate an action feature vector containing joint position information, where the action feature vector includes joint angles and movement trajectories, input the selected action feature vector into a trained action classification model, and output the current action category of the elderly, where the action category includes standing and falling.

[0018] As a further solution of the present invention: In the natural language processing module, the process of using a semantic analysis model to detect the semantics of text is as follows:

[0019] Input the text into a pre-trained semantic analysis model, use a part-of-speech tagging tool and syntactic analysis technology to extract a feature vector containing semantic and syntactic information, compare the feature vector of the semantic and syntactic information with a pre-constructed complete semantic template library to generate a semantic coherence score, and detect logical breakpoints, where the logical breakpoints include missing subjects, mismatched predicate objects, and ambiguous pronoun references. Mark the text with a semantic coherence score lower than a preset score threshold and the number of logical breakpoints exceeding a preset number threshold as scattered text.

[0020] As a further solution of the present invention: In the historical instruction matching module, the process of extracting keywords from scattered text is as follows:

[0021] Train a bidirectional transducer model based on a preset instruction keyword library, use a word segmentation technology to perform word segmentation on the text, and use a part-of-speech tagging tool to tag the part of speech for each word segment. Identify noun, verb, and device name keywords and mark them as candidate words. Count the frequency of each candidate word in the text, screen out high-frequency words, calculate the semantic weight of each high-frequency word in the context association, and screen out keywords from the high-frequency words according to a preset weight threshold.

[0022] As a further solution of the present invention: The process of matching the keywords with historical instruction data in the current time and scenario is as follows:

[0023] Collect the historical instruction data of the elderly and construct a historical instruction database. Extract each historical instruction and annotate accurate time information and scenario information for each historical instruction. Input the extracted keywords into the historical instruction database, screen out the historical instructions containing the keywords from the historical instruction database, and obtain the historical instructions that match the current time and scenario from the historical instructions containing the keywords. If a matching historical instruction is obtained, the matching is successful and the corresponding instruction is directly output. Otherwise, the subsequent process is triggered.

[0024] As a further solution of the present invention: In the action feature matching module, the process of determining whether the action feature of the elderly is a directed action is as follows:

[0025] Based on the images of the elderly collected by the camera, the positions of the hand key points are extracted by using a hand gesture estimation model. The hand key points include fingertips, knuckles and the palm center. Analyze the spatial distribution of the hand key points and detect the relative positions of the wrist and elbow, and use a head pose estimation model to analyze the orientation of the line of sight. Record the durations of the hand pointing and the line of sight orientation in the target direction respectively, and determine the action features with durations exceeding the preset threshold as directive actions.

[0026] As a further solution of the present invention: the process of generating the target instruction by the action feature matching module is as follows:

[0027] When it is determined as a directive action, use a line of sight tracking algorithm to generate a line of sight focus area, calculate the direction vector of the extended line of the finger, construct an indoor environment point cloud map through indoor space modeling technology, label the names of each object in the room, find the intersection position of the line of sight focus area and the extended line of the finger, and obtain the name of the indoor object corresponding to the intersection position from the point cloud map. Based on whether the object supports network control and is compatible with mainstream protocols, judge whether it belongs to the service scope of smart home. If it does, judge whether to generate the target instruction for the object according to the current time, scene and historical behavior data.

[0028] Advantages of the present invention:

[0029] The present invention obtains the action features of the elderly from the images of the elderly collected by the camera, uses speech recognition technology to convert the speech signal of the elderly into text, combines a semantic analysis model to judge the integrity of the semantics. If the semantics are broken, combined with the historical medical data, real-time physiological data and current action features of the elderly, first screen whether there is a risk of speech disorder caused by sudden diseases, so as to detect the health of the elderly. After excluding the factors of sudden diseases, further extract the keywords in the incomplete semantics and match them with the historical instructions in the current time and scene. If no matching item is found, by analyzing the directive action and locking the target object pointed to, and then combining with the historical instructions to generate a target instruction that conforms to the time and space context, thus effectively solving the problem of incomplete semantics caused by the elderly being ill, realizing accurate demand response while ensuring the health monitoring of the elderly, significantly improving the aging-friendly service ability of the smart home system, and improving the experience of home care for the elderly. Brief Description of the Drawings

[0030] The following further describes the present invention with reference to the accompanying drawings.

[0031] Figure 1 It is a schematic diagram of the module of the present invention. Detailed Embodiments

[0032] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0033] Please refer to Figure 1 As shown, the present invention is a home-based elderly care service system based on smart home, including:

[0034] A data acquisition module, mainly used to obtain the voice signals of the elderly and collect the motion images of the elderly. First, a high-sensitivity and low-noise microphone is selected. This microphone can capture the voice signals emitted by the elderly as clearly as possible in different environments. For example, in the room where the elderly live, there may be various background noises, such as the sound of the air conditioner and the wind outside the window. The high-sensitivity microphone can effectively reduce the interference of these noises on the voice signals.

[0035] After the voice signals are collected, preprocessing work is immediately carried out. Advanced noise reduction algorithms (such as spectral subtraction, Wiener filtering, etc.) are used. Spectral subtraction estimates the noise spectrum and subtracts the noise spectrum from the spectrum of the noisy voice signal to obtain a clean voice signal. Wiener filtering filters the noisy voice signal according to the minimum mean square error criterion to make the output signal as close as possible to the clean voice signal. Secondly, endpoint detection is performed. Commonly used endpoint detection methods include the double-threshold method based on energy and zero-crossing rate. By setting appropriate energy thresholds and zero-crossing rate thresholds, the voice signals can be effectively separated from the background noise, avoiding the introduction of unnecessary noise data in subsequent processing. Then, feature extraction is performed on the preprocessed voice signals. Usually, Mel Frequency Cepstral Coefficients (MFCC) are used. It simulates the characteristics of the human auditory system and can well reflect the spectral characteristics of the voice signals. By calculating MFCC, the voice signals are converted into a series of feature vectors, laying a foundation for subsequent speech recognition. Finally, speech recognition technology is used to convert the preprocessed voice signals into text. Currently, the mainstream speech recognition technology is an end-to-end model based on deep learning, such as a model based on the Transformer architecture.

[0036] In terms of image acquisition, cameras will be reasonably arranged in the living environment of the elderly to ensure that the actions of the elderly can be comprehensively and clearly captured. For example, wide-angle cameras are installed in the main activity areas such as the living room and bedroom to ensure full coverage without blind spots. After the images are collected, preprocessing such as image enhancement and denoising is performed on the images. This can improve the contrast and brightness of the images, make the details in the images clearer, and remove the noise generated by the camera itself or environmental factors, thereby improving the quality of the images. Then, human pose estimation technology is used to identify the action features of the elderly from the images. Human pose estimation is an important research direction in the field of computer vision, and there are currently many mature algorithms, such as OpenPose, HRNet, etc. These algorithms can accurately detect the key points of the human body, such as the head, shoulders, elbows, wrists, etc., and construct the human pose based on the positions and connection relationships of these key points. By analyzing consecutive multiple frames of images, the specific actions of the elderly, such as walking, sitting down, standing up, raising the hand, etc., can be further identified.

[0037] The natural language processing module mainly constructs a semantic analysis model based on grammar rules and lexical semantic associations. In terms of grammar rules, a complete grammar system including morphological rules and syntactic rules will be constructed. Morphological rules are used to analyze the part of speech and morphological changes of words, while syntactic rules are used to analyze the structure and components of sentences. For example, through syntactic rules, it can be determined whether a sentence is in the subject-predicate-object structure, the subject-linking verb-predicative structure, or other types of structures. In terms of lexical semantic associations, a semantic network will be constructed. A semantic network is a graph structure representing the semantic relationships between words, where nodes represent words and edges represent the semantic relationships between words, such as synonymy, antonymy, hyponymy, etc. Through the semantic network, the semantic connotations of words and the semantic connections between words can be understood more deeply.

[0038] Deep learning algorithms are used to train the constructed grammar rules and semantic network to obtain a semantic analysis model. Commonly used deep learning algorithms include recurrent neural networks (RNN) and their variants, such as long short-term memory networks (LSTM), gated recurrent units (GRU), etc. The trained semantic analysis model is used to perform semantic detection on the text. The process of semantic detection is to input the text into the semantic analysis model, and the model analyzes and judges the semantics of the text based on its own structure and parameters, including aspects such as the integrity, logic, and rationality of the sentence.

[0039] If the semantics of the text are complete, conforming to grammar rules and semantic logic, then the corresponding instructions can be directly output. For example, if the elderly person says "I want to turn on the TV", this sentence has complete semantics and expresses a clear intention. If the semantics of the text are incomplete, there are grammar errors, logical confusion, or ambiguous expressions, etc., then it is marked as fragmented text. For example, the elderly person may only say "TV...", this sentence has incomplete semantics and it is not clear what the elderly person specifically wants to do with the TV. At this time, it needs to be marked as fragmented text for further analysis and processing later.

[0040] Health anomaly detection module. The health anomaly detection module needs to obtain the historical medical data and real-time physiological data of the elderly. The historical medical data can be obtained from the hospital's electronic medical record system, including information such as the elderly person's past medical history, disease diagnosis, treatment records, etc. The real-time physiological data is collected through various wearable devices or sensors. For example, a smart bracelet can monitor the elderly person's heart rate, blood pressure, sleep quality and other data in real time, and a smart mattress can monitor the elderly person's breathing rate, body movement conditions, etc.

[0041] Integrate the obtained historical medical data, real-time physiological data and current action characteristics to construct a comprehensive health data set. Comprehensively judge the health status of the elderly based on the integrated health data set. For example, if the elderly person's heart rate suddenly exceeds the normal range and at the same time symptoms such as difficulty breathing and slow movement appear, then it can be judged that the elderly person may have a health anomaly. When it is judged that the health status of the elderly is abnormal, the system will immediately trigger a health anomaly alarm and send the health anomaly information of the elderly to the elderly person's contacts by means of text message alarm and APP push alarm.

[0042] Historical instruction matching module. When the health status of the elderly is normal, a preset instruction keyword library is set up. The instruction keyword library is constructed based on the elderly person's previous instruction records and common life needs. During the construction process, a large amount of instruction data will be analyzed and sorted out to extract the keywords. For example, for instructions related to household appliance control, the keywords may include "turn on", "turn off", "adjust", "TV", "air conditioner", "light", etc.; for instructions related to life services, the keywords may include "order food", "buy medicine", "make an appointment with a doctor", etc.

[0043] The instruction keyword library will be continuously updated and improved. As the elderly person uses the system for a longer time, new instructions will be continuously recorded and analyzed, and the keyword library will be adjusted and expanded accordingly to improve the accuracy and comprehensiveness of instruction matching.

[0044] Extract keywords from scattered text using a preset instruction keyword library, and match the extracted keywords with historical instruction data in the current time and scenario. The matching process is to screen out all instructions containing the keywords in the historical instruction data, and obtain the historical instructions that meet the current time and scenario from the historical instructions containing the keywords. If the matching is successful, the corresponding instruction is directly output.

[0045] For example, if the extracted keyword is "TV", by matching with the historical instruction data, it is found that there is a historical instruction "Turn on the TV" and both the time and scenario match. Then, the instruction "Turn on the TV" can be directly output to let the smart home system perform the corresponding operation.

[0046] Action feature matching module. When the instruction matching fails, the action feature matching module uses human pose estimation and object detection technologies to determine whether the action feature of the elderly is a pointing action. A pointing action refers to an action where the elderly clearly points to an object or direction through body movements, such as raising a finger to point at an item, turning the head to look at a certain place, etc. Through human pose estimation, the body pose information of the elderly can be accurately obtained, such as the position and angle of the hand; through object detection, the objects in the image and their positions can be identified. Combining the two, analyze the spatial relationship between the elderly's body movements and the objects. If the elderly's action clearly points to an object and lasts for a certain period of time, then it can be determined that the action is a pointing action.

[0047] Then, determine whether the object corresponding to the pointing action belongs to the service scope of the smart home. The service scope of the smart home includes various devices and items that can be controlled and managed through the smart system, such as smart home appliances, smart door locks, smart curtains, etc. For example, if the elderly's pointing action points to a lamp and it is confirmed that the smart TV belongs to the service scope of the smart home, then determine whether to generate a target instruction for this object according to the current time and scenario. For example, before going to bed at night, if the elderly points to the bedroom light, combining the current time and scenario, a target instruction "Turn off the bedroom light" can be generated.

[0048] The judgment basis includes time factors, scenario factors, and the daily habits of the elderly, etc. Time factors can be divided into different time periods, such as daytime, evening, morning, night, etc., and different time periods may have different requirements. Scenario factors include the environment where the elderly are located, such as the living room, bedroom, kitchen, etc., and different scenarios may have different service requirements. The daily habits of the elderly are analyzed and summarized based on the elderly's past behavior data.

[0049] If the object corresponding to the directive action does not fall within the service scope of the smart home, or although it belongs to the service scope but a reasonable target instruction cannot be generated according to the current time and scenario, then the system will generate an exception prompt and send it to the contacts of the elderly. The exception prompt will describe the action situation of the elderly and the current judgment result, so that the contacts can further understand the situation and take corresponding measures.

[0050] The above has described in detail an embodiment of the present invention, but the content described is only a preferred embodiment of the present invention and cannot be considered as limiting the implementation scope of the present invention. All equivalent changes and improvements made according to the scope of the present invention application should still fall within the patent coverage scope of the present invention.

Claims

1. A home-based elderly care service system based on smart home, characterized in that, Including: A data acquisition module, which is used to collect the voice signals sent by the elderly, preprocess the voice signals, convert the preprocessed voice signals into text, collect the images of the elderly through a camera, and identify the action features of the elderly from the images; A natural language processing module, which is used to construct a semantic analysis model based on grammar rules and lexical semantic associations, use the semantic analysis model to detect the semantics of the text, and if the semantics are incomplete, mark it as fragmented text; A health anomaly detection module, which is used to obtain the historical medical data and real-time physiological data of the elderly, judge the health status of the elderly according to the historical medical data, real-time physiological data and current action features, and if the health status is abnormal, trigger a health anomaly alarm and send it to the contacts of the elderly; A historical instruction matching module, which is used to, when the health status of the elderly is normal, use a preset instruction keyword library to extract keywords from the fragmented text, and match the keywords with the historical instruction data at the current time and scene. If the match is successful, directly output the corresponding instruction; An action feature matching module, which is used to, when the instruction matching fails, judge whether the action feature of the elderly is a directive action, and judge whether the object corresponding to the directive action belongs to the service scope of the smart home. If so, judge whether to generate a target instruction for the object according to the current time and scene; if not, generate an anomaly prompt and send it to the contacts of the elderly.

2. The home-based elderly care service system based on smart home according to claim 1, characterized in that, In the data acquisition module, the process of preprocessing the voice signal is as follows: The collected voice signal is denoised by spectral subtraction, the denoised voice signal is framed according to a fixed frame length and frame shift, and based on the double-threshold endpoint detection method of short-time energy and zero-crossing rate, by setting a high threshold, the rough starting point of the framed voice signal is detected, and by setting a low threshold, the precise starting point of the framed voice signal is detected. The silent segments and non-speech segments are removed based on the rough starting point and the precise starting point.

3. The home-based elderly care service system based on smart home according to claim 1, characterized in that, In the data acquisition module, the process of identifying the action features of the elderly from the images is as follows: A continuous image sequence is obtained through a camera, and the position of the elderly is detected from the images using a target detection algorithm. A lightweight human pose estimation model is used to extract the bone key point coordinates of the elderly in the images, and an action feature vector containing joint position information is generated. The action feature vector includes joint angles and movement trajectories. The selected action feature vector is input into a trained action classification model, and the current action category of the elderly is output. The action category includes standing and falling.

4. The home-based elderly care service system based on smart home according to claim 1, characterized in that, In the natural language processing module, the process of using the semantic analysis model to detect the semantics of the text is as follows: The text is input into a pre-trained semantic analysis model, and a feature vector containing semantic and syntactic information is extracted using a part-of-speech tagging tool and syntactic analysis technology. The feature vector of the semantic and syntactic information is compared with a pre-constructed complete semantic template library for similarity, a semantic coherence score is generated, and logical breakpoints are detected. The logical breakpoints include missing subjects, mismatched predicate objects, and ambiguous pronoun references. The text with a semantic coherence score lower than the preset score threshold and the number of logical breakpoints exceeding the preset number threshold is marked as fragmented text.

5. The home-based elderly care service system based on smart home according to claim 1, characterized in that, In the historical instruction matching module, the process of extracting keywords from scattered text is as follows: Based on a preset instruction keyword library, train a bidirectional transducer model, perform word segmentation on the text using word segmentation technology, and use a part-of-speech tagging tool to tag the part of speech of each word segment. Identify noun, verb, and device name keywords and mark them as candidate words. Count the frequency of each candidate word in the text, screen out high-frequency words, calculate the semantic weight of each high-frequency word in the context association, and screen out keywords from the high-frequency words according to a preset weight threshold.

6. The home-based elderly care service system based on smart home according to claim 1, characterized in that In the historical instruction matching module, the process of matching the keywords with historical instruction data at the current time and in the current scenario is as follows: Collect the historical instruction data of the elderly and build a historical instruction database. Extract each historical instruction and label accurate time information and scenario information for each historical instruction. Input the extracted keywords into the historical instruction database, screen out the historical instructions containing the keywords from the historical instruction database, and obtain the historical instructions that match the current time and scenario from the historical instructions containing the keywords. If a matching historical instruction is obtained, the matching is successful and the corresponding instruction is directly output; otherwise, the subsequent process is triggered.

7. The home-based elderly care service system based on smart home according to claim 1, characterized in that In the action feature matching module, the process of determining whether the action feature of the elderly is a directed action is as follows: Based on the images of the elderly collected by the camera, use a hand pose estimation model to extract the positions of the key points of the hand. The key points of the hand include fingertips, finger joints, and the palm center. Analyze the spatial distribution of the key points of the hand and detect the relative positions of the wrist and elbow, and use a head pose estimation model to analyze the direction of the line of sight. Record the duration of the hand pointing and the line of sight direction in the target direction respectively. Determine the action feature with a duration exceeding the preset threshold as a directed action.

8. A home-based elderly care service system based on smart home according to claim 1, characterized in that, In the action feature matching module, the process of generating a target instruction is as follows: When it is determined to be a directed action, use a line-of-sight tracking algorithm to generate a line-of-sight focus area, calculate the direction vector of the extended line of the finger, build an indoor environment point cloud map through indoor space modeling technology, label the names of each object in the room, find the intersection position of the line-of-sight focus area and the extended line of the finger, and obtain the name of the indoor object corresponding to the intersection position from the point cloud map. Based on whether the object supports network control and is compatible with mainstream protocols, determine whether it belongs to the service scope of smart home. If it does, determine whether to generate a target instruction for the object according to the current time, scenario, and historical behavior data.

Images