Sensor-based state detection method and system for elderly living at home

Abstract

The invention discloses a sensor-based state detection method and system for the elderly living at home, and relates to the technical field of state monitoring. The method comprises the following steps: deploying a plurality of sensors in a home environment, identifying a first activity track of a home elder based on a DTW algorithm, judging whether there is an abnormal behavior, identifying a first action and a first posture of the home elder based on an OpenPose algorithm, establishing a rule engine, mapping a first space and a preset second space to a triangular mesh, and determining whether there is an abnormal behavior or not. Analyzing whether points, sections and surfaces included in the mapped triangular mesh have intersection points, generating a first probability or a second probability by combining spatial information in the mesh, and outputting an early warning evaluation value through combination of a first curve, a second curve, a first state quantity and a second state quantity; the method can improve the spatial resolution of capturing the actions of the elderly, accurately recognize the action postures, avoid judgment errors caused by different behavioral habits of the elderly living at home, and improve the overall performance of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of state monitoring, and in particular to a sensor-based state detection method and system for elderly people at home. Background Art

[0002] At present, sensor-based technologies have been widely used in health monitoring of the elderly. For example, smart bracelets, smart insoles, smart watches and other devices have been widely used in health monitoring of the elderly at home. They can monitor the elderly's movement, posture, heart rate, body temperature and other health data in real time, detect abnormalities in time and send warnings; For example, in smart homes, environmental sensors such as temperature, humidity, air quality, and light are widely used. With the development of society, the current state detection methods and systems for the elderly at home based on sensors have developed from traditional single sensor monitoring to multi-modal sensor fusion and intelligent analysis. The data collected by sensors can be analyzed in real time to determine the activities, health status and occurrence of emergencies of the elderly. However, the behavioral trajectories and physical conditions of elderly people at home vary from person to person. For example, one elderly person at home may perform well, but another elderly person at home may have a large deviation, making it impossible to accurately determine whether there is abnormal behavior in the behavioral trajectory. For example, in determining whether the movement posture of an elderly person at home is abnormal, the collected movement posture is usually compared with the built-in movement template of the system, and the degree of similarity is calculated to determine the abnormal behavior of the elderly (such as falling). In the case of complex or atypical movements, relying solely on the degree of similarity may lead to misjudgment or recognition failure. For example, when an elderly person is stretching, the amplitude and speed of the movement are similar to certain abnormal behaviors (such as convulsions), which may be misjudged as abnormal behavior by the system. This not only causes a lot of waste of time and computing resources, but also leads to reduced recognition accuracy and delayed response time. Summary of the invention

[0003] 1. Technical issues to be resolved In view of the deficiencies in the prior art, the present invention provides a method and system for detecting the status of elderly people at home based on sensors, which solves the problems raised in the background technology.

[0004] (II) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: In a first aspect, the present application provides a method for detecting the state of an elderly person at home based on a sensor, the method comprising: Acquire first sensor data, second sensor data and third sensor data of the elderly at home; Based on the DTW algorithm, the first sensor data, the second sensor data and the third sensor data are processed and analyzed to identify the first activity trajectory of the elderly at home, and the time distance feature, the sensor distance feature and the sensor attribute feature are extracted based on the first activity trajectory, and a first rule engine is built to determine whether the behavior type of the first activity trajectory is normal behavior or abnormal behavior; Based on the OpenPose algorithm, the second sensor data is processed and analyzed to identify the first action and the first posture of the elderly person at home, and the action features and posture features are extracted based on the first action and the first posture. A second rule engine is built to map the first space and the preset second space to a triangular mesh, and analyze whether there are intersections among the points, segments, and surfaces contained in the mapped triangular mesh. Combined with the spatial information in the mesh, a first probability that the first activity trajectory belongs to abnormal behavior or a second probability that it is normal behavior is generated.

[0005] Further, after the first activity trajectory is identified, the first sensor data is imported into a pre-built curve analysis model, a first curve is established, and a first angle and a first duration are output to generate a first state quantity; Under the condition that it is determined that the first activity trajectory has abnormal behavior, the abnormal type is identified based on the abnormal behavior, and a second curve is established in combination with the first probability of the abnormal behavior to identify the peak value and the second duration to generate a second state quantity; The first state quantity and the second state quantity are imported into a pre-built early warning evaluation model to generate an early warning evaluation value, and a plurality of early warning signals are triggered based on the early warning evaluation value.

[0006] Furthermore, the first sensor data includes but is not limited to heart rate, blood sugar, blood pressure and respiratory rate; the second sensor data includes but is not limited to video; and the third sensor data includes but is not limited to temperature and humidity, and door switch.

[0007] Furthermore, the steps of building the first rule engine include: Based on the KL algorithm, the time distance feature, the sensor distance feature and the sensor attribute feature are matched and analyzed to determine whether the deviation degree of the first activity trajectory is within a preset range, and the preset range is dynamically set based on the genetic algorithm; When the match is successful, that is, within the preset range, it is determined to be normal behavior; Otherwise, it is determined to be an abnormal behavior; when the matching fails, the time distance feature, the sensor distance feature and the sensor attribute feature are input into the preset trajectory evaluation model to obtain the deviation trajectory of the first activity trajectory and the corresponding deviation degree.

[0008] Furthermore, the steps of building the second rule engine include: Divide the home environment into a triangular mesh and divide it into a first partition and a second partition; wherein the triangular mesh is in a three-dimensional form to form an enclosing mesh; Determine the static layer and the dynamic layer based on the second sensor data, combine the action features and the posture features, generate the first space, and discretize the first space into points, segments or surfaces respectively; wherein the line type of the segment includes a straight line segment and a circular curve; Map the first space and the preset second space onto the triangular mesh, analyze whether there are intersections on the to-be-intersected points, to-be-intersected segments, and to-be-intersected surfaces, and if there are intersections, generate the first sub-probability by combining the spatial information in the triangular mesh; otherwise, generate the second sub-probability by conversion analysis; Combine the triangular meshes associated with the to-be-intersected points, to-be-intersected segments, and to-be-intersected surfaces, extract the corresponding sub-probabilities under the same triangular mesh, screen and analyze the sub-probabilities based on the preset thresholds of the first partition and the second partition, retain the sub-probabilities greater than the preset thresholds, and if the first sub-probability and the second sub-probability exist in the same triangular mesh, eliminate the smaller sub-probability and compare and analyze it again with the preset threshold; process the mean of the sub-probabilities under the same intersection points, and combine the sub-probabilities under different intersection points to obtain the corresponding first probability or second probability.

[0009] Furthermore, if there is an intersection, the first sub-probability is obtained by combining the spatial information in the grid, including: When points are included: extract the first position point and its point coordinates, take the first position point as the first center and draw a circle with r as the radius to form a first sphere, when the second position point exists in the circumscribed cube of the first sphere, take the distance value as the radius, calculate the intersection and union ratio of the second sphere formed by the second position point and the circumscribed cube of the first sphere, and set the formula to obtain the first sub-probability; The formula is: ; In the formula, μ1 represents the first sub-probability corresponding to the point, exp(·) represents the exponential function, dis represents the distance value, and V sub represents the intersection-and-union ratio, λ represents the attenuation index, and σ1 and σ2 are weight correction coefficients; When a straight line segment is included: extract the coordinates of the first line segment and its endpoints, calculate the intersection of the second line segment and the straight line where the first line segment is located by using the first parametric equation, calculate the ratio of the length of the intersection on the first line segment to the length of the second line segment by combining the Gaussian distribution model, and perform weighted summation to obtain the integral to obtain the first sub-probability; When a circular curve is included: extract the second center of the circle, find the intersection of the curves where the second center of the circle and the third center of the circle are located by using the second parameter equation, determine the first deflection angle based on the line connecting the second center of the circle and the intersection, determine the second deflection angle based on the line connecting the second center of the circle and the start and end points of the circular curve of the second center of the circle, determine the third deflection angle based on the line connecting the start and end points of the circular curve of the second center of the circle and the third center of the circle, determine the intersection interval based on the second deflection angle and the third deflection angle, and when the first deflection angle is a subset of the intersection interval, obtain the first sub-probability in combination with the curvature of the circular curve; When a face is included: extract the first plane, calculate the segment sandwiched by the first plane and the second plane, calculate the area enclosed by each endpoint of the segment and each vertex of the first plane, and combine the second angle between the center of gravity of the first plane and the segment to obtain the first sub-probability.

[0010] Further, based on the abnormal behavior, the abnormal type is identified, and in combination with the first probability of the abnormal behavior, a peak value and a second duration are identified to generate a second state quantity, including: A measurement period is set based on a time series. In the same measurement period, the first probabilities are aggregated into a first set, the abnormal types are aggregated into a second set, and the cumulative number of abnormal behaviors and the first percentage set of each abnormal type in the measurement period are obtained; the influence relationship between each element in the first set and the second set is analyzed, and each element in the first percentage set is weighted to generate a degree value; In combination with the first probability, a second curve of each first probability-degree value is drawn, a peak is identified based on the second curve, and a second state quantity is generated in combination with a second duration of each peak and a corresponding peak quantity.

[0011] Further, the first state quantity and the second state quantity are imported into a pre-built early warning evaluation model, including: Based on the time series, the preset time interval is extracted, and the third state vector is constructed based on the first state quantity and the second state quantity, and is aggregated into R. The calculation formula based on the early warning evaluation model is: ; Where warn represents the warning assessment value, T represents the time interval, η represents the weight correction coefficient, P1(T) represents the first curve obtained in the time interval T, P2(T) represents the second curve obtained in the time interval T, and c represents the constant bias term.

[0012] In a second aspect, the present application provides a sensor-based status detection system for elderly people at home, the system comprising: A sensor acquisition module, for acquiring first sensor data, second sensor data and third sensor data of the elderly at home; The state detection module processes and analyzes the first sensor data, the second sensor data and the third sensor data based on the DTW algorithm, identifies the first activity trajectory of the elderly at home, extracts the time distance feature, the sensor distance feature and the sensor attribute feature based on the first activity trajectory, and builds a first rule engine to determine whether the behavior type of the first activity trajectory is normal behavior or abnormal behavior; based on the OpenPose algorithm, the second sensor data is processed and analyzed to identify the first action and the first posture of the elderly at home, and the action feature and the posture feature are extracted based on the first action and the first posture, and a second rule engine is built to map the first space and the preset second space to the triangular mesh, analyze whether there are intersections among the points, segments and faces contained in the mapped triangular mesh, and generate a first probability that the first activity trajectory belongs to abnormal behavior or a second probability that the first activity trajectory belongs to normal behavior in combination with the spatial information in the mesh; the state detection module is built with a first determination unit and a second determination unit; The first determination unit imports the first sensing data into a pre-built curve analysis model, establishes a first curve, and outputs a first angle and a first duration to generate a first state quantity; the second determination unit, under the condition that it is determined that the first activity trajectory has abnormal behavior, identifies the abnormal type based on the abnormal behavior, establishes a second curve in combination with the first probability of the abnormal behavior, identifies the peak value and the second duration to generate a second state quantity; The early warning feedback module, under the condition of determining that abnormal behavior exists, imports the first state quantity and the second state quantity into a pre-built early warning evaluation model, generates an early warning evaluation value, and triggers a plurality of early warning signals based on the early warning evaluation value.

[0013] (III) Beneficial effects The present invention provides a sensor-based state detection method and system for elderly people at home, which has the following beneficial effects: 1. The present invention uses multiple sensors and a DTW algorithm to identify the first activity trajectory point, and extracts the time distance feature, sensor distance feature, and sensor attribute feature to find the deviation trajectory and degree of deviation of the first activity trajectory point from the daily behavior pattern, so as to judge the abnormal behavior of the elderly at home; 2. The present invention identifies the movements and postures of the elderly at home based on the OpenPose algorithm, constructs a first space, combines the second space, maps the first space and the second space to a triangular mesh, determines the points to be intersected, the segments to be intersected, and the surfaces to be intersected in the triangular mesh, so as to calculate the first probability of abnormal behavior, and indirectly determines the degree of overlap between the first space and the second space, thereby avoiding misjudgment caused by improper setting of the similarity threshold, and improving the spatial resolution of capturing the elderly's movements, thereby more accurately determining whether there is an intersection in the movement, so as to improve the recognition accuracy, and quickly respond to the movements of the elderly at home, thereby reducing the computational complexity and improving the overall performance of the system; 3. The present invention identifies the first angle and the first duration through the first curve to obtain the first state quantity; identifies the abnormal type through the abnormal behavior, and establishes the second curve in combination with the first probability of the abnormal behavior to identify the peak value and the second duration to obtain the second state quantity; generates an early warning evaluation value through the first state quantity and the second state quantity in combination with the first curve and the second curve, so that the family members of the elderly at home can have an intuitive understanding of the elderly's condition. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of a method for detecting the state of an elderly person at home according to an exemplary embodiment; Figure 2 The present invention is a module diagram of a sensor-based status detection system for elderly people at home according to an exemplary embodiment. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Example 1

[0017] The embodiment of the present invention provides a method for detecting the state of elderly people at home based on sensors; Figure 1 is a flow chart of a method for detecting the state of an elderly person at home according to an exemplary embodiment; Figure 1 , the method comprises the following steps: S1. Deploy a number of sensors in a home to collect and obtain first sensor data, second sensor data, and third sensor data of the elderly at home; The first sensor data includes but is not limited to heart rate, blood sugar, blood pressure and respiratory rate; the second sensor data includes but is not limited to video; the third sensor data includes but is not limited to temperature and humidity data, door switch; In this embodiment, several sensors are deployed, such as motion sensors, door magnetic sensors, light sensors, infrared motion sensors, temperature and humidity sensors, and other related wearable sensors, to collect human activity data. The above sensors are not shown in the figure and are adaptively installed or worn at corresponding positions. For example, in daily life, when an elderly person at home arrives at a certain position, the corresponding motion sensor can be triggered, and the sensor state is ON. When the elderly person does not move, the state of the sensor is switched to OFF after the number of seconds defined by the rule (for example, 1.25 seconds). A sensor event is generated at a certain moment, including date, time, sensor identifier, and sensor message value. The sequence of these sensor events can be mapped to different daily behaviors to prepare for the follow-up. S2. Based on the DTW algorithm, the first sensor data, the second sensor data, and the third sensor data are processed and analyzed to identify the first activity trajectory of the elderly at home, and the time distance feature, the sensor distance feature, and the sensor attribute feature are extracted based on the first activity trajectory, and a first rule engine is built to determine whether the behavior type of the first activity trajectory is normal behavior or abnormal behavior; In this embodiment, a first model is built by the DTW algorithm to analyze the first activity trajectory of the elderly at home. The first model is trained to identify several different activity behaviors. After the first sensor data, the second sensor data and the third sensor data are input, the first model calculates the dynamic time warping distance between the first sensor, the second sensor and the third sensor to obtain the similarity measure at each moment, and generates a vector containing several similarity values ​​by merging several moments, and the similarity value corresponds to an activity behavior category; by comparing several similarity values, the one with the largest similarity is selected as the classification result output; in addition, the first model includes using a hybrid CNN+LSTM deep learning model, setting batch_size=64, epochs=1000, loss function loss='biary_crossentropy', optimizer optimizer='adam', using the Keras deep learning library, and taking 65% of the data as a training set, and 35% of the data as a test set for training, so as to quickly and accurately judge the activity behavior of the elderly at home, and evolve into the first activity trajectory of the elderly at home based on the time series; The steps of building the first rule engine include: Based on the KL algorithm, the time distance feature, the sensor distance feature and the sensor attribute feature are matched and analyzed to determine whether the deviation degree of the first activity trajectory is within a preset range, and the preset range is automatically set based on the genetic algorithm; When the match is successful: If it is within the preset range, it is considered normal behavior; Otherwise, it is determined to be an abnormal behavior; when the matching fails, the time distance feature, the sensor distance feature and the sensor attribute feature are input into the preset trajectory evaluation model to obtain the deviation trajectory of the first activity trajectory and the corresponding deviation degree; In this embodiment, a plurality of sensors are distributed in various areas of the house, and a corresponding sensor sequence is obtained through the first sensor, the second sensor, and the third sensor, and its representation form is: Data={D1, D2, ..., Dn}; wherein n is the number of days for collecting a plurality of sensor data; Di={T Di , S Di , M Di}, i=1, 2...N; T Di is the time series of data collected by Di Tianxia, ​​S Di Di represents the sensor sequence under the world, M is the sensor state, and is expressed as ON (or OPEN) and OFF (or CLOSE); by correlating the sensor data information with the time information through the regional information, it is possible to identify whether the first activity trajectory of the corresponding elderly person at home conforms to the daily behavior pattern. The EM clustering algorithm can be used to perform distance clustering on the time distance features and the sensor distance features, and the distance is defined as the key factor, and the corresponding weight and score are assigned, and the weight and score are kept synchronized. The corresponding evaluation result is obtained by multiplying the weight and score to determine whether the first activity trajectory conforms to the daily behavior pattern; for example: the smaller the distance, the smaller the weight, that is, the smaller the evaluation result obtained, which means that it conforms to the daily behavior pattern; Then, the KL algorithm is used to match and analyze the time distance feature, sensor distance feature and sensor attribute feature. At this time, the matching analysis is essentially mutation point detection, and the time distance feature, sensor distance feature and sensor attribute feature can be regarded as a feature set. By finding mutation points on the set, the abnormal situation of the first activity trajectory can be identified. It also includes the mutation point detection result based on the automatic setting of the threshold of the genetic algorithm, which specifically includes: the parameter setting of the genetic algorithm: including the size of the population, the problem solution interval, the coding length, the number of iterations, the crossover rate, the selection rate, the mutation rate, the mutation rate, etc.; the initialization of the population; the calculation of the fitness of the population: including the use of the fitness function; the genetic operation: including selection, coding, crossover, mutation, decoding and forming a new population; judging whether the iteration termination condition is met: if it is met, the optimal solution is obtained, otherwise, the fitness of the population is continued to be calculated; combined with the genetic algorithm, the KL algorithm involved in this step can be understood as an improved KL algorithm; S3, after identifying the first activity trajectory, importing the first sensor data into a pre-built curve analysis model, establishing a first curve, and outputting a first angle and a first duration to generate a first state quantity; Specifically include: Constructing a curve analysis model: marking the input information as a parameter set Q, wherein the set Q includes a number of element values, marking any element value as q, taking the time series as the X-axis and any element value as the Y-axis, establishing a two-dimensional rectangular coordinate system, obtaining the first curve, and drawing a standard threshold curve corresponding to the element value on the two-dimensional rectangular coordinate system, comparing and analyzing the first curve with the corresponding standard threshold curve, obtaining the first angle θ1 and the intersection time formed by the first intersection of the first curve and the standard threshold curve, and obtaining the intersection time again to calculate the first duration ct1; wherein 0°<θ<90°; Applying the curve analysis model: substituting the first sensor data into the curve analysis model to obtain first angles and first durations corresponding to the heart rate data, blood sugar data, blood pressure data, and respiratory rate data, respectively; Based on the combination of the first angle and the duration, a formula is set to calculate the first state quantity; Then the formula based on the first state quantity is: ; In the formula, state1 represents the first state quantity, β represents the weight coefficient, and β>0; It should be noted that when the first angle θ1 (0°<θ<90°) is larger, sinθ1 is larger, and the corresponding first state quantity is larger, indicating that the probability of abnormal behavior of the elderly at home is greater; S4. Based on the OpenPose algorithm, the second sensor data is processed and analyzed to identify the first action and the first posture of the elderly person at home, and the action features and posture features are extracted based on the first action and the first posture. A second rule engine is built to map the first space and the preset second space onto a triangular mesh, and analyze whether there are intersections among the points, segments, and surfaces contained in the mapped triangular mesh, and generate a first probability that the first activity trajectory belongs to abnormal behavior or a second probability that the first activity trajectory belongs to normal behavior in combination with the spatial information in the mesh; In this embodiment, the second model is built by the Open Pose algorithm to analyze the first action and the second posture of the elderly at home. The second model is trained to identify several different interactive behaviors. After the second sensor data is input, the second model extracts the skeleton key point information of the elderly at home, analyzes the action of the elderly at home, and calculates and obtains a vector containing multiple probability values, each probability value corresponds to an interactive behavior category; by comparing these probability values, the behavior category with the largest probability value is selected and output as the classification result; wherein the skeleton key point information includes 15 bones and 14 joints; During this process, the abnormal types of abnormal behaviors are also identified for subsequent analysis; for example: stumbling or falling, prolonged immobility, abnormal posture or movements, abnormal gait, abnormal limb behavior, uncoordinated limb movements; The steps of building the second rule engine include: Perform triangulation on the home environment and divide it into a first partition and a second partition; Based on the second sensing data, a static layer and a dynamic layer are determined, and the first static space and the first dynamic space are correspondingly obtained in combination with the action features and the posture features. The first static space and the first dynamic space are merged into the first space, and the first space is discretized into points, segments or surfaces respectively; the first space and the preset second space are mapped to the triangular mesh, and whether there are intersections on the points to be intersected, the segments to be intersected and the surfaces to be intersected are analyzed, that is, whether there are intersections on the points, segments and surfaces contained in the mapped triangular mesh is analyzed, and if there are intersections, a first sub-probability is generated in combination with the spatial information in the triangular mesh; otherwise, a second sub-probability is generated through conversion analysis; wherein the line type of the segment includes a straight line segment and a circular curve; In this embodiment, the conversion analysis is to obtain the probability density function under different behaviors or at different times through statistical methods, so as to calculate the information entropy under different behaviors or at different time points, and assign weights and scores to quantify the uncertainty of the relationship, and calculate the second sub-probability under the triangular mesh, for example: a certain behavior may occur in a certain area of ​​the triangular mesh, and the triangular areas to which the behavior belongs are determined by coordinate mapping or other methods; for example: a certain behavior may occur at a certain point in time (for example: it can be determined based on the historical frequency of occurrence of the behavior in each area or the historical spatial overlap), and the behavior is mapped to the triangular mesh area and the probability within the area is calculated. The specific calculation method is formulated according to the actual situation and will not be described in detail here; It should be noted that the preset second space is usually a space where the elderly at home are close to the risk area or where the elderly at home are in an abnormal state. The first space and the preset second space are mapped to the triangular mesh. When the first space and the second space partially overlap, there will be an intersection, and there will be corresponding points to be intersected, segments to be intersected, and surfaces to be intersected; when there is an intersection, it is identified that the elderly at home have abnormal behavior; in the analysis of the "surface", the boundary line between the first space and the preset second space is included in the analysis of the segments to be intersected; Combine the triangular meshes associated with the points to be intersected, the segments to be intersected, and the surfaces to be intersected, extract the first sub-probability or the second sub-probability corresponding to the same triangular mesh, and determine the first threshold and the second threshold preset for the first partition of the triangular mesh, or the third threshold and the fourth threshold preset for the second partition, and the first threshold is greater than the third threshold, and the second threshold is greater than the fourth threshold; if the first sub-probability or the second sub-probability exists in the same triangular mesh, when the first sub-probability is greater than the first threshold or the third threshold, retain the first sub-probability, otherwise, eliminate the first sub-probability; when the second sub-probability is greater than the second threshold or the fourth threshold, retain the second sub-probability, otherwise, eliminate the second sub-probability; if the first sub-probability and the second sub-probability exist in the same triangular mesh, compare and analyze the first sub-probability and the second sub-probability, eliminate the smaller sub-probability, and compare and analyze again with the threshold of the corresponding partition; The first threshold and the third threshold are respectively set to correspond to the preset threshold corresponding to the first sub-probability in the first partition and the second partition; the second threshold and the fourth threshold are respectively set to correspond to the preset threshold corresponding to the second sub-probability in the first partition and the second partition; For example: Assume that the first threshold is 60% which is greater than the third threshold is 50%, and the third threshold is 60% which is greater than the fourth threshold is 50%; Assuming that the extracted triangular mesh is in the first partition, when the first sub-probability is greater than the first threshold value of 60%, the first sub-probability is retained; the second sub-probability is greater than the third threshold value of 60%, the second sub-probability is retained; assuming that the extracted triangular mesh is in the second partition, when the first sub-probability is greater than the third threshold value of 50%, the first sub-probability is retained; when the second sub-probability is greater than the fourth threshold value of 50%, the second sub-probability is retained; Assuming that both the first sub-probability and the second sub-probability exist in the same partition, first compare the first sub-probability and the second sub-probability. Assuming that the first sub-probability is greater than the second sub-probability, retain the first sub-probability, and compare the first sub-probability with the preset threshold value in the corresponding partition again to obtain a comparison result for subsequent analysis. Then, the first sub-probability or the second sub-probability obtained at the same intersection is averaged, and the first sub-probability at different intersections is combined with the first sub-probability or the second sub-probability at different intersections is combined with the second sub-probability, and weighted average is performed to obtain the first probability or the second probability; wherein all triangular meshes are in three-dimensional form to form an enclosing mesh; If there is an intersection, the first sub-probability is obtained by combining the spatial information in the grid, including: Convert the grid into position coordinate information, and the first spatial representation form is: ; In the formula, (x, y, z) represents the relative coordinates in the first space, (x l ,yl , z l ) represents the absolute coordinates in the first space, (X abc , Y abc , Z abc ) represents the horizontal, vertical and vertical coordinates of the corresponding origin under the grid; When points are included: extract the first position point corresponding to the first space and its point coordinates, take the first position point as the first center and draw a circle with r as the radius to form a first sphere, when the second position point of the second space exists in the circumscribed cube of the first sphere, calculate the distance value between the first position point and the second position point, take the second position point as the center and the distance value as the radius, calculate the intersection and union ratio of the second sphere formed by the second position point and the circumscribed cube of the first sphere, and set the formula to obtain the first sub-probability; The formula is: ; In the formula, μ1 represents the first sub-probability corresponding to the point, exp(·) represents the exponential function, dis represents the distance value, and V sub represents the intersection-and-union ratio, λ represents the attenuation exponent, and its value range is [0, 1], σ1 and σ2 are both weight correction coefficients; It should be noted that when the intersection-and-union ratio is larger, it means that the two spaces overlap more, the distance between the first position point and the second position point is closer, and the first sub-probability is greater; When a straight line segment is included: extract the first line segment corresponding to the first space and its endpoint coordinates, and use the first parametric equation to find the intersection of the straight line where the second line segment in the second space is located and the straight line where the first line segment is located. When the intersection exists, combine the Gaussian distribution model to calculate the ratio of the length of the intersection on the first line segment and the second line segment, and perform weighted summation to obtain the integral to obtain the first sub-probability; When a circular curve is included: extract the second center of the circle corresponding to the first space, find the intersection of the curve where the second center of the circle and the third center of the circle in the second space are located by a second parametric equation, calculate the first deflection angle of the line connecting the second center of the circle and the intersection relative to the coordinate axis; project the line connecting the start and end points of the circular curve between the second center of the circle and the second center of the circle onto the coordinate axis to determine the second deflection angle, project the line connecting the start and end points of the circular curve between the second center of the circle and the third center of the circle onto the coordinate axis to form the second deflection angle, determine the intersection interval based on the second deflection angle and the third deflection angle, and when the first deflection angle is a subset of the intersection interval, combine the curvature of the circular curve to obtain the first sub-probability; The coordinate axes are the X-axis, Y-axis, and Z-axis. Taking the case where there is only one intersection point as an example, the representation form of the intersection interval is: ; Wherein, φ represents the intersection interval, pzi represents any second deflection angle, A is a set formed by the second deflection angles, including pz1, pz2, pz3, pz4, pz5 and pz6, which respectively represent the second deflection angles of the second center of the circle and the connecting line of the starting and ending points of the circular curve of the second center of the circle relative to the X-axis, the Y-axis and the Z-axis; pqj represents any third deflection angle, B is a set formed by the third deflection angles, including pq1, pq2, pq3, pq4, pq5 and pq6, which respectively represent the third deflection angles of the connecting line of the starting and ending points of the circular curve of the second center of the circle and the third center of the circle relative to the X-axis, the Y-axis and the Z-axis; Then the formula for the first sub-probability is: ; Where μ2 represents the first sub-probability corresponding to the circular curve, pwk represents the first deflection angle, φ start Indicates the starting angle of the intersection interval, φ end Indicates the ending angle of the intersection interval, r k represents the radius of the second circle center, then 1 / r k Represents the curvature. The curvature of a circle usually causes the angle to change, thus affecting the final probability distribution. ε represents the weight correction coefficient. When a surface is included: extract a first plane corresponding to the first space and its normal vector, calculate the segment between the first plane and the second plane of the second space by plane equation, calculate the area enclosed by each endpoint of the segment and each vertex of the first plane, and combine the second angle between the center of gravity of the first plane and the segment to obtain the first sub-probability; The formula is: ; Wherein, μ3 represents the first sub-probability corresponding to the face, θ2 represents any second angle between the center of gravity of the first plane and the segment, area represents the area enclosed by each endpoint and each vertex of the first plane, Ψ1 and Ψ2 represent weight correction coefficients; It should be noted that if the area increases and the second angle approaches 0 degrees, it means that the first plane and the second plane intersect and overlap to a greater extent, then the corresponding first sub-probability will increase significantly. If the enclosed area is smaller or the second angle is larger (close to 90 degrees), then the first sub-probability will decrease significantly, indicating that the probability of abnormal behavior is smaller. It should be noted that the first parameter equation, the second parameter equation and the plane equation involved in this article are built-in intersection point calculation models of the system, which are set according to actual conditions and will not be described in detail here; In a complex or dynamically changing home environment, the intersection judgment method can better adapt to environmental noise and interference. For example, when the elderly are doing daily activities, other objects or people in the environment may interfere, but the intersection judgment method can focus on the action itself and reduce the possibility of misjudgment. By converting the traditional action similarity solution into the intersection solution, it directly focuses on the action and posture instead of relying on specific action patterns or templates. It can more flexibly adapt to the action characteristics and posture characteristics of different elderly people and avoid misjudgment caused by different behavioral habits of elderly people at home. At the same time, the intersection judgment method usually does not need to traverse the entire space or perform complex calculations, thereby reducing the calculation complexity, ensuring that the system can process sensor data in real time and respond quickly, and improving the overall performance of the system. S5. Under the condition that it is determined that the first activity trajectory has abnormal behavior, identify the abnormal type based on the abnormal behavior, establish a second curve in combination with the first probability of the abnormal behavior, identify the peak value and the second duration, so as to generate a second state quantity; The step of generating the second state quantity includes: A calculation period is set based on a time series. In the same calculation period, the first probabilities are aggregated into a first set, the abnormal types are aggregated into a second set, and the cumulative number of abnormal behaviors and the first proportion set of each abnormal type in the calculation period are obtained; Analyze the influence relationship between each element in the first set and the second set, and assign corresponding weights to each element in the first proportion set of each abnormal type based on the different influence degrees of each first probability on each abnormal type, multiply and accumulate them to generate a degree value; In combination with the first probability, a second curve of each first probability-degree value is drawn in a two-dimensional rectangular coordinate system, a peak is identified based on the second curve, and a second duration of each peak and a corresponding peak value are obtained, and a formula is set to calculate a second state quantity; Then the formula for the second state quantity is: ; In the formula, state2 represents the second state quantity, fz represents the peak value, ct2 represents the second duration, α represents the weight coefficient, and α>0; it should be noted that the higher the peak value, the larger the value corresponding to the second state quantity; It should be noted that the influence relationship involved here can be a relationship curve between each first probability and each abnormal type, which can be set through a Bayesian network or an SVM model, and the detailed steps will not be repeated; on the curve, it can be seen that under a certain abnormal type, the first probability has a corresponding numerical value (or probability), and the weight is determined by the value of the first probability. For example, the first probability of 90% corresponds to weight allocation strategy one, the first probability of 80% corresponds to weight allocation strategy two, and the first probability of 70% corresponds to weight allocation strategy three; the weight allocation strategy can also be determined by the interval in which the first probability is within, for example, the first probability is less than 90% and greater than 80% corresponds to weight allocation strategy one, the first probability is less than 80% and greater than 70% corresponds to weight allocation strategy two, the first probability is less than 70% and greater than 60% corresponds to weight allocation strategy three; the first probability is less than 60% corresponds to weight allocation strategy four. The embodiment of the present invention does not limit the different influence relationships between the first probability and the degree of influence of each abnormal type; S6, importing the first state quantity and the second state quantity into a pre-built early warning evaluation model, generating an early warning evaluation value, and triggering a plurality of early warning signals based on the early warning evaluation value; Based on the time series, the preset time interval is extracted, and the third state vector is constructed based on the first state quantity and the second state quantity, and then aggregated. The representation form is: ; ; Where R is the state vector set, , , as well as Respectively represent the state vectors in the first time interval, the second time interval, the i-th time interval, and the n-th time interval, and the value range of i is [1, n]; state1 i Represents the first state quantity in the i-th time interval, state2 i represents the second state quantity in the i-th time interval; The calculation formula of the early warning assessment model is: ; Wherein, warn represents the warning assessment value, T represents the time interval, η represents the weight correction coefficient, which is used to make the two curves similar or matching by measuring the degree of the first curve and the second curve, P1(T) represents the first curve obtained in the time interval T, P2(T) represents the second curve obtained in the time interval T, and c represents the constant bias term, which is used to avoid the negative infinity of the logarithmic result; It should be noted that the warning signal involved here can be determined by the value of the warning evaluation value, for example: when the warning evaluation value is the average plus one standard deviation, a first-level warning signal is generated; when the warning evaluation value is the average plus two standard deviations, a second-level warning signal is generated; when the warning evaluation value is the average plus three standard deviations, a third-level warning signal is generated; it can also be determined by the interval of the warning evaluation value, for example: when the warning evaluation value is greater than the average plus one standard deviation and less than or equal to its average plus two standard deviations, it is a first-level warning signal; when the warning evaluation value exceeds the average plus two standard deviations and is less than or equal to its average plus three standard deviations, it is a second-level warning signal; when the warning evaluation value is greater than the average plus three standard deviations, a third-level warning signal is generated. The embodiment of the present invention does not limit the relationship between the warning evaluation value and the warning signal setting; after the corresponding warning signal is generated, an email, text message, system notification or other alarm mechanism can be sent to remind family members.

[0018] Example 2

[0019] The embodiment of the present invention provides a state detection system for elderly people at home based on sensors; Figure 2 is a module diagram of a sensor-based state detection system for elderly people at home according to an exemplary embodiment; please refer to Figure 2 The system includes: a sensor acquisition module, a state detection module and an early warning feedback module, and the modules are connected in communication; wherein the state detection module has a first determination unit and a second determination unit built therein; A sensor acquisition module, for acquiring first sensor data, second sensor data and third sensor data of the elderly at home; The state detection module processes and analyzes the first sensor data, the second sensor data, and the third sensor data based on the DTW algorithm, identifies the first activity trajectory of the elderly at home, extracts the time distance feature, the sensor distance feature, and the sensor attribute feature based on the first activity trajectory, and builds a first rule engine to determine whether the behavior type of the first activity trajectory is normal behavior or abnormal behavior; based on the OpenPose algorithm, the second sensor data is processed and analyzed to identify the first action and the first posture of the elderly at home, extracts the action feature and the posture feature based on the first action and the first posture, and builds a second rule engine, which at least includes analyzing the grids through which the corresponding intersecting line segments in the first space and the risk feature space pass, and combining the spatial information in the grid to generate a first probability that the first activity trajectory belongs to abnormal behavior or a second probability that it is normal behavior; The first determination unit imports the first sensing data into a pre-built curve analysis model, establishes a first curve, and outputs a first angle and a first duration to generate a first state quantity; the second determination unit, under the condition that it is determined that the first activity trajectory has abnormal behavior, identifies the abnormal type based on the abnormal behavior, establishes a second curve in combination with the first probability of the abnormal behavior, identifies the peak value and the second duration to generate a second state quantity; The early warning feedback module, under the condition of determining that abnormal behavior exists, imports the first state quantity and the second state quantity into a pre-built early warning evaluation model, generates an early warning evaluation value, and triggers a plurality of early warning signals based on the early warning evaluation value.

[0020] The weight correction coefficient is determined by the coefficient of variation method, which is a method of assigning weights to each indicator based on the degree of variation between the current value of each evaluation indicator and the target value. If the numerical difference of an indicator is large, it can clearly distinguish the evaluated objects, indicating that the indicator has rich discrimination information, and thus the indicator should be given a larger weight. On the contrary, if the numerical difference of each evaluated object on a certain indicator is small, then the ability of this indicator to distinguish the evaluated objects is weak, and thus the indicator should be given a smaller weight. This method directly uses the information contained in each indicator to obtain the weight of the indicator through calculation, and therefore is objective.

[0021] In the application, the several formulas involved are all calculated by removing the dimensions and taking their numerical values. The formula is a formula obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The formula is set by technical personnel in this field according to actual conditions.

[0022] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. A person of ordinary skill in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0023] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0024] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A sensor-based method for detecting the state of elderly people at home, characterized in that: The method comprises: Acquire first sensor data, second sensor data and third sensor data of the elderly at home; Based on the DTW algorithm, the first sensor data, the second sensor data and the third sensor data are processed and analyzed to identify the first activity trajectory of the elderly at home, and the time distance feature, the sensor distance feature and the sensor attribute feature are extracted based on the first activity trajectory, and a first rule engine is built to determine whether the behavior type of the first activity trajectory is normal behavior or abnormal behavior; Based on the OpenPose algorithm, the second sensor data is processed and analyzed to identify the first action and the first posture of the elderly person at home, and the action features and posture features are extracted based on the first action and the first posture. A second rule engine is built to map the first space and the preset second space to a triangular mesh, and analyze whether there are intersections among the points, segments, and surfaces contained in the mapped triangular mesh. Combined with the spatial information in the mesh, a first probability that the first activity trajectory belongs to abnormal behavior or a second probability that it is normal behavior is generated.

2. A sensor-based state detection method for elderly people at home according to claim 1, characterized in that: After the first activity trajectory is identified, the first sensor data is imported into a pre-built curve analysis model, a first curve is established, and a first angle and a first duration are output to generate a first state quantity; Under the condition that it is determined that the first activity trajectory has abnormal behavior, the abnormal type is identified based on the abnormal behavior, and a second curve is established in combination with the first probability of the abnormal behavior to identify the peak value and the second duration to generate a second state quantity; The first state quantity and the second state quantity are imported into a pre-built early warning evaluation model to generate an early warning evaluation value, and a plurality of early warning signals are triggered based on the early warning evaluation value.

3. A sensor-based state detection method for elderly people at home according to claim 1, characterized in that: The first sensor data includes but is not limited to heart rate, blood sugar, blood pressure and respiratory rate; the second sensor data includes but is not limited to video; the third sensor data includes but is not limited to temperature and humidity, and door switch.

4. A sensor-based state detection method for elderly people at home according to claim 1, characterized in that: The steps to build the first rule engine include: Based on the KL algorithm, the time distance feature, the sensor distance feature and the sensor attribute feature are matched and analyzed to determine whether the deviation degree of the first activity trajectory is within a preset range, and the preset range is dynamically set based on the genetic algorithm; When the match is successful, that is, within the preset range, it is determined to be normal behavior; Otherwise, it is determined to be an abnormal behavior; when the matching fails, the time distance feature, the sensor distance feature and the sensor attribute feature are input into the preset trajectory evaluation model to obtain the deviation trajectory of the first activity trajectory and the corresponding deviation degree.

5. A sensor-based state detection method for elderly people at home according to claim 1, characterized in that: The steps to build the second rule engine include: Divide the home environment into a triangular mesh and divide it into a first partition and a second partition; wherein the triangular mesh is in a three-dimensional form to form an enclosing mesh; Determine the static layer and the dynamic layer based on the second sensor data, combine the action features and the posture features, generate the first space, and discretize the first space into points, segments or surfaces respectively; wherein the line type of the segment includes a straight line segment and a circular curve; Map the first space and the preset second space onto the triangular mesh, analyze whether there are intersections on the to-be-intersected points, to-be-intersected segments, and to-be-intersected surfaces, and if there are intersections, generate the first sub-probability by combining the spatial information in the triangular mesh; otherwise, generate the second sub-probability by conversion analysis; Combine the triangular meshes associated with the to-be-intersected points, to-be-intersected segments, and to-be-intersected surfaces, extract the corresponding sub-probabilities under the same triangular mesh, screen and analyze the sub-probabilities based on the preset thresholds of the first partition and the second partition, retain the sub-probabilities greater than the preset thresholds, and if the first sub-probability and the second sub-probability exist in the same triangular mesh, eliminate the smaller sub-probability and compare and analyze it again with the preset threshold; process the mean of the sub-probabilities under the same intersection points, and combine the sub-probabilities under different intersection points to obtain the corresponding first probability or second probability.

6. A sensor-based state detection method for elderly people at home according to claim 5, characterized in that: If there is an intersection, the first sub-probability is obtained by combining the spatial information in the grid, including: When points are included: extract the first position point and its point coordinates, take the first position point as the first center and draw a circle with r as the radius to form a first sphere, when the second position point exists in the circumscribed cube of the first sphere, take the distance value as the radius, calculate the intersection and union ratio of the second sphere formed by the second position point and the circumscribed cube of the first sphere, and set the formula to obtain the first sub-probability; The formula is: ; In the formula, μ1 represents the first sub-probability corresponding to the point, exp(·) represents the exponential function, dis represents the distance value, and V sub represents the intersection-and-union ratio, λ represents the attenuation index, and σ1 and σ2 are weight correction coefficients; When a straight line segment is included: extract the coordinates of the first line segment and its endpoints, calculate the intersection of the second line segment and the straight line where the first line segment is located by using the first parametric equation, calculate the ratio of the length of the intersection on the first line segment to the length of the second line segment by combining the Gaussian distribution model, and perform weighted summation to obtain the integral to obtain the first sub-probability; When a circular curve is included: extract the second center of the circle, find the intersection of the curves where the second center of the circle and the third center of the circle are located by using the second parameter equation, determine the first deflection angle based on the line connecting the second center of the circle and the intersection, determine the second deflection angle based on the line connecting the second center of the circle and the start and end points of the circular curve of the second center of the circle, determine the third deflection angle based on the line connecting the start and end points of the circular curve of the second center of the circle and the third center of the circle, determine the intersection interval based on the second deflection angle and the third deflection angle, and when the first deflection angle is a subset of the intersection interval, obtain the first sub-probability in combination with the curvature of the circular curve; When a face is included: extract the first plane, calculate the segment sandwiched by the first plane and the second plane, calculate the area enclosed by each endpoint of the segment and each vertex of the first plane, and combine the second angle between the center of gravity of the first plane and the segment to obtain the first sub-probability.

7. A sensor-based state detection method for elderly people at home according to claim 2, characterized in that: Identifying an abnormal type based on the abnormal behavior, combining the first probability of the abnormal behavior, identifying a peak value and a second duration, so as to generate a second state quantity, including: A measurement period is set based on a time series. In the same measurement period, the first probabilities are aggregated into a first set, the abnormal types are aggregated into a second set, and the cumulative number of abnormal behaviors and the first percentage set of each abnormal type in the measurement period are obtained; the influence relationship between each element in the first set and the second set is analyzed, and each element in the first percentage set is weighted to generate a degree value; In combination with the first probability, a second curve of each first probability-degree value is drawn, a peak is identified based on the second curve, and a second state quantity is generated in combination with a second duration of each peak and a corresponding peak quantity.

8. A sensor-based state detection method for elderly people at home according to claim 2, characterized in that: Importing the first state quantity and the second state quantity into a pre-built early warning evaluation model includes: Based on the time series, the preset time interval is extracted, and the third state vector is constructed based on the first state quantity and the second state quantity, and is aggregated into R. The calculation formula based on the early warning evaluation model is: ； Where warn represents the warning assessment value, T represents the time interval, η represents the weight correction coefficient, P1(T) represents the first curve obtained in the time interval T, P2(T) represents the second curve obtained in the time interval T, and c represents the constant bias term.

9. A sensor-based status detection system for elderly people at home, characterized in that: The system comprises: A sensor acquisition module, for acquiring first sensor data, second sensor data and third sensor data of the elderly at home; The state detection module processes and analyzes the first sensor data, the second sensor data and the third sensor data based on the DTW algorithm, identifies the first activity trajectory of the elderly at home, extracts the time distance feature, the sensor distance feature and the sensor attribute feature based on the first activity trajectory, and builds a first rule engine to determine whether the behavior type of the first activity trajectory is normal behavior or abnormal behavior; based on the OpenPose algorithm, the second sensor data is processed and analyzed to identify the first action and the first posture of the elderly at home, and the action feature and the posture feature are extracted based on the first action and the first posture, and a second rule engine is built to map the first space and the preset second space to the triangular mesh, analyze whether there are intersections among the points, segments and faces contained in the mapped triangular mesh, and generate a first probability that the first activity trajectory belongs to abnormal behavior or a second probability that the first activity trajectory belongs to normal behavior in combination with the spatial information in the mesh; the state detection module is built with a first determination unit and a second determination unit; The first determination unit imports the first sensing data into a pre-built curve analysis model, establishes a first curve, and outputs a first angle and a first duration to generate a first state quantity; the second determination unit, under the condition that it is determined that the first activity trajectory has abnormal behavior, identifies the abnormal type based on the abnormal behavior, establishes a second curve in combination with the first probability of the abnormal behavior, identifies the peak value and the second duration to generate a second state quantity; The early warning feedback module, under the condition of determining that abnormal behavior exists, imports the first state quantity and the second state quantity into a pre-built early warning evaluation model, generates an early warning evaluation value, and triggers a plurality of early warning signals based on the early warning evaluation value.

Images