Door-to-door pension service supervision system based on millimeter wave radar and multi-source sensing fusion

Abstract

A door-to-door pension service supervision system based on millimeter-wave radar and multi-source sensing fusion comprises the millimeter-wave radar, a microphone, a gyroscope, a satellite positioning module, a communication module, a loudspeaker, two processors, a battery, a memory card, a power key and a service key, and door-to-door verification and service quality evaluation of nursing workers are achieved. On-site verification of nursing workers is completed through satellite positioning, recognition, positioning, tracking and activeness evaluation of nursing workers and old people are completed through a millimeter wave radar monitoring algorithm in combination with a microphone and a gyroscope, and finally nursing worker service quality evaluation is formed.

Description

Technical Field

[0001] This invention relates to technologies in the field of health and elderly care, specifically a multi-source sensing fusion monitoring system for in-home elderly care services that integrates four types of sensors: millimeter-wave radar, Global Navigation Satellite System (GNSS), microphone, and gyroscope. Background Technology

[0002] Currently, the regulation of in-home elderly care services faces challenges in the field of health and elderly care, especially in the areas of verifying caregivers' arrival and assessing service quality, where there is a lack of effective technological products. Summary of the Invention

[0003] This invention addresses the challenges of supervising in-home elderly care services by proposing a monitoring system (hereinafter referred to as the monitoring system) that integrates four types of sensors: millimeter-wave radar, satellite positioning, microphone, and gyroscope. This system enables verification of caregivers' arrival and assessment of service quality.

[0004] This invention is achieved through the following technical solution: This invention is a self-powered portable system with a built-in rechargeable battery, and its overall dimensions are approximately 10cm × 10cm × 2cm. It can be carried by caregivers to the elderly person's home.

[0005] Firstly, regarding the issue of verifying caregivers' home visits, the monitoring system integrates a satellite positioning module, which can locate the location of caregivers when they provide services, ensuring that caregivers provide home visits.

[0006] Secondly, for the assessment of caregiver service quality, the monitoring system integrates three sensing modules: millimeter-wave radar, microphone, and gyroscope.

[0007] The millimeter-wave radar is the core of this invention, mainly consisting of two parts: millimeter-wave radar hardware and millimeter-wave radar algorithm. The millimeter-wave radar hardware generates millimeter-wave radar point cloud data, and the millimeter-wave radar algorithm analyzes the millimeter-wave radar point cloud to perform clustering, target extraction, matching, Kalman tracing, caregiver and elderly identification, and caregiver behavior identification. The quality of caregiver services is evaluated through the caregiver's movement trajectory, the positional relationship between the caregiver and the elderly, and the caregiver's behavioral activity.

[0008] The microphone has recording and uploading functions, and is used to record the sound of the service process, which is an important supplement to confirm the quality of caregiver services.

[0009] The gyroscope is used to assist the millimeter-wave radar in identifying service scene switching.

[0010] In addition to the three sensing modules, the monitoring system also integrates a communication module, which can upload monitoring data to the cloud and achieve unified management of information from multiple devices.

[0011] Technical effect This invention comprehensively solves the problem of supervising home-based elderly care services provided by caregivers; it solves the problem of verifying caregivers' home visits through a satellite positioning module; and it solves the problem of assessing the quality of caregiver services by combining millimeter-wave radar with microphones and gyroscopes. Attached Figure Description

[0012] Figure 1 A structural diagram of the in-home elderly care service supervision system.

[0013] Figure 2 This is a physical circuit diagram of a home-based elderly care service supervision system.

[0014] Figure 3 A flowchart of the workflow for the in-home elderly care service supervision system.

[0015] Figure 4 The diagram shows the schematic of a millimeter-wave radar circuit in the embodiment. In the diagram, LAN stands for Low-Noise Amplifier, PGA stands for Programmable Gain Amplifier, PA stands for Power Amplifier, ADC stands for Analog-to-Digital Converter, and FMCW WaveformGenerator refers to a linear frequency modulated continuous wave waveform generator.

[0016] Figure 5 This is a flowchart of a millimeter-wave radar algorithm. Detailed Implementation

[0017] like Figure 1 , Figure 2 The diagram shown is a structural block diagram of the in-home elderly care service monitoring system used in this embodiment, including: millimeter-wave radar, microphone, gyroscope, satellite positioning module, communication module, speaker, two processors, battery, memory card, power button, and service button; the specific workflow of the monitoring system is as follows: Figure 3 As shown: When a caregiver arrives at the home and presses the power button, the device turns on and prompts the caregiver to point the device at the service scene via a speaker. After the caregiver presses the service button, the system activates millimeter-wave radar monitoring, recording, satellite positioning, and timing functions. During the service, if the gyroscope detects movement of the device, it determines that the scene has changed and restarts the millimeter-wave radar monitoring function. When the caregiver presses the service button again, a voice prompt indicates that the service has ended, and the device uploads the recorded information to the cloud. The system then uses the millimeter-wave radar monitoring information, recording, satellite positioning, and service duration to evaluate the quality of the service.

[0018] The millimeter-wave radar includes millimeter-wave radar hardware and millimeter-wave radar monitoring algorithms; such as Figure 4As shown, the millimeter-wave radar hardware includes a radar front-end consisting of four transmit (TX) links and four receive (RX) links, and a baseband processing unit consisting of four ADCs, a microcontroller processing unit, and a point cloud generation unit. The transmit links emit electromagnetic waves into the environment, which are reflected back to the receive links by the target and converted into baseband signals. After sampling by the ADCs, the microcontrollers and the point cloud generation module process the signals to obtain environmental point cloud data. The point cloud data is further transmitted to processor 1 and processed by the millimeter-wave radar monitoring algorithm to obtain millimeter-wave radar monitoring information.

[0019] The specific steps of the millimeter-wave radar monitoring algorithm are as follows: Figure 5 As shown, it is divided into two parts: point cloud processing and target generation, and service quality assessment.

[0020] The point cloud processing and target generation algorithm is described in detail below: Define the point cloud of frame t as a point set obtained by parsing. ,in For the nth point of the t-th frame, Let be the total number of points in frame t; let be the set of historical targets. ,in For the m-th historical target in frame t, Let be the number of historical targets in frame t; each historical target At least include: location Initial position Continuous matching frame count Remaining lifespan Target state (0 = candidate / unconfirmed, 1 = confirmed), and trajectory .

[0021] 1) Point Cloud Input and Parsing (Frame-by-Frame, Coordinate Transformation): After the point cloud is input, it is parsed frame-by-frame according to the point cloud packing protocol to obtain a single-frame spherical coordinate system point cloud set: ,in Let n be the distance from the radar to the nth point. It is the azimuth angle. For pitch / horizontal angles (as defined by the device); obtain the Cartesian coordinate system point cloud through coordinate transformation: .

[0022] 2) Boundary noise point filtering (height / ROI constraint): Perform boundary noise filtering on the point cloud, removing points that do not meet the height constraint. Remove, where d is the installation height or the effective height value determined by the installation height, and retain only the point cloud within the core height range (XOY plane ROI constraints can be superimposed if necessary).

[0023] 3) DBSCAN clustering: This method clusters the filtered point cloud onto a plane. Performing DBSCAN clustering yields the following results: The current target cluster and noise point set: , ,in This represents the k-th cluster. The number of points within the cluster; This is the set of noise points, i.e., the noise cluster.

[0024] 4) Hungarian matching and noise matching (cross-frame correlation and lifespan): ① Extraction of location information for current cluster / historical targets: For each current target cluster Calculate the average value of points within a cluster as the cluster location: The historical target location is ,in ; ② Hungarian matching (results in a set of matched pairs and a set of unmatched pairs): using and Construct the cost matrix, use the Hungarian algorithm to complete the matching, and obtain the set of matching pairs: , Indicate historical goals With the current cluster Matching, total There are matching pairs, of which the set of targets with no matching history is: No match found in the current target set: ; ④ Noise matching (noise density lifespan): Matches unmatched current target clusters. Add the points to the noise point set For each target without a matching history In its position Surrounding radius Statistical noise points in the neighborhood ,in Let be the typical radius of the point cluster (e.g., 1m); if ,in If the noise point threshold is set (e.g., 2), then its remaining lifespan is... Increase Frames (e.g., 5), with an upper limit constraint: ,in This is the maximum lifespan limit (e.g., 100 frames).

[0025] 5) Kalman filter update (matching pair position update and trajectory writing): For each matching pair Using the current cluster position as the observation: Combining the historical target state (prediction / update) from the previous frame, the historical target position is updated using Kalman filtering: And write the new location into the target trajectory: .

[0026] 6) Target Update (New / Confirmed / Mismatched Delete): ① New historical target generation, generated from unmatched current clusters: For each unmatched current target cluster... Generate new historical goals and initialize (let) ): , in Set the initial survival period for candidate targets and initialize their trajectories. ; ② Update the properties of matched historical targets: For each matched historical target : ; like and ,but: And set its lifespan: ,in Determine the number of matching frames required for the target. The lifecycle of the confirmed target It can be set to 10 frames. It can be set to 100 frames; ③ Update and remove the properties of unmatched historical targets: For each unmatched historical target : like Then from Remove the historical target from the list.

[0027] The service quality assessment algorithm is described in detail below: Definition: At most one caregiver primary objective exists at any given time. With an elderly person as a secondary goal Add the following evaluation field to each historical goal: Caregiver Flag Elderly symbol Count of elderly candidates Near-caregiver count ; Elderly anchor point and anchor point radius .

[0028] 1) Caregiver Identification Update (Main Target Selection): From Selecting caregivers as the main target and update : ① Construction of the candidate set of caregivers: Constructing the candidate set Its elements satisfy: It also has "active" displacement constraints: ,in The displacement constraint threshold is used, while also satisfying stability constraints: ,in It is the minimum number of consecutive matching frames required, and also requires Those who are already elderly cannot be used as caregivers; ② Caregiver reselection trigger conditions: Reselection will be triggered if any of the following conditions are met: a) There is currently no caregiver. b) The current caregiver is removed in this frame; c) The current caregiver's lifespan is reduced due to continuous mismatch. ,in For the maximum lifespan of caregivers, The maximum mismatch threshold for triggering reselection; ③ Caregiver primary target selection rules: When reselection is triggered, select: , in Weights (or "compare first") Compare again (Lexicographical order rule); after selection, maintain the caregiver's flag: ,in For the goal caregiver sign This indicates the maintenance caregiver's trajectory for all: .

[0029] 2) Candidate Elderly Person Identification (Low Mobility Range Targets): For targets other than caregivers, identify "candidate elderly persons" and update the list. , ; ① Window length setting: If a caregiver is present, a short window will be used. If a caregiver is not available, use a long window. ; ② Calculation of movement range: Take the target recent Frame trajectory point set (insufficient) (Frames do not participate in the candidate selection). , Calculate the movement range: , ③ Candidate determination and count update: If the following conditions are met: ,in The threshold for the movement range of the candidate is used. If the survival threshold is set for counting candidate targets, then: ,otherwise: ; ④ Proximity to Caregiver Count (Activated when a caregiver is present): If a caregiver is present, calculate the distance between the caregiver and the target. , like ,in Let the distance threshold be the distance between the candidate target and the caregiver. ,otherwise: .

[0030] 3) Elderly Identification Update (Secondary Target Selection and Error Detection): Selecting secondary targets for elderly individuals from the candidate elderly population. and update : ①Reselection trigger condition for elderly: Reselection will be triggered if any of the following conditions are met: a) There are currently no elderly people. b) The current elderly person has been removed or their lifespan. c) The current continuous mismatch among the elderly reduces their lifespan. ,in For the maximum lifespan of the elderly, The maximum mismatch threshold for elderly individuals to trigger reselection; ② Elderly Candidate Set and Selection Rules: Constructing the Elderly Candidate Set: ,in Set a candidate confirmation threshold (e.g., 20 frames); if a caregiver exists, prioritize that selection. Largest one: , If a caregiver is unavailable, then choose [the caregiver]. The largest, the second largest smallest: Select and update caregiver badge: ,in For the goal The symbol of an elderly person; ③ Establishing anchor points for the elderly: When the elderly's goals stably reach a threshold (e.g.) and When setting anchor points: The radius of the anchor point is This is used for stability constraints in subsequent bonding scenarios.

[0031] 4) Caregiver Activity Assessment: Caregiver Activity ACT is a parameter ranging from 0 to 1. The calculation method is as follows: .

[0032] 5) Service quality assessment, assessment window Inside, tracking the caregiver's movements Calculate service metrics and output service quality: ①Activity range indicators: , ② Average velocity index (trajectory difference): ; ③ Percentage of caregivers living close to elderly (distance threshold between caregiver and elderly person): In the formula, I represents an indicator function, which takes the value of 1 when the condition inside the parentheses is true, and 0 otherwise; ④ Tracking reliability penalty (caregiver mismatch / low survival): Percentage of mismatched frames with available caregivers within the window: I (caregiver not matched in this frame), where I in the formula represents an indicator function; ⑤ Service quality rating: ,in , , , These are the weights of the four indicators.

[0033] 6) Outputs: Caregiver / elderly person identification tags, their locations and trajectories, caregiver activity level, and rating. .

[0034] The recording function refers to recording ambient sounds through a microphone, writing them to a storage device such as an SD card, and uploading them to the cloud for storage.

[0035] The aforementioned satellite positioning function refers to obtaining the current location of the device through satellite positioning systems such as BeiDou, GPS, or Galileo.

[0036] The aforementioned timing function refers to timing the entire service process through the processor's internal timing mechanism.

[0037] The aforementioned gyroscope-sensing device movement refers to using a gyroscope to sense the deflection state of the device and confirm whether the device has moved.

[0038] Uploading to the cloud refers to transmitting local recording data, caregiver tracking, elderly person tracking, caregiver activity levels, service quality assessments, and other data to a cloud storage device for unified management.

[0039] In the text, " "Indicates assignment to the left;" "" indicates that the conclusion on the right is true when the condition on the left is true; max represents the largest and min represents the smallest.

[0040] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A home-based elderly care service monitoring system based on millimeter-wave radar and multi-source sensor fusion, enabling verification of caregivers' home visits and assessment of service quality, characterized in that: include: Millimeter-wave radar, microphone, gyroscope, satellite positioning module, communication module, speaker, two processors, battery, memory card, power button, service button; Satellite positioning is used to verify caregivers' home visits. Millimeter-wave radar monitoring algorithms are used to identify, locate, track, and assess the activity levels of caregivers and elderly people, ultimately forming a caregiver service quality assessment. Gyroscopes are used to determine service scenario switching, and microphones are used to record scene audio, supplementing the service quality assessment.

2. The in-home elderly care service monitoring system based on millimeter-wave radar and multi-source sensor fusion as described in claim 1, characterized in that: The millimeter-wave radar surveillance algorithm consists of two parts: point cloud processing and target generation algorithms, and service quality assessment algorithms.

3. The in-home elderly care service monitoring system based on millimeter-wave radar and multi-source sensor fusion as described in claim 2, characterized in that: The point cloud processing and target generation algorithm is as follows: Define the point cloud of frame t as a point set obtained through parsing. ,in For the nth point of the t-th frame, Let be the total number of points in frame t; let be the set of historical targets. ,in For the m-th historical target in frame t, Let be the number of historical targets in frame t; each historical target At least include location Initial position Continuous matching frame count Remaining lifespan Target state 0 indicates unconfirmed, 1 indicates confirmed, and the trajectory. ; 1) Point Cloud Input and Parsing: After the point cloud is input, it is parsed frame by frame according to the point cloud packing protocol to obtain a single frame of point cloud set in spherical coordinates: ,in Let n be the distance from the radar to the nth point. It is the azimuth angle. The horizontal angle is used; the Cartesian coordinate system point cloud is obtained through coordinate transformation: ; 2) Boundary noise point filtering: Perform boundary noise filtering on the point cloud, removing points that do not meet the height constraints. Remove, where d is the installation height, and retain only the point cloud within the core height range; 3) DBSCAN clustering: This method clusters the filtered point cloud onto a plane. Performing DBSCAN clustering yields the following results: The current target cluster and noise point set: , ,in This represents the k-th cluster. The number of points within the cluster; This refers to the set of noise points, i.e., noise clusters. 4) Hungarian matching and noise matching: ① Extraction of location information of current cluster and historical targets: For each current target cluster Calculate the average value of points within a cluster as the cluster location: The historical target location is ,in ; ② Hungarian matching yields a set of matched pairs and a set of unmatched pairs: and Construct the cost matrix, use the Hungarian algorithm to complete the matching, and obtain the set of matching pairs: , Indicate historical goals With the current cluster Matching, total There are 1 matching pairs; among which the set of targets with no matching history is: No match found in the current target set: ; ④ Noise matching, noise density continuation: Resolve all unmatched current target clusters Add the points to the noise point set For each target without a matching history In its position Surrounding radius Statistical noise points in the neighborhood ,in is the typical radius of the point cluster; like: ,in If the noise point count threshold is used, then its remaining lifespan is... Increase Frames, and apply upper limit constraints: ,in This represents the maximum lifespan limit. 5) Kalman filter update, matching pair position update and trajectory writing: For each matching pair Using the current cluster position as the observation: Based on the historical target state of the previous frame, the historical target position is updated using Kalman filtering: And write the new location into the target trajectory: ; 6) Target Update: ① New historical target generation, generated from unmatched current clusters: For each unmatched current target cluster... Generate new historical goals and initialize, let : ， in Set the initial survival period for candidate targets and initialize their trajectories. ; ② Update the properties of matched historical targets: For each matched historical target : ， like and ,but: And set its lifespan: ,in Determine the number of matching frames required for the target. The lifecycle of a confirmed target; ③ Update and remove the properties of unmatched historical targets: For each unmatched historical target : , ,like Then from Remove the historical target from the list.

4. The in-home elderly care service monitoring system based on millimeter-wave radar and multi-source sensor fusion as described in claim 2, characterized in that: The service quality assessment algorithm is defined as follows: at most one caregiver primary objective exists at any given time. With an elderly person as a secondary goal ; for each historical goal Add the following assessment field: Caregiver Marks Elderly symbol Count of elderly candidates Near-caregiver count ; Elderly anchor point and anchor point radius ; 1) Caregiver identification update, i.e., primary target selection: from Selecting caregivers as the main target and update : ① Construction of the candidate set of caregivers: Constructing the candidate set Its elements satisfy: , It also has "active" displacement constraints: ,in The displacement constraint threshold is used, while also satisfying stability constraints: ,or ,in It is the minimum number of consecutive matching frames required, and also requires Those who are already elderly cannot be used as caregivers; ② Caregiver reselection trigger conditions: Reselection will be triggered if any of the following conditions are met: a) There is currently no caregiver. b) The current caregiver is removed in this frame; c) The current caregiver's lifespan is reduced due to continuous mismatch. ,in For the maximum lifespan of caregivers, The maximum mismatch threshold for triggering reselection; ③ Caregiver primary target selection rules: When reselection is triggered, select: ,in Weighting; after selection, maintain the caregiver flag: ,in For the goal caregiver sign This indicates the maintenance caregiver's trajectory for all: ; 2) Candidate elderly person identification, i.e., low-mobility target: For targets other than caregivers, identify "candidate elderly persons" and update the list. , ; ① Window length setting: If a caregiver is present, a short window will be used. If a caregiver is not available, use a long window. ; ② Calculation of movement range: Take the target recent The set of frame trajectory points is insufficient. Frames are not considered for the candidate pool. ， Calculate the movement range: ，， ③ Candidate determination and count update: If the following conditions are met: ,in The threshold for the movement range of the candidate is used. If the survival threshold is set for counting candidate targets, then: ,otherwise: ; ④ Approaching caregiver count, activated when a caregiver is present: If a caregiver is present, calculate the distance between the caregiver and the target. ， like ,in Let the distance threshold be the distance between the candidate target and the caregiver. ,otherwise: ; 3) Elderly identification and updating, i.e., secondary target selection: Selecting elderly secondary targets from the candidate elderly. and update ; ①Reselection trigger condition for elderly: Reselection will be triggered if any of the following conditions are met: a) There are currently no elderly people. b) The current elderly person has been removed or their lifespan. c) The current continuous mismatch among the elderly reduces their lifespan. ,in For the maximum lifespan of the elderly, The maximum mismatch threshold for elderly individuals to trigger reselection; ② Elderly Candidate Set and Selection Rules: Constructing the Elderly Candidate Set: ,in This is the candidate confirmation threshold; if a caregiver exists, it is selected first. Largest one: ， If a caregiver is unavailable, then choose [the caregiver]. The largest, the second largest smallest: Select to update the senior citizen icon: ,in For the goal The symbol of an elderly person; ③ Establishing anchor points for elderly individuals: When the elderly person's target stably reaches the threshold, set an anchor point: The radius of the anchor point is This is used for stability constraints in subsequent contact scenarios; 4) Caregiver Activity Assessment: Caregiver Activity ACT is a parameter ranging from 0 to 1. The calculation method is as follows: ; 5) Service quality assessment, assessment window Inside, tracking the caregiver's movements Calculate service metrics and output service quality: ①Activity range indicators: ; ② Average speed index: ； ③ Proportion of caregivers , The effective service distance threshold; ④ Tracking reliability penalty: using the proportion of mismatched frames within the window by the caregiver or The percentage of frames below the threshold indicates: ； ⑤ Service quality rating: ,in , , , These are the weights of the four indicators.

Images