Nursing behavior recognition method, system and product based on radar

Abstract

The invention provides a radar-based nursing behavior identification method, system and product, and the method comprises the steps: carrying out the target detection of collected radar echo data, and generating a target 2D point cloud based on a target detection result; performing clustering processing based on the target 2D point cloud, and determining position information of the nursed person based on the clustered target 2D point cloud; performing point cloud distribution on the clustered target 2D point cloud based on the position information of the nursed person to obtain a nurse point cloud; performing trajectory tracking on the point cloud of the nursing personnel to obtain a trajectory coordinate sequence; extracting nursing behavior characteristic parameters, and performing nursing behavior compliance judgment based on the nursing behavior characteristic parameters; and generating a nursing behavior compliance auditing report based on the nursing behavior compliance determination result. According to the nursing behavior identification method provided by the invention, the positioning of the old and nursing workers can be realized without any prior position information, and the inspection of nursing behaviors in different scenes can be supported.

Description

Technical Field

[0001] This application relates to the field of radar technology, and in particular to a radar-based method, system, and product for recognizing nursing behaviors. Background Technology

[0002] With the aging of the population, home-based elderly care has become a major social necessity. In recent years, the development of internet information technology has brought new opportunities to the elderly care field. How to leverage technology to build new home-based elderly care service models has become a key focus for both academia and industry. However, current home-based elderly care services face many challenges: on the one hand, there is room for improvement in service satisfaction rates; on the other hand, it is difficult to effectively obtain service quality evaluations when caregivers provide services, leading to low management efficiency. Currently, the most common method for supervising home-based elderly care services is computer vision technology, employing solutions including video recording and facial recognition. However, while these methods can record the care process, the costs of equipment purchase, deployment, and maintenance are high, video analysis consumes significant computing resources, and there is a risk of privacy leaks. Some solutions also utilize Bluetooth beacon technology and satellite positioning technology; however, these technologies can only locate personnel and cannot effectively supervise the care process.

[0003] Furthermore, in actual nursing scenarios, nursing needs are diverse. Besides routine bed care, it is also common to transfer disabled elderly individuals to off-bed settings (such as wheelchairs) for care. When providing care in different scenarios, there are challenges in accurately locating and distinguishing the caregiver from the elderly. Simultaneously, developing reasonable nursing indicators and achieving automated auditing of nursing behaviors is also one of the current technical difficulties.

[0004] Therefore, there is an urgent need for a nursing behavior recognition method that can effectively address the difficulties in identifying the location of caregivers and the elderly in different scenarios, recognize nursing actions, and effectively monitor the nursing process. Summary of the Invention

[0005] The purpose of this application is to provide a radar-based method, system, and product for identifying nursing behaviors, which can at least solve the problems in related technologies of difficulty in identifying the location of caregivers and elderly people in different scenarios and difficulty in effectively supervising the nursing process.

[0006] To address the aforementioned technical problems, the first aspect of this application provides a radar-based nursing behavior recognition method, comprising:

[0007] Target detection is performed on the collected radar echo data, and a 2D point cloud of the target is generated based on the target detection results;

[0008] Clustering is performed on the target 2D point cloud, and the location information of the person being cared for is determined based on the clustered target 2D point cloud.

[0009] Based on the location information of the person being cared for, point cloud allocation is performed on the clustered target 2D point cloud to obtain the point cloud of the caregiver;

[0010] Trajectory tracking was performed on the point cloud of the nursing staff to obtain a trajectory coordinate sequence;

[0011] Nursing behavior feature parameters are extracted based on the trajectory coordinate sequence, and the compliance of nursing behavior is determined based on the nursing behavior feature parameters.

[0012] A nursing behavior compliance audit report is generated based on the nursing behavior compliance assessment results.

[0013] A second aspect of this application provides a radar-based nursing behavior recognition system, comprising:

[0014] The point cloud generation module is used to perform target detection on the collected radar echo data and generate a 2D point cloud of the target based on the target detection results.

[0015] The location acquisition module is used to perform clustering processing based on the target 2D point cloud, and determine the location information of the person being cared for based on the clustered target 2D point cloud;

[0016] The point cloud allocation module is used to allocate the clustered target 2D point cloud based on the location information of the person being cared for, so as to obtain the point cloud of the caregiver.

[0017] The trajectory tracking module is used to track the trajectory of the nursing staff point cloud to obtain a trajectory coordinate sequence;

[0018] The behavior determination module is used to extract nursing behavior feature parameters based on the trajectory coordinate sequence, and to determine the compliance of nursing behavior based on the nursing behavior feature parameters.

[0019] The report generation module is used to generate nursing behavior compliance audit reports based on the nursing behavior compliance assessment results.

[0020] A third aspect of this application provides a radar, including a memory and a processor, wherein the processor is used to execute a computer program stored in the memory, and when the processor executes the computer program, it implements the steps of the nursing behavior recognition method described in the first aspect of the embodiments of this application.

[0021] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the nursing behavior recognition method described in the first aspect of the embodiments of this application.

[0022] As can be seen from the above, the embodiments of this application first perform target detection on the collected radar echo data, and generate a target 2D point cloud based on the target detection results. Then, clustering processing is performed on the target 2D point cloud, and the location information of the cared-for personnel is determined based on the clustered target 2D point cloud. Next, point cloud allocation is performed on the clustered target 2D point cloud based on the location information of the cared-for personnel to obtain the nursing personnel point cloud. Then, trajectory tracking is performed on the nursing personnel point cloud to obtain the trajectory coordinate sequence. Next, nursing behavior feature parameters are extracted based on the trajectory coordinate sequence, and nursing behavior compliance is determined based on the nursing behavior feature parameters. Finally, a nursing behavior compliance audit report is generated based on the nursing behavior compliance determination result. The nursing behavior recognition method proposed in this invention can locate the elderly and caregivers without requiring any prior location information, supporting the auditing of nursing behaviors in different scenarios and greatly increasing the versatility of this recognition method. Furthermore, this method does not require the collection of actual image data, enabling the identification of personnel in a scene without infringing on privacy, and simultaneously identifying nursing behaviors. It allows for full-process monitoring of the nursing process, effectively improving the effectiveness of nursing supervision. Moreover, it can effectively judge the compliance of nursing behaviors and provide audit reports, effectively reducing the time cost of auditing nursing work orders.

[0023] It should be understood that the description in this section is not intended to identify key or important features of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the related technologies or the technical solutions in the embodiments of this application, the drawings used in the description of the related technologies or the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and not all embodiments. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating the nursing behavior recognition method provided in this application embodiment;

[0026] Figure 2 A detailed flowchart illustrating the nursing behavior recognition method provided in the embodiments of this application;

[0027] Figure 3 This is a schematic diagram of the program modules of the nursing behavior recognition system provided in the embodiments of this application;

[0028] Figure 4A block diagram of a radar provided in an embodiment of this application;

[0029] Figure 5 A block diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application more apparent and understandable, this application will be clearly and completely described below in conjunction with its embodiments and accompanying drawings. Throughout, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. It should be understood that the various embodiments of this application described below are merely illustrative of this application and are not intended to limit this application. That is, all other embodiments obtained by those skilled in the art based on the various embodiments of this application without creative effort are within the scope of protection of this application. Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0031] With the aging population, home-based elderly care has become a major social necessity. However, several challenges remain: firstly, the service satisfaction rate for home-based elderly care is low; secondly, it is difficult to effectively obtain service quality evaluations from caregivers, leading to low management efficiency. In recent years, the rapid development of "Internet + information technology" has impacted many industries and also brought opportunities to the elderly care sector. How to utilize "Internet + technology" to create a new service model for "Internet + home-based elderly care" has become a topic of great concern to academia and relevant departments.

[0032] Currently, the main technologies for supervising home-based elderly care services include computer vision-based human behavior recognition technology (such as law enforcement recorder recording at the nursing site, facial recognition, etc.), Bluetooth beacon technology, and GPS / BeiDou positioning technology. Each of these technologies has its own advantages and disadvantages. While law enforcement recorders can record the nursing process, the purchase, deployment, and maintenance costs of the equipment are high, video analysis costs are high, and there is a risk of privacy leakage. On the other hand, technologies such as facial recognition, Bluetooth beacons, and GPS / BeiDou positioning can only locate the location of caregivers and cannot effectively supervise the nursing process.

[0033] However, in actual nursing scenarios, care needs are diverse. Besides routine bed care, it is also common to transfer disabled elderly people to off-bed settings (such as wheelchairs) for care. When providing care in different scenarios, there are challenges in accurately locating and distinguishing the caregiver from the elderly person. Furthermore, developing reasonable care indicators and implementing automated auditing of caregiving behaviors is also a current technical challenge.

[0034] Therefore, there is an urgent need for a nursing behavior recognition method that can effectively address the difficulties in identifying the location of caregivers and the elderly in different scenarios, recognize nursing actions, and effectively monitor the nursing process.

[0035] In summary, this application provides a method for identifying nursing behaviors. For details, please refer to [link / reference needed]. Figure 1 , Figure 1 This is a flowchart illustrating a radar-based nursing behavior recognition method provided in an embodiment of this application. The nursing behavior recognition method includes the following steps 101 to 106.

[0036] Step 101: Detect targets from the collected radar echo data and generate a 2D point cloud of the target based on the target detection results.

[0037] Multi-channel millimeter-wave radar is used to collect spatial electromagnetic wave reflection signals. The echo data is processed for target detection. Then, by analyzing multi-dimensional features such as distance, velocity, and angle in the signal, the position information of effective target points is extracted, thereby generating 2D point cloud data that represents the spatial distribution of the target. This point cloud data contains the target's coordinate information, providing a basic input for subsequent human target recognition.

[0038] In this embodiment of the application, target detection is performed on the collected radar echo data, and a 2D point cloud of the target is generated based on the target detection result. This includes: preprocessing the collected radar echo signal to obtain a multi-dimensional echo signal containing information in the range dimension, velocity dimension, and angle dimension; performing target detection based on the multi-dimensional echo signal; and generating a 2D point cloud of the target based on the target detection result.

[0039] Specifically, a multi-channel millimeter-wave radar continuously transmits electromagnetic waves into the monitoring space and receives echo signals reflected from objects. After signal preprocessing such as signal amplification, mixing, and analog-to-digital conversion, a multi-dimensional echo signal containing information in the range, velocity, and angle dimensions is obtained. Then, power spectrum analysis is performed on the multi-dimensional echo signal, and target detection algorithms such as constant false alarm rate are used to determine the existence of the target and parameters such as range and velocity. For the effective target points detected, angle and range information are obtained, and the range, velocity, and angle information are mapped into two-dimensional coordinates to generate a 2D point cloud representing the spatial distribution of the target.

[0040] In this embodiment, the acquired radar echo signal is preprocessed to obtain a multidimensional echo signal containing range, velocity, and angle dimensions. This includes: preprocessing the acquired raw radar echo signal to obtain a first discrete echo signal y(m,n,k); where m is the slow time dimension, representing the m-th linear frequency modulated continuous wave signal, n is the fast time dimension, representing the n-th sampling point, and k is the antenna dimension, representing the received signal of the k-th channel; performing a fast Fourier transform on the first discrete echo signal in the fast time dimension to obtain a second discrete echo signal y′(m,r,k); where the second discrete echo signal is a range-slow time-antenna dimension signal, r∈[1,N], representing range cell sampling; and performing range clutter suppression and coherent accumulation on the second discrete echo signal to obtain an initial multidimensional echo signal. The final multidimensional echo signal is obtained by performing a Fast Fourier Transform on the initial multidimensional echo signal in the slow time dimension. Among them, the multidimensional echo signal is a range-Doppler-antenna dimensional signal, v∈[1,M], which represents Doppler element sampling.

[0041] Specifically, firstly, the radar continuously transmits electromagnetic wave signals into space. After being reflected by objects, the signals are received by the radar receiver. Then, after being sampled by a signal amplifier, mixer, and ADC, a discrete echo signal containing the distance, velocity, and angle dimensions is obtained, which is the first discrete echo signal.

[0042] The first discrete echo signal can be represented as y(m,n,k), where m is the slow time dimension, representing the m-th linear frequency modulated continuous wave signal; n is the fast time dimension, representing the n-th sampling point; and k is the antenna dimension, representing the received signal of the k-th channel.

[0043] Then, based on the first discrete echo signal, a Fast Fourier Transform is first performed in the fast time dimension to convert the signal from the time domain to the frequency domain, resulting in the second discrete echo signal. The second discrete echo signal, also known as the range-slow time-antenna dimension signal, can be represented as y′(m,r,k), where r∈[1,N], representing range cell sampling.

[0044] Furthermore, range-dimensional clutter suppression and coherent accumulation are performed on the second discrete echo signal. Environmental noise and non-target echo interference are filtered out using methods such as moving average algorithms or time-domain accumulation, resulting in the multidimensional signal after clutter suppression and coherent accumulation, i.e., the initial multidimensional echo signal, which can be expressed as:

[0045] Finally, a Fast Fourier Transform is performed on the initial multidimensional echo signal in the slow time dimension to further extract the target velocity information, resulting in the final multidimensional echo signal, i.e., the range-Doppler-antenna dimension signal, which can be expressed as: Where v∈[1,M], it represents Doppler unit sampling.

[0046] It is understood that the embodiments of this application use multi-channel millimeter-wave radar for second-level compliance audit of nursing behavior. It can be either a single-transmitter multi-receiver radar or a multi-transmitter multi-receiver radar. This application does not make any specific limitation on this.

[0047] In this embodiment of the application, target detection based on the multidimensional echo signal includes: acquiring the Doppler power spectrum of the multidimensional echo signal; extracting peaks from the Doppler power spectrum based on a constant false alarm rate algorithm, and performing target detection on the extracted peak points to obtain target detection results; wherein, the target detection results include valid target points.

[0048] Specifically, the multidimensional echo signal contains information in the range dimension r, velocity dimension v, and antenna dimension k, taking the range-Doppler-antenna dimension signal. The power spectrum of the signal can be used to obtain the Doppler power spectrum. By analyzing the Doppler power spectrum of the signal, the presence of a target within the radar detection range and the target's distance and velocity information can be directly reflected. Then, the peak values ​​in the power spectrum are extracted using the Constant False Alarm Rate (CFAR) algorithm. The CFAR algorithm specifically includes Cell-Averaging Constant False Alarm Rate (CA-CFAR) and Ordered Statistics Constant False Alarm Rate (OS-CFAR). By setting a detection threshold that matches the intensity of environmental clutter, the peak points corresponding to the real target are screened out, noise and interference are eliminated, and valid target points are obtained, thereby determining the existence of the target and its preliminary position and velocity parameters.

[0049] Furthermore, a target 2D point cloud is generated based on the target detection results, including: extracting the range-Doppler index of the effective target points; performing beamforming based on the multidimensional echo signal corresponding to the range-Doppler index to extract the angle index of the effective target points; establishing a spatial coordinate system based on the radar installation method, and obtaining the 2D coordinates of the effective target points based on the range-Doppler index and the angle index to generate the target 2D point cloud.

[0050] Specifically, firstly, the range-Doppler (v,r) index of the effective target point is extracted. This index corresponds to the range and velocity characteristics of the target in the multidimensional echo signal. Then, the index corresponding to the above index is taken... The antenna-dimensional signal is used for beamforming, and the peak power spectrum energy index after beamforming is taken as the angle index corresponding to the effective points. Thus, the distance, velocity, angle, and energy information of each effective point of the target are known, enabling precise target location. Commonly used beamforming methods include Digital Beamforming (DBF), Multiple Signal Classification (MUSIC), and Minimum Variance Distortionless Response (MVDR) algorithm (also known as the Capon algorithm). Finally, a spatial coordinate system is established according to the millimeter-wave radar installation method (the spatial coordinate system varies depending on the installation method; common installation methods include side mounting at 45°, side mounting, top mounting, etc.). In the spatial coordinate system, the coordinates (x, y, y) of all effective points of the target can be obtained using the distance and angle information. i ,y i ), that is, the coordinates of all point cloud imaging points. All coordinate points constitute the target 2D point cloud, which intuitively represents the spatial distribution of the human body within the radar detection range.

[0051] Step 102: Perform clustering processing based on the target 2D point cloud, and determine the location information of the person being cared for based on the clustered target 2D point cloud.

[0052] After acquiring the target 2D point cloud, clustering is first performed on the target 2D point cloud. The purpose of clustering is to aggregate the discrete target 2D point cloud into several point cloud clusters with clear spatial affiliation based on the spatial distribution density, distance correlation, and other characteristics of the point cloud, thereby initially distinguishing different targets in the nursing scene (such as the cared-for personnel, nursing personnel, or environmental interference points), and obtaining the clustered target 2D point cloud.

[0053] When determining the location information of caregivers based on clustered target 2D point clouds, the main approach leverages the relatively stable location and low movement frequency of caregivers (such as the elderly) within the care scenario. This is combined with information such as the spatial persistence and distribution stability of each clustered point cloud, selecting clusters that match the caregiver's behavioral characteristics. The core location of this cluster, such as the cluster centroid or the concentrated area of ​​point cloud distribution within the cluster, is then used as the caregiver's location information. The entire process does not rely on pre-defined scenario data, such as pre-marked bed locations; the location of caregivers can be determined solely through the spatial and behavioral characteristics of the clustered target 2D point clouds themselves.

[0054] In this embodiment of the application, clustering processing is performed on the target 2D point cloud, and the location information of the person being cared for is determined based on the clustered target 2D point cloud. This includes: performing clustering processing on the target 2D point cloud to obtain a clustered target 2D point cloud; processing the clustered target 2D point cloud based on a people prediction algorithm to obtain the number of human targets in the target 2D point cloud; updating a heat map based on the clustered target 2D point cloud and the number of human targets; wherein, the heat map is a numerical matrix determined based on the radar detection range and the resolution of the heat map; and determining the location information of the person being cared for based on the heat map.

[0055] Specifically, after the point cloud data is generated, the radar point cloud needs to be clustered. Clustering algorithms include, but are not limited to, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), as well as K-means, Divisive Clustering, etc. Based on the spatial distribution characteristics of the point cloud, adjacent points are aggregated into clusters, thereby separating point cloud sets of different targets. In detail, taking the DBSCAN clustering algorithm as an example, clustering can be divided into three steps: 1) Find the number of samples in the neighborhood of each sample. If the number is greater than or equal to the set minimum number, then the sample is a core point; 2) Find the samples that are directly reachable and density-reachable from each core sample, and these samples are also core samples. All non-core samples are temporarily ignored; 3) If a non-core sample is within the -neighborhood of a core sample, then the non-core sample is a boundary sample; otherwise, it is noise.

[0056] A people prediction algorithm is used to analyze the clustered 2D point cloud of the target. The core of the algorithm is to identify whether each cluster of point clouds conforms to the characteristics of human targets, and based on this, to count the actual number of human targets in the 2D point cloud. Obtaining this number of human targets can clarify the current personnel composition in the nursing scene, such as whether only patients are being cared for, or whether both patients and caregivers are present.

[0057] First, let's clarify the essence of a heatmap: it is a numerical matrix constructed based on the radar detection range and a preset heatmap resolution. Each numerical unit in the matrix corresponds to a sub-region within the radar detection space, and the numerical value is used to reflect the frequency and persistence of point cloud occurrences within that sub-region.

[0058] When updating the heatmap, clustered 2D point clouds of targets are used as the data source. The clustered point clouds determine the specific sub-region location of each target in the radar detection space in the current frame, and the values ​​of the corresponding units in the numerical matrix are updated accordingly. At the same time, the update strategy of the heatmap is adjusted based on the previously acquired number of human targets, ultimately completing the real-time update of the heatmap. By combining the clustered point cloud and the number of human targets, it can be ensured that the heatmap accurately reflects the spatial distribution and positional stability of human targets in the nursing scenario.

[0059] Because the location of individuals being cared for (such as the elderly) in nursing settings is typically relatively stable, their corresponding point cloud will continuously appear within specific sub-regions of the heatmap, causing the values ​​in these sub-regions to be significantly higher than those in other regions. Therefore, by analyzing the updated heatmap, we can identify the heatmap high points with the highest values ​​or those that meet the characteristics of location stability. These regions represent the core activity areas of the individuals being cared for. The center of these regions can then be used as the location information for the individuals being cared for, enabling accurate determination of their location.

[0060] In this embodiment of the application, the clustered target 2D point cloud is processed based on a people prediction algorithm to obtain the number of human targets in the target 2D point cloud. This includes: processing the clustered target 2D point cloud based on a first people prediction algorithm model to determine whether the number of human targets in the target 2D point cloud is 0; when the number of human targets is not 0, performing feature extraction on the clustered target 2D point cloud and generating a feature vector based on the feature values ​​obtained from the feature extraction; and processing the feature vector based on a second people prediction algorithm model to obtain whether the number of human targets in the target point cloud is 1 or 2.

[0061] Specifically, in nursing scenarios, due to the complexity of nursing behaviors, the appearance of point clouds is also very diverse. Sometimes the point cloud of caregivers is mixed with the point cloud of the elderly, sometimes they are separate, and there may also be interfering point clouds. It is difficult to distinguish them using traditional methods such as clustering algorithms. This invention will combine a trained model to assist in the prediction of the number of people. The specific steps are as follows: The number prediction algorithm judges the number of people (0, 1, 2) based on the clustered point cloud information. First, the number of people in the clustered point cloud is detected. The initial state is no one (0 people). If the number of point clouds meets N_1 for T_1 consecutive seconds, the state changes from no one to someone. If the number of point clouds meets N_2 for T_2 consecutive seconds, the state changes from someone to no one.

[0062] If someone is identified as being present, the system further determines whether there are 1 or 2 people. Here, feature extraction is performed on the point cloud clustering results to input the 1 / 2 person judgment model. The specific steps for point cloud feature extraction based on the clustering results are as follows: First, select the two clusters with the largest number of points in the clustering results. Then, for each cluster, use nine features as feature values: number of points within the cluster, cluster area, cluster density, cluster aspect ratio, cluster dispersion, proportion of high-energy points, area of ​​high-energy points, density of high-energy points, and cluster centroid. The combination of the total number of point cloud clusters and the feature values ​​of the two clusters is called the feature vector. If a cluster does not exist (when the number of clusters is less than or equal to 1), then all feature values ​​of that cluster are set to zero.

[0063] This invention trains a 1-person / 2-person classification model based on point cloud features of a known dataset. Commonly used classification models include Support Vector Machine (SVM), Random Forest, k-Nearest Neighbors (k-NN), Convolutional Neural Network (CNN), etc. This model can determine the number of people based on the aforementioned point cloud features. To prevent accidental misjudgments, the judgment results are smoothed. That is, if the model consistently judges that there is someone for a period of time, the initial state is 1 person. If the number of point cloud points meets N_3 for T_3 consecutive seconds, the state changes from 1 person to 2 people. If the number of point cloud points meets N_4 for T_4 consecutive seconds, the state changes from 1 person to 2 people.

[0064] In the embodiments of this application, the feature values ​​include the number of points within a cluster, cluster area, cluster density, cluster aspect ratio, cluster dispersion degree, high energy percentage, high energy area, high energy density, and cluster centroid;

[0065] After clustering, the two clusters with the largest number of point clouds are selected. For each cluster, multiple feature values ​​are extracted, including the number of points within the cluster, cluster area, cluster density, cluster aspect ratio, cluster dispersion, proportion of high-energy points, area of ​​high-energy points, density of high-energy points, and cluster centroid. If the actual number of clusters in the clustering result is less than or equal to 1 (i.e., only one cluster exists or there are no clusters), all feature values ​​of the missing cluster are set to zero to maintain consistency of feature dimensions. Finally, the total number of clusters obtained from clustering is combined with the feature values ​​of the two clusters to form a feature vector. Here, if we assume C... j The point set of the j-th cluster can be represented as C j ={(x1,y1,p1),(x2,y2,p2),…,(x n ,y n ,p n )}, where (x i ,y i) represents the coordinates of the i-th point within the cluster, p i represents the energy value of this point. This feature vector integrates the structural features and energy features of the point cloud distribution, providing key input parameters for the subsequent human in-bed state judgment model.

[0066] Specifically, the eigenvalue of the cluster class includes at least nine types: the number of points within the cluster, the cluster area, the cluster density, the cluster aspect ratio, the degree of cluster dispersion, the proportion of large energy, the large energy area, the large energy density, and the cluster centroid.

[0067] Feature 1: The number of points within the cluster: [[ID=X]]

[0068] Feature j,1 = n

[0069] Feature 2: The cluster area:

[0070] Let the vertex sequence formed by the points within the cluster (arranged counterclockwise) be:

[0071] [[ID=2X]]

[0072] where, h k ∈ {1, 2,..., n}, m < n, then the area of the polygon enclosed by the vertex sequence is the cluster area:

[0073]

[0074] Feature 3: The cluster density:

[0075]

[0076] Feature 4: The cluster aspect ratio:

[0077]

[0078] where, represents the largest element x in the C j set, i , represents the smallest element x in the C j set, max(W i , H j ) represents the larger value of W j , H j , min(W j , H j ) represents the smaller value of W j , H j , H j in.

[0079] Feature 5: The degree of cluster dispersion:

[0080] The centroid of the cluster can be expressed as: Note: There seem to be some inconsistent or unclear notations in the original text (such as the repeated use of some tags without clear meaning and the "X" in the middle which might be a mistake). The translation is done as accurately as possible based on the provided text.

[0081]

[0082] The degree of cluster dispersion can be expressed as the average distance from a point to the cluster centroid:

[0083]

[0084] in, Representing point (x) i ,y i The distance from the center of mass.

[0085] Feature 6: High energy percentage

[0086] The average power at all points is:

[0087]

[0088] Among them, points above TH are high-energy points, therefore H j ={C j |p i >TH} is the set of indices for high-power points within the cluster. The number of high-power points within the cluster is . therefore

[0089]

[0090] Feature 7: Large energy area:

[0091] Suppose that the vertex sequence (arranged counterclockwise) formed by the points within the high-energy point cluster is as follows:

[0092]

[0093] in, Therefore, the area of ​​the polygon enclosed by the vertex sequence is the large-energy area:

[0094]

[0095] Feature 8: High energy density

[0096] Feature j,8 =Feature j,7 / Feature j,6

[0097] Feature 9: Cluster centroid:

[0098]

[0099] In this embodiment of the application, updating the heatmap based on the clustered target 2D point cloud and the number of human targets includes: if the current frame is the first frame of point cloud, then initializing the overall heatmap R. mapInitialize the current heatmap R for pre-update. cur ; Calculate the R value of all point clouds in the current frame cur The position is determined, and the corresponding unit is set to 1; if the current frame is the first frame point cloud, the previous frame heatmap R is initialized. last ; Calculate the degree of change T of the heatmap between the current frame and the previous frame. pc The degree of change in the heat map T pc The calculation expression is:

[0100]

[0101] Where l represents the radar's detection range, and r represents the resolution of the heatmap; the current heatmap R cur Save as the heatmap of the previous frame corresponding to the next frame R last ; For the overall heat map R map The overall heatmap calculation expression is updated as follows:

[0102] R map =αR map +ωR cur ,

[0103] Where α (0 < α < 1) is the heatmap update factor; the closer α is to 1, the slower the heatmap updates over time; ω is the weight of the current heatmap, determined by the degree of change T of the heatmap. pc The result, along with the number of human targets n, determines its expression:

[0104]

[0105] Among them, A1, B1, A2, and B2 are constant terms that are greater than 0.

[0106] Specifically, in nursing scenarios, determining the location of elderly individuals solely based on point cloud information is challenging due to a lack of prior knowledge. This invention leverages the relatively stable and infrequent movement of elderly individuals in nursing settings, combining this with a heatmap updated in real-time based on point cloud information to pinpoint their location. The specific steps are as follows: When using the heatmap to determine the location of elderly individuals, it requires both information on the number of people and clustered point cloud information. First, an l×r numerical matrix R is defined. map The values ​​of l and r depend on the radar's detection range and the required resolution (which can be adjusted as needed), and are initially set to 0. This matrix is ​​the initial heatmap. Then, after each point cloud data occurrence, this numerical matrix is ​​updated; the numerical matrix R... map The update rules are as follows:

[0107] 1) Initialize the overall heat map R map (Execute only when the first frame of the point cloud arrives)

[0108] 2) Initialize the current heatmap R for pre-update. cur ;

[0109] 3) Calculate the R value of all point clouds in the current frame. cur The position is determined, and the corresponding unit is set to 1;

[0110] 4) Initialize the previous current heatmap R last (Execute only when the first frame of the point cloud arrives)

[0111] 5) Calculate the degree of change T of the heatmap between the current frame and the previous frame. pc ;

[0112]

[0113] 6) Save the current heatmap R cur Used for updating the heatmap in the next frame, i.e.:

[0114] R last =R cur ;

[0115] 7) Update the overall heatmap R map ;

[0116] R map =αR map +ωR cur ,

[0117] Where 0 < α < 1 is the heatmap update factor, and the closer the value of α is to 1, the slower the heatmap updates over time; ω is the weight of the current heatmap, determined by T. pc Together with the current number of people, it determines:

[0118]

[0119] Among them, A1, B1, A2, and B2 are constant terms greater than 0, which means that the heat map with a small change when the number of people is 1 has a higher weight, and the heat map with a small change when the number of people is 2 has a higher weight.

[0120] 8) Repeat steps 2), 4), 5), 6), and 7).

[0121] If the number of people in the current frame is not 0, then there are elderly people in the environment by default, and the location of the elderly person is the index corresponding to the maximum value in the heatmap.

[0122] Step 103: Based on the location information of the people being cared for, perform point cloud allocation on the clustered target 2D point cloud to obtain the point cloud of the people being cared for.

[0123] After determining the location information of the person being cared for, point cloud allocation is performed on the clustered target 2D point cloud using this location information as the core reference. The core logic of point cloud allocation is as follows: based on the spatial correlation between each cluster of point clouds and the location of the person being cared for, each cluster is divided into two categories: point clouds belonging to the person being cared for and point clouds not belonging to the person being cared for. Among them, point cloud clusters with a strong spatial correlation with the location of the person being cared for, such as point cloud clusters within a specific spatial range centered on the location of the person being cared for, are determined to belong to the person being cared for; the remaining point cloud clusters with a weaker spatial correlation with the location of the person being cared for are determined to belong to the caregiver, thus completing the point cloud allocation and ultimately obtaining the caregiver's point cloud.

[0124] Furthermore, in this embodiment, point cloud allocation is performed on the clustered target 2D point cloud based on the location information of the person being cared for to obtain a caregiver point cloud. This includes: allocating the clustered target 2D point cloud based on the location information of the person being cared for and the number of human targets; when the number of human targets is 0, no point cloud allocation is performed; when the number of human targets is 1, the point cloud within a preset radius of the location of the person being cared for is allocated to the person being cared for, and no point cloud is allocated to the caregiver; when the number of human targets is 2, the clustered target 2D point cloud includes a corresponding first cluster and a second cluster, and the number of point clouds contained in the first cluster is greater than the number of point clouds contained in the second cluster; if the first cluster... If the point cloud area of ​​a cluster is greater than a preset area or the number of point clouds in the second cluster is less than a preset number, then the point cloud of the first cluster, which is within a preset radius of the location of the person being cared for, is assigned to the person being cared for, and the remaining point cloud of the first cluster is assigned to the caregiver. If the point cloud area of ​​the first cluster is less than a preset area and the number of point clouds in the second cluster is greater than a preset number, then the first distance between the first cluster and the location of the person being cared for and the second distance between the second cluster and the location of the person being cared for are calculated. The first distance and the second distance are compared, and based on the comparison result, the cluster with a smaller distance to the location of the person being cared for is assigned to the person being cared for, and the cluster with a larger distance to the location of the person being cared for is assigned to the caregiver.

[0125] Specifically, after clustering, the point cloud will be further processed based on the number of people identified. However, in mobile care scenarios, the diverse forms of caregiving lead to complex point cloud distributions, making it difficult for traditional clustering algorithms to effectively distinguish between the point clouds of elderly people and caregivers. To address this issue, this invention allocates point clouds based on the prior results of the number of people identified and the location of the elderly. The specific steps are as follows: if the current number of people is 0, then no point cloud will be allocated; if the current number of people is 1, then point clouds within a radius d near the elderly person's location will be allocated to the elderly person, and no point clouds will be allocated to caregivers; if the current number of people is 2, the point clouds will first be sorted according to the number of points in the clustering results (the number of point clouds is divided into first cluster, second cluster, etc. from largest to smallest), and then processed in two ways: the first case is when the point clouds of the elderly and caregivers are severely mixed, and the characteristic of the point cloud is that the area occupied by the number of points in the first cluster is greater than T. s Or the number of points in the second cluster is less than N s In the first case, the point cloud within a radius d near the elderly person's location in the first cluster will be assigned to the elderly person, and the remaining point cloud will be assigned to the caregiver. In the second case, the point clouds of the elderly person and the caregiver can be separated by the clustering algorithm. In this case, the cluster closer to the elderly person's location will be assigned to the elderly person, and the other cluster will be assigned to the caregiver.

[0126] Step 104: Track the trajectory of the nursing staff point cloud to obtain the trajectory coordinate sequence.

[0127] For the identified nursing staff, the continuous tracking of the target's movement trajectory is achieved by associating point cloud data over time, and a trajectory coordinate sequence containing timestamps is obtained. This trajectory sequence records the target's movement path in space, and can be used to extract nursing behavior features in subsequent steps to provide data support.

[0128] Specifically, after obtaining the clustering results in the aforementioned steps, the clusters are assigned using a clustering assignment algorithm, and then a trajectory tracking algorithm is used for trajectory tracking, including trajectory creation, trajectory updating, and trajectory deletion. Commonly used clustering assignment algorithms include, but are not limited to, the Munkres' Assignment Algorithm and the Joint Probabilistic Data Association algorithm, used to assign point cloud clustering results to different trajectories. Commonly used trajectory tracking algorithms include, but are not limited to, Stanley Control, Model Predictive Control (MPC), Extended Kalman Filter (EKF), and Particle Filter algorithms. Finally, based on the tracking results, a decision is made on whether to create, update, or delete a trajectory.

[0129] Step 105: Extract nursing behavior feature parameters based on trajectory coordinate sequence, and determine the compliance of nursing behavior based on nursing behavior feature parameters.

[0130] Specifically, based on the target trajectory coordinate sequence, feature parameters representing nursing behavior are extracted, and then the feature parameters are fused and analyzed through preset judgment rules. After comparison with the compliance threshold, the compliance judgment result of nursing behavior is generated.

[0131] In this embodiment, the parameters include nursing displacement parameters, nursing goal existence parameters, nursing gross motion parameters, nursing area continuity parameters, nursing gross motion continuity parameters, and nursing effectiveness behavior matching parameters; nursing behavior feature parameters are extracted based on trajectory coordinate sequences, including: obtaining the trajectory coordinate sequence corresponding to the nursing staff; setting a sliding time window to continuously record the maximum displacement within each time window; wherein, the maximum displacement D max The expression is:

[0132] D max =max t＜i＜j＜t+Twin D ij ,

[0133] Where t is the starting point of the time window, Twin is the length of the sliding time window, i and j are any two points in the trajectory coordinate sequence within the sliding time window, and D is the starting point of the time window. ij D represents the displacement between i and j. ij The expression is:

[0134]

[0135] The nursing displacement parameter σ1 is extracted based on the maximum displacement; the expression for the nursing displacement parameter σ1 is:

[0136]

[0137] Where N is the duration of the nursing action, Twin is the sliding time window length, u is the step function, and Dthre is the preset distance threshold; the number of targets in the radar detection space is obtained based on the trajectory coordinate sequence, and the nursing target existence parameter σ2 is extracted; the expression for the nursing target existence parameter σ2 is:

[0138]

[0139] Where Pt is the number of targets in the radar detection space at time t; range compensation is performed on the radar echo signal based on the compensation curve to obtain the compensated radar echo signal; where the compensation curve is represented as the sequence cpl=[1,c1,…c N-1], c represents the signal value of each range unit in the sequence, and N represents the maximum value of the range unit; the body motion index is obtained based on the compensated radar echo signal, and the nursing gross motion parameter σ3 is extracted; where, the expression of the nursing gross motion parameter σ3 is:

[0140]

[0141] Among them, B t Let t be the body movement index, and Bthre be the preset body movement index threshold. When there are two targets in the radar detection space, the first preset algorithm is used to determine whether the nursing actions of the targets within the preset time threshold meet the regional continuity condition, and the nursing regional continuity parameter σ4 is extracted based on the judgment result. The second preset algorithm is used to determine whether the body movement index of the targets within the preset time threshold is greater than the preset body movement threshold, and the nursing large movement continuity parameter σ5 is obtained by accumulating the judgment results. The third preset algorithm is used to determine whether the nursing actions of the targets within the preset time threshold meet the effectiveness condition, and the nursing effectiveness behavior matching parameter σ6 is extracted based on the judgment result.

[0142] Specifically:

[0143] (1) Nursing large displacement parameters:

[0144] Obtain the trajectory coordinate sequence corresponding to the human target outside the boundary. Using a sliding time window (usually several seconds to tens of seconds) and the point cloud sampling time as the step time, calculate the maximum displacement for each time window. Use the number of maximum displacements throughout the entire nursing process as the displacement parameter. The N coordinate values ​​of the target trajectory outside the bed during the nursing process can be represented as follows:

[0145] P={(x t ,y t )|t∈(1,N)},

[0146] Assuming the sliding time window length is Twin, then the maximum displacement D max The expression is:

[0147] D max =max t＜i＜j＜t+Twin D ij ,

[0148] Where t is the starting point of the time window, i and j are any two points in the trajectory coordinate sequence within the sliding time window, and D ij D represents the displacement between i and j. ij The expression is:

[0149]

[0150] The nursing displacement parameter σ1 is extracted based on the maximum displacement; the expression for the nursing displacement parameter σ1 is:

[0151]

[0152] Where N is the duration of the nursing action, Twin is the length of the sliding time window, u is the step function, and Dthre is the preset distance threshold, which is generally set to tens to hundreds of centimeters.

[0153] (2) Nursing goals have parameters:

[0154] Trajectory tracking can identify the number of targets within the radar detection space, while nursing parameter extraction involves statistically analyzing the number of targets within the radar detection space. Let Pt be the number of targets monitored by the radar at time t, then the nursing parameter σ² can be expressed by the following formula:

[0155]

[0156] Where N is the duration of the nursing action, and u is the step function.

[0157] (3) Nursing gross motor parameters:

[0158] To address the issue of energy attenuation with increasing distance in millimeter-wave radar, resulting in lower body motion values ​​for the same action at greater ranges, this invention performs distance compensation on the energy value before calculating the body motion index. Specifically, this is achieved by experimentally fitting a compensation curve to dynamic objects (such as fans) at different locations in the environment and then normalizing it. This compensation curve can be viewed as a sequence cpl = [1, c1, ... c N-1 ], where N represents the maximum value of the distance unit, and the signal of each distance unit will be divided by the corresponding number in the compensation sequence.

[0159] The motion index is obtained based on radar echo signals. It is used to calculate the intensity of motion of moving targets within the radar detection range at each moment. Common methods for obtaining the motion index are divided into time-domain analysis and frequency-domain analysis. Time-domain analysis includes amplitude threshold detection, energy spectrum analysis, integral analysis, root mean square error analysis, and optimized combinations of these methods. Frequency-domain analysis primarily characterizes the motion index by the ratio of the number of spectral energy values ​​exceeding a set threshold to the total number of signals. The nursing macro-motion parameter is determined by statistically analyzing the motion index exceeding the set threshold. Let B be the motion index at time t. t The extraction of the body motion parameter σ3 can be expressed by the following formula:

[0160]

[0161] Among them, B t Let t be the body movement index, and Bthre be the preset body movement index threshold, which is generally set to 30-50.

[0162] (4) Nursing area continuity parameters

[0163] During the nursing process, a caregiver should maintain a nursing action within a specific area for a certain period of time. Therefore, the caregiver's trajectory is tracked over a certain time period to determine if the trajectory falls within a certain area. If this condition is met, the current nursing action is considered continuous; otherwise, it is considered discontinuous. The sequence of continuous states per second during the nursing process is accumulated using a sliding window and its validity is determined: 1 for valid and 0 for invalid. The results satisfying continuous states are summed to obtain the nursing area continuity parameter σ4.

[0164] (5) Nursing major movement continuity parameters

[0165] This is an extension of the continuity parameter for nursing areas. Besides maintaining a position in one area for a certain period, nursing actions also involve significant movements, requiring the body movement value during this time to exceed a certain threshold. Therefore, the system checks if the nursing process is continuous every second; if so, it calculates whether the current body movement value exceeds the set body movement threshold T. cm If so, it means that the moment is in the midst of a major nursing action, and the results of the major nursing action will be accumulated to obtain the continuity parameter σ5 of the major nursing action.

[0166] (6) Nursing effectiveness behavior matching parameters

[0167] During the care process, there is a strong interaction between the caregiver and the elderly. The point clouds of the two people can easily converge and exhibit high-energy properties. Therefore, this parameter is calculated using the point cloud heatmap. Similarly, the current heatmap R in (5) will be calculated in the same way here. cur Compared to the previous current heatmap R last Unlike (5), the overall heat map R map The update rules are different for R map In each update of R cur No weights need to be configured.

[0168] R map =αR map +R cur

[0169] After obtaining the overall heatmap, the values ​​greater than the threshold T6 in each frame of the heatmap are clustered. The area and sum of the clusters with the largest clustering results are calculated. If the area and sum of the clusters are greater than the thresholds S6 and P6, then the frame is considered a valid nursing action. Finally, the sum of all frames that meet the conditions is the nursing effectiveness action matching parameter σ6.

[0170] In this embodiment, a nursing behavior score is obtained by fusing nursing displacement parameters, nursing goal existence parameters, nursing gross motor parameters, nursing area continuity parameters, nursing gross motor continuity parameters, and nursing effectiveness behavior matching parameters based on a nursing score model and preset weights; wherein, the nursing score model is defined as:

[0171]

[0172] Where x represents the value of each nursing parameter, t is the threshold, and m is the theoretical upper limit of each nursing parameter; the expression for the nursing behavior score is:

[0173] score=(w1σ1+w2σ2+w3σ3+w4σ4+w5σ5+w6σ6),

[0174] Among them, w1, w2, w3, w4, w5, and w6 are preset weight parameters, and w1+w2+w3+w4+w5+w6=1;

[0175] The compliance of nursing behaviors is determined based on nursing behavior scores.

[0176] Specifically, each nursing parameter has a corresponding score, ranging from 0 to 10. The value of each nursing parameter is x, the threshold is t, and the theoretical upper limit is m. Therefore, the scoring model for each nursing parameter is defined as follows:

[0177]

[0178] The scoring model adopts a tiered scoring approach. For example, the score will only be greater than 6 points if the nursing parameter value x is greater than the threshold value t. To make the nursing scoring model more comprehensive and reasonable, the value x should be considered in conjunction with the absolute duration and the proportion. The absolute duration corresponds to the specific value of each indicator, and the proportion corresponds to the ratio of the specific value of each indicator to the total nursing duration. Therefore, each nursing parameter has a corresponding theoretical upper limit and threshold. The theoretical upper limit is further divided into the theoretical upper limit of proportion and the theoretical upper limit of absolute duration, and the threshold is further divided into the proportional threshold and the absolute duration threshold.

[0179] Finally, different weights are assigned to the six parameters, and the total score is calculated, denoted as 'score', as follows:

[0180] score=(w1σ1+w2σ2+w3σ3+w4σ4+w5σ5+w6σ6),

[0181] Among them, w1, w2, w3, w4, w5, and w6 are preset weight parameters, and w1+w2+w3+w4+w5+w6=1. The specific weight values ​​of these six parameters are determined by training and learning from actual data samples. The score result can be used as the basis for whether the work order is approved in the end, and thre is set as the threshold score for whether the work order is approved.

[0182] Step 106: Generate a nursing behavior compliance audit report based on the nursing behavior compliance determination results.

[0183] Specifically, based on the compliance assessment results of nursing behaviors, an audit report is generated that includes the target trajectory, statistical analysis of characteristic parameters, and compliance conclusions. This report can be used for nursing service quality assessment and work order auditing, providing data support for elderly care service management.

[0184] As can be seen from the above, the embodiments of this application first perform target detection on the collected radar echo data, and generate a target 2D point cloud based on the target detection results. Then, clustering processing is performed on the target 2D point cloud, and the location information of the cared-for personnel is determined based on the clustered target 2D point cloud. Then, point cloud allocation is performed on the clustered target 2D point cloud based on the location information of the cared-for personnel to obtain the nursing personnel point cloud. Then, trajectory tracking is performed on the nursing personnel point cloud to obtain the trajectory coordinate sequence. Next, nursing behavior feature parameters are extracted based on the trajectory coordinate sequence, and nursing behavior compliance is judged based on the nursing behavior feature parameters. Finally, a nursing behavior compliance audit report is generated based on the nursing behavior compliance judgment result. The nursing behavior recognition method proposed in this invention can locate the elderly and caregivers without requiring any prior location information, supporting the auditing of nursing behaviors in different scenarios and greatly increasing the versatility of this recognition method. This method does not require the collection of actual image data, enabling the identification of personnel in a scene without infringing on privacy, and simultaneously identifying nursing behaviors. It can monitor the entire nursing process, effectively improving the effectiveness of nursing process supervision. Furthermore, it can effectively judge the compliance of nursing behaviors, has up to six nursing parameter indicators, and provides audit reports, effectively reducing the time cost of auditing nursing work orders. In addition, compared with existing long-term care insurance auditing methods, the equipment deployment and maintenance costs of this solution are low.

[0185] It should be understood that the sequence number of each step in this embodiment does not imply the order in which the steps are executed. The execution order of each step should be determined by its function and internal logic, and should not constitute a unique limitation on the implementation process of this application embodiment.

[0186] In summary, the detailed process of the nursing behavior recognition method involved in the embodiments of this application can be found in [reference needed]. Figure 2 Specifically:

[0187] Step 201: Perform signal preprocessing on the collected radar echo signal to obtain a multidimensional echo signal containing information in the range dimension, velocity dimension, and angle dimension;

[0188] Step 202: Target detection is performed based on multidimensional echo signals, and a 2D point cloud of the target is generated based on the target detection results;

[0189] Step 203: Perform clustering processing on the target 2D point cloud to obtain the clustered target 2D point cloud;

[0190] Step 204: Process the clustered target 2D point cloud based on the people prediction algorithm to obtain the number of human targets in the target 2D point cloud;

[0191] Step 205: Update the heat map based on the clustered target 2D point cloud and the number of human targets, and determine the location information of the person being cared for based on the heat map;

[0192] Step 206: Based on the location information of the people being cared for, perform point cloud allocation on the clustered target 2D point cloud to obtain the point cloud of the people being cared for;

[0193] Step 207: Track the identified human target to obtain a trajectory coordinate sequence;

[0194] Step 208: Extract nursing large displacement parameters, nursing goal existence parameters, nursing large movement parameters, nursing area continuity parameters, nursing large movement continuity parameters, and nursing effectiveness behavior matching parameters based on trajectory coordinate sequence;

[0195] Step 209: Based on preset weights, the nursing large displacement parameters, nursing goal existence parameters, nursing large movement parameters, nursing area continuity parameters, nursing large movement continuity parameters, and nursing effectiveness behavior matching parameters are fused and judged to obtain the nursing behavior score;

[0196] Step 210: Determine the compliance of nursing behaviors based on nursing behavior scores, and generate a nursing behavior compliance audit report based on the nursing behavior compliance determination results.

[0197] For a more detailed process of each step in steps 201 to 210, please refer to the relevant sections shown above. The embodiments of this application will not be repeated here.

[0198] Please see Figure 3 , Figure 3 A radar-based nursing behavior recognition system is provided for embodiments of this application. This system can be used to implement the nursing behavior recognition method involved in the embodiments of this application. The radar-based nursing behavior recognition system 300 mainly includes:

[0199] The point cloud generation module 301 is used to perform target detection on the collected radar echo data and generate a 2D point cloud of the target based on the target detection results.

[0200] The location acquisition module 302 is used to perform clustering processing based on the target 2D point cloud, and determine the location information of the person being cared for based on the clustered target 2D point cloud;

[0201] The point cloud allocation module 303 is used to allocate the clustered target 2D point cloud based on the location information of the person being cared for, so as to obtain the point cloud of the person being cared for.

[0202] The trajectory tracking module 304 is used to track the trajectory of the nursing staff point cloud to obtain the trajectory coordinate sequence.

[0203] The behavior determination module 305 is used to extract nursing behavior feature parameters based on the trajectory coordinate sequence and to determine the compliance of nursing behavior based on the nursing behavior feature parameters.

[0204] The report generation module 306 is used to generate a nursing behavior compliance audit report based on the nursing behavior compliance determination results.

[0205] In some embodiments of this example, when the point cloud generation module 301 performs the function of detecting targets from collected radar echo data and generating a 2D point cloud of the target based on the target detection results, it is used to: perform signal preprocessing on the collected radar echo signals to obtain multidimensional echo signals containing range, velocity, and angle information; perform target detection based on the multidimensional echo signals; and generate a 2D point cloud of the target based on the target detection results.

[0206] In some embodiments of this example, when the point cloud generation module 301 performs the function of preprocessing the acquired radar echo signal to obtain a multidimensional echo signal containing range, velocity, and angle dimensions, it is used to: preprocess the acquired original radar echo signal to obtain a first discrete echo signal y(m,n,k); where m is the slow time dimension, representing the m-th linear frequency modulated continuous wave signal, n is the fast time dimension, representing the n-th sampling point, and k is the antenna dimension, representing the received signal of the k-th channel; perform a fast Fourier transform on the first discrete echo signal in the fast time dimension to obtain a second discrete echo signal y′(m,r,k); where the second discrete echo signal is a range-slow time-antenna dimension signal, r∈[1,N], representing range cell sampling; and perform range clutter suppression and coherent accumulation on the second discrete echo signal to obtain an initial multidimensional echo signal. The final multidimensional echo signal is obtained by performing a Fast Fourier Transform on the initial multidimensional echo signal in the slow time dimension. Among them, the multidimensional echo signal is a range-Doppler-antenna dimensional signal, v∈[1,M], which represents Doppler element sampling.

[0207] In some embodiments of this example, when the point cloud generation module 301 performs the function of target detection based on the multidimensional echo signal, it is used to: acquire the Doppler power spectrum of the multidimensional echo signal; extract the peaks of the Doppler power spectrum based on the constant false alarm rate algorithm, and perform target detection on the extracted peak points to obtain the target detection result; wherein, the target detection result includes valid target points.

[0208] In some embodiments of this example, when the point cloud generation module 301 performs the function of generating a target 2D point cloud based on the target detection result, it is used to: extract the range-Doppler index of the effective target point; perform beamforming based on the multidimensional echo signal corresponding to the range-Doppler index, and extract the angle index of the effective target point; establish a spatial coordinate system based on the radar installation method, and obtain the 2D coordinates of the effective target point based on the range-Doppler index and the angle index, thereby generating a target 2D point cloud.

[0209] In some embodiments of this example, when the location acquisition module 302 performs the function of clustering based on the target 2D point cloud and determining the location information of the person being cared for based on the clustered target 2D point cloud, it is used to: perform clustering processing on the target 2D point cloud to obtain the clustered target 2D point cloud; process the clustered target 2D point cloud based on the number prediction algorithm to obtain the number of human targets in the target 2D point cloud; update the heat map based on the clustered target 2D point cloud and the number of human targets; wherein, the heat map is a numerical matrix determined based on the radar detection range and the resolution of the heat map; and determine the location information of the person being cared for based on the heat map.

[0210] Furthermore, in some embodiments of this example, when the location acquisition module 302 performs the function of processing the clustered target 2D point cloud based on the people prediction algorithm to obtain the number of human targets in the target 2D point cloud, it is specifically used to: process the clustered target 2D point cloud based on the first people prediction algorithm model to determine whether the number of human targets in the target 2D point cloud is 0; when the number of human targets is not 0, perform feature extraction on the clustered target 2D point cloud and generate a feature vector based on the feature values ​​obtained from the feature extraction; process the feature vector based on the second people prediction algorithm model to obtain that the number of human targets in the target point cloud is 1 or 2.

[0211] Furthermore, in some embodiments of this example, when the location acquisition module 302 performs the function of updating the heatmap based on the clustered target 2D point cloud and the number of human targets, it is specifically used to: if the current frame is the first frame of point cloud, initialize the overall heatmap R. mapInitialize the current heatmap R for pre-update. cur ; Calculate the R value of all point clouds in the current frame cur The position is determined, and the corresponding unit is set to 1; if the current frame is the first frame point cloud, the previous frame heatmap R is initialized. last ; Calculate the degree of change T of the heatmap between the current frame and the previous frame. pc The degree of change in the heat map T pc The calculation expression is:

[0212]

[0213] Where l represents the radar's detection range, and r represents the resolution of the heatmap; the current heatmap R cur Save as the heatmap of the previous frame corresponding to the next frame R last ; For the overall heat map R map The overall heatmap calculation expression is updated as follows:

[0214] R map =αR map +ωR cur ,

[0215] Where α (0 < α < 1) is the heatmap update factor; the closer α is to 1, the slower the heatmap updates over time; ω is the weight of the current heatmap, determined by the degree of change T of the heatmap. pc The result, along with the number of human targets n, determines its expression:

[0216]

[0217] Among them, A1, B1, A2, and B2 are constant terms that are greater than 0.

[0218] Furthermore, in some embodiments of this example, when the point cloud allocation module 303 performs the function of allocating the clustered target 2D point cloud based on the location information of the person being cared for to obtain the point cloud of the caregiver, it is specifically used to: allocate the clustered target 2D point cloud based on the location information of the person being cared for and the number of human targets; when the number of human targets is 0, no point cloud allocation is performed; when the number of human targets is 1, the point cloud within the preset radius of the location of the person being cared for is allocated to the person being cared for, and the point cloud is not allocated to the caregiver; when the number of human targets is 2, the clustered target 2D point cloud includes the corresponding first cluster and second cluster, and the number of point clouds contained in the first cluster is greater than that in the second cluster. The number of point clouds contained in the class; if the point cloud area of ​​the first cluster class is greater than the preset area or the number of point clouds in the second cluster class is less than the preset number, then the point clouds of the first cluster class contained within the preset radius of the location of the person being cared for are assigned to the person being cared for, and the remaining point clouds of the first cluster class are assigned to the caregiver; if the point cloud area of ​​the first cluster class is less than the preset area and the number of point clouds in the second cluster class is greater than the preset number, then the first distance between the first cluster class and the location of the person being cared for and the second distance between the second cluster class and the location of the person being cared for are calculated, the first distance and the second distance are compared, and based on the comparison result, the cluster class with a smaller distance from the location of the person being cared for is assigned to the person being cared for, and the cluster class with a larger distance from the location of the person being cared for is assigned to the caregiver.

[0219] In some embodiments of this example, the nursing behavior feature parameters in the behavior determination module 305 include nursing large displacement parameters, nursing goal existence parameters, nursing large movement parameters, nursing area continuity parameters, nursing large movement continuity parameters, and nursing effectiveness behavior matching parameters; when the trajectory tracking module 304 performs the function of extracting nursing behavior feature parameters based on trajectory coordinate sequences, it is used to: obtain the trajectory coordinate sequence corresponding to the nursing staff; set a sliding time window to continuously record the maximum displacement within each time window; wherein, the maximum displacement D max The expression is:

[0220] D max =max t＜i＜j＜t+Twin D ij ,

[0221] Where t is the starting point of the time window, Twin is the length of the sliding time window, i and j are any two points in the trajectory coordinate sequence within the sliding time window, and D is the starting point of the time window. ij D represents the displacement between i and j. ij The expression is:

[0222]

[0223] The nursing displacement parameter σ1 is extracted based on the maximum displacement; the expression for the nursing displacement parameter σ1 is:

[0224]

[0225] Where N is the duration of the nursing action, Twin is the sliding time window length, u is the step function, and Dthre is the preset distance threshold; the number of targets in the radar detection space is obtained based on the trajectory coordinate sequence, and the nursing target existence parameter σ2 is extracted; the expression for the nursing target existence parameter σ2 is:

[0226]

[0227] Where Pt is the number of targets in the radar detection space at time t; the body motion index is obtained based on the radar echo signal, and the nursing gross motion parameter σ3 is extracted; where the expression of the nursing gross motion parameter σ3 is:

[0228]

[0229] Among them, B t Let t be the body movement index, and Bthre be the preset body movement index threshold. When there are two targets in the radar detection space, the first preset algorithm is used to determine whether the nursing actions of the targets within the preset time threshold meet the regional continuity condition, and the nursing regional continuity parameter σ4 is extracted based on the judgment result. The second preset algorithm is used to determine whether the body movement index of the targets within the preset time threshold is greater than the preset body movement threshold, and the nursing large movement continuity parameter σ5 is obtained by accumulating the judgment results. The third preset algorithm is used to determine whether the nursing actions of the targets within the preset time threshold meet the effectiveness condition, and the nursing effectiveness behavior matching parameter σ6 is extracted based on the judgment result.

[0230] In some embodiments of this example, when the behavior determination module 305 performs the function of determining the compliance of nursing behavior based on nursing behavior feature parameters, it is used to: fuse and determine the nursing large displacement parameters, nursing goal existence parameters, nursing large movement parameters, nursing area continuity parameters, nursing large movement continuity parameters, and nursing effectiveness behavior matching parameters based on the nursing score model and preset weights to obtain a nursing behavior score; wherein, the nursing score model is defined as:

[0231]

[0232] Where x represents the value of each nursing parameter, t is the threshold, and m is the theoretical upper limit of each nursing parameter; the expression for the nursing behavior score is:

[0233] score=(w1σ1+w2σ2+w3σ3+w4σ4+w5σ5+w6σ6),

[0234] Among them, w1, w2, w3, w4, w5, and w6 are preset weight parameters, and w1+w2+w3+w4+w5+w6=1; the compliance of nursing behavior is determined based on the nursing behavior score.

[0235] In detail, each module in the radar-based nursing behavior recognition system 300 provided in this embodiment of the invention adopts the same characteristics as described above during use. Figure 1 The same technical means are used for identifying nursing behaviors, and can produce the same technical effects, so I will not go into details here.

[0236] Please see Figure 4 , Figure 4 A block diagram of a radar provided in an embodiment of this application.

[0237] like Figure 4 As shown, this application embodiment also provides a radar that can be used to implement the nursing behavior recognition method in the foregoing embodiments. The radar includes a memory 401, at least one processor 402, a signal generator 403, and a signal receiver 404. The memory 401 is used to store at least one program, and when the at least one program is executed by the at least one processor 402, the at least one processor 402 executes the nursing behavior recognition method provided in this application embodiment.

[0238] Please see Figure 5 , Figure 5 A block diagram of a computer-readable storage medium provided in an embodiment of this application.

[0239] like Figure 5 As shown, this application embodiment also provides a computer-readable storage medium 500, on which executable instructions 510 are stored, which, when executed, perform the nursing behavior recognition method provided in this application embodiment.

[0240] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art.

[0241] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., Digital Video Disk, DVD), or a semiconductor medium (e.g., Solid State Disk).

[0242] It should be noted that the various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For product-related embodiments, since they are similar to method-related embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method-related embodiments.

[0243] It should also be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0244] The above description of the disclosed embodiments enables those skilled in the art to implement or use the content of this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A radar-based care behavior recognition method, characterized by, The method comprises: target detection is performed on collected radar echo data, and a target 2D point cloud is generated based on the target detection result; clustering processing is performed based on the target 2D point cloud, and the position information of the person being cared for is determined based on the clustered target 2D point cloud; point cloud distribution is performed on the clustered target 2D point cloud based on the position information of the person being cared for, to obtain a caregiver point cloud; trajectory tracking is performed on the caregiver point cloud, to obtain a trajectory coordinate sequence; nursing behavior characteristic parameters are extracted based on the trajectory coordinate sequence, and nursing behavior compliance is determined based on the nursing behavior characteristic parameters; a nursing behavior compliance audit report is generated based on the nursing behavior compliance determination result.

2. The care action recognition method according to claim 1, characterized by, The clustering processing based on the target 2D point cloud and the determination of the position information of the person being cared for based on the clustered target 2D point cloud comprise: clustering processing is performed on the target 2D point cloud, to obtain a clustered target 2D point cloud; the number of human body targets in the target 2D point cloud is obtained by processing the clustered target 2D point cloud based on a number prediction algorithm; a heat map is updated based on the clustered target 2D point cloud and the number of human body targets; wherein the heat map is a numerical matrix determined based on a radar detection range and a resolution of the heat map; the position information of the person being cared for is determined based on the heat map.

3. The care action recognition method according to claim 2, characterized in that, The processing of the clustered target 2D point cloud based on the number prediction algorithm to obtain the number of human body targets in the target 2D point cloud comprises: whether the number of human body targets in the target 2D point cloud is 0 is determined by processing the clustered target 2D point cloud based on a first number prediction algorithm model; when the number of human body targets is not 0, feature extraction is performed on the clustered target 2D point cloud, and a feature vector is generated based on the feature values obtained by feature extraction; the number of human body targets in the target point cloud is obtained by processing the feature vector based on a second number prediction algorithm model.

4. The care behavior recognition method according to claim 3, characterized in that, The updating of the heat map based on the clustered target 2D point cloud and the number of human body targets comprises: If the current is the first frame of point cloud, initialize the overall heat map R map ; Initialize the current thermal map R for pre-update cur ; Calculate the position of all point clouds belonging to R cur of the current frame, and set the corresponding cell to 1; If the current is the first frame of point cloud, initialize the last frame heat map R last ; Calculate the heat map change degree T of the current frame and the previous frame pc The calculation expression of the heat map change degree T pc is as follows: wherein l represents the detection range of the radar, and r represents the resolution of the heat map; save the current heat map R cur as the previous frame heat map R last corresponding to the next frame For the overall heat map R map The overall heatmap calculation expression is updated as follows: R map = aR map + ωR cur , Wherein, a (0 < a < 1) is a heat map update factor, the value of a is closer to 1, the heat map is updated slower with time; ω is the weight of the current heat map, which is determined by the heat map change degree T pc And the number of human body targets n result jointly determine, the expression is: wherein A1, B1, A2, and B2 are constant terms greater than 0.

5. The care action recognition method according to claim 2, characterized by, The point cloud distribution based on the position information of the person being cared for and the number of human body targets to obtain the caregiver point cloud comprises: point cloud distribution is performed on the clustered target 2D point cloud based on the position information of the person being cared for and the number of human body targets; when the number of human body targets is 0, no point cloud distribution is performed; when the number of human body targets is 1, the point cloud within a preset radius of the position of the person being cared for is distributed to the person being cared for, and no point cloud is distributed to the caregiver; when the number of human body targets is 2, the clustered target 2D point cloud includes a corresponding first cluster and a second cluster, and the number of point clouds included in the first cluster is greater than the number of point clouds included in the second cluster; If the point cloud area of the first cluster class is greater than a preset area or the point cloud quantity of the second cluster class is less than a preset quantity, the first cluster class of points contained within a preset radius of the caregiver position is assigned to the caregiver, and the remaining first cluster class of points is assigned to the care receiver; If the point cloud area of the first cluster class is less than a preset area and the point cloud quantity of the second cluster class is greater than a preset quantity, a first distance of the first cluster class from the caregiver position and a second distance of the second cluster class from the caregiver position are calculated, the first distance and the second distance are compared, and based on the comparison result, the cluster class with a smaller distance from the caregiver position is assigned to the caregiver, and the cluster class with a larger distance from the caregiver position is assigned to the care receiver.

6. The care action recognition method according to claim 1, characterized by, The nursing behavior characteristic parameters include a nursing large displacement parameter, a nursing target existence parameter, a nursing large action parameter, a nursing area continuity parameter, a nursing large action continuity parameter, and a nursing effectiveness behavior matching parameter; The nursing behavior characteristic parameters are extracted based on the trajectory coordinate sequence, including: The trajectory coordinate sequence corresponding to the nursing staff is obtained; The sliding time window is set to continuously record the maximum displacement amount in each time window; wherein the maximum displacement amount D max The expression is: D max = max t＜i＜j＜t+Twin D ij , Where t is the starting point of the time window, Twin is the length of the sliding time window, i and j are any two points in the trajectory coordinate sequence within the sliding time window, and D ij D represents the displacement between i and j. ij The expression is: The nursing large displacement parameter σ1 is extracted based on the maximum displacement amount; wherein the expression of the nursing large displacement parameter σ1 is: wherein N is the duration of the nursing behavior, Tw in is the sliding time window length, u is a step function, and Dthre is a preset distance threshold; The number of targets in the radar detection space is obtained based on the trajectory coordinate sequence, and the nursing target existence parameter σ2 is extracted; wherein the expression of the nursing target existence parameter σ2 is: wherein Pt is the number of targets in the radar detection space at time t; Compensate the radar echo signal based on the compensation curve to obtain a compensated radar echo signal; wherein the compensation curve is expressed as a sequence cpl=[1, c1,…c N-1 ], c represents the value of the signal of each distance unit in the sequence, and N represents the maximum value of the distance unit. The body motion index is obtained based on the compensated radar echo signal, and the nursing large action parameter σ3 is extracted; wherein the expression of the nursing large action parameter σ3 is: Wherein, B t is the body motion index at time t, and Bthre is a preset body motion index threshold value. When the number of targets in the radar detection space is two, whether the nursing action of the target within a preset time threshold satisfies the area continuity condition is judged based on a first preset algorithm, and the nursing area continuity parameter σ4 is extracted based on the judgment result; Whether the body motion index of the target within a preset time threshold is greater than a preset body motion threshold is judged based on a second preset algorithm, and the nursing large action continuity parameter σ5 is accumulated based on the judgment result; Whether the nursing action of the target within a preset time threshold satisfies the effectiveness condition is judged based on a third preset algorithm, and the nursing effectiveness behavior matching parameter σ6 is extracted based on the judgment result.

7. The care behavior recognition method according to claim 6, characterized by, The nursing behavior compliance is determined based on the nursing behavior characteristic parameters, including: The nursing large displacement parameter, the nursing target existence parameter, the nursing large action parameter, the nursing area continuity parameter, the nursing large action continuity parameter, and the nursing effectiveness behavior matching parameter are fused and determined based on a nursing score model and a preset weight to obtain a nursing behavior score; wherein the nursing score model is defined as: Wherein, x is the value of each nursing parameter, t is the threshold, m is the theoretical upper limit value of each nursing parameter; The expression of the nursing behavior score score is: score=(w1σ1+w2σ2+w3σ3+w4σ4+w5σ5+w6σ6), Wherein, w1, w2, w3, w4, w5, w6 are preset weight parameters, and w1+w2+w3+w4+w5+w6=1; Based on the nursing behavior score, the compliance of nursing behavior is determined. 8.A radar-based care behavior recognition system, characterized by, It includes: A point cloud generation module is configured to perform target detection on the collected radar echo data, and generate a target 2D point cloud based on the target detection result; A position acquisition module is configured to perform clustering processing based on the target 2D point cloud, and determine the position information of the nursing person based on the clustered target 2D point cloud; A point cloud distribution module is configured to perform point cloud distribution on the clustered target 2D point cloud based on the position information of the nursing person, to obtain a nursing person point cloud; A trajectory tracking module is configured to perform trajectory tracking on the nursing person point cloud, to obtain a trajectory coordinate sequence; A behavior determination module is configured to extract nursing behavior feature parameters based on the trajectory coordinate sequence, and determine the compliance of nursing behavior based on the nursing behavior feature parameters; A report generation module is configured to generate a nursing behavior compliance audit report based on the compliance of nursing behavior.

9. A radar, characterized by It includes a memory and a processor, wherein: The processor is configured to execute a computer program stored in the memory; When the processor executes the computer program, the steps of the nursing behavior recognition method in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, the steps of the nursing behavior recognition method in any one of claims 1 to 7 are implemented.

Images