An old person fall monitoring and early warning method based on millimeter wave radar and camera

Abstract

This invention discloses a fall detection and early warning method for the elderly based on millimeter-wave radar and a camera, belonging to the interdisciplinary field of deep learning and data processing technology. First, it collects daily activity data of the elderly to construct a labeled dataset. Then, it optimizes the hyperparameter vector using an improved gray wolf optimization algorithm to obtain the target multimodal attention weights and target dynamic threshold parameters. Next, it uses millimeter-wave radar to acquire real-time point cloud data, extracts features through a point cloud spatiotemporal coding network, and obtains a sliding window wake-up flag. When the sliding window wake-up flag is active, it wakes up the camera to collect real-time image data, extracts the set of key points of the human skeleton, and calculates visual confidence. Finally, it fuses the millimeter-wave radar feature vector, the camera visual feature vector, and the time-series feature vector, analyzes the fall probability through a feature recognition model, and provides tiered early warnings. This effectively improves the accuracy of fall detection for the elderly, reduces the false alarm rate and alarm delay rate, and provides a more reliable guarantee for the safety of the elderly.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of deep learning and data processing technology, and specifically relates to a method for monitoring and early warning of falls in the elderly based on millimeter-wave radar and cameras. Background Technology

[0002] Existing fall detection technologies mainly include wearable device-based methods and vision-based methods. While wearable devices provide direct data, they require prolonged use by the elderly, leading to poor compliance and issues such as forgetting to charge them. Single-vision monitoring (cameras), although providing rich image information, is greatly affected by lighting conditions and raises concerns about infringing on the elderly's privacy. In recent years, millimeter-wave radar-based detection schemes have gained attention due to their advantages such as privacy protection and immunity to light effects; however, radar data is relatively weak in static object recognition and detailed feature extraction, making it prone to false alarms. Currently, although there are attempts at multi-sensor fusion, prolonged camera operation not only consumes computing power but also poses a serious risk of privacy leakage; relying solely on fixed radar thresholds for wake-up is prone to missed detections or false wake-ups. Simple feature stitching cannot effectively handle the spatiotemporal heterogeneity of radar point clouds and image data, resulting in limited detection accuracy. Traditional methods rely on manual experience to set detection thresholds (such as height and speed thresholds), making it difficult to adapt to individual differences among elderly people and environmental changes. Therefore, existing technologies are inaccurate in fall detection for the elderly and have poor adaptability. Summary of the Invention

[0003] This invention provides a method for detecting and warning of falls in the elderly based on millimeter-wave radar and a camera, in order to solve the problems of inaccurate fall detection and poor adaptability of existing technologies.

[0004] A method for detecting and warning of falls in the elderly based on millimeter-wave radar and cameras includes: Collect daily activity data of the target elderly and construct a labeled dataset based on the daily activity data; the labeled dataset includes sample data and the corresponding labels of the sample data; Candidate hyperparameter vectors are constructed, and based on the sample data and the corresponding labels, the candidate hyperparameter vectors are optimized using an error function that fuses fall detection accuracy, false alarm rate, and alarm delay rate, as well as an improved gray wolf optimization algorithm, to obtain a target hyperparameter vector; the target hyperparameter vector includes at least target multimodal attention weights and target dynamic threshold parameters; Real-time point cloud data is acquired using millimeter-wave radar. Features are extracted from the real-time point cloud data using a pre-set point cloud spatiotemporal coding network to obtain point cloud geometric feature vectors and point cloud kinematic feature vectors. Based on the point cloud geometric feature vectors, point cloud kinematic feature vectors, and target dynamic threshold parameters, single-frame wake-up information is obtained, and a sliding window wake-up flag is obtained based on the single-frame wake-up information. The sliding window wake-up flag includes a wake-up flag or a non-wake-up flag. When the sliding window wake-up flag is set to wake-up, the camera is woken up to perform real-time image acquisition, obtain real-time image data, and use a pre-set skeletal point extraction model to extract the set of human skeletal key points in the real-time image data; obtain the angle between the human body and the horizontal plane based on the set of human skeletal key points, and obtain the visual confidence level based on the angle and the target dynamic threshold parameter. Based on the point cloud geometric feature vector, point cloud kinematic feature vector, included angle, and visual confidence, millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector are constructed respectively. The millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector are then weighted and fused according to the target multimodal attention weight to obtain a fused feature vector. The fused feature vector is analyzed using a pre-set feature recognition model to obtain the fall probability. The fall determination result is determined based on the fall probability and the target dynamic threshold parameter. Finally, a fall classification warning is given based on the fall determination result, realizing fall monitoring and warning for the elderly based on millimeter-wave radar and camera.

[0005] Furthermore, daily activity data of the target elderly are collected, and a labeled dataset is constructed based on the daily activity data, including: The system collects millimeter-wave radar data and camera image data of elderly people in non-fall states, as well as millimeter-wave radar data and camera image data of elderly people in simulated fall and / or fall states. Using millimeter-wave radar data and camera image data as sample data, non-fall states are constructed as non-fall labels, and simulated falls and / or fall states are constructed as fall labels, thus obtaining the labeling labels corresponding to the sample data.

[0006] Further, candidate hyperparameter vectors are constructed, and based on the sample data and the corresponding labels, the candidate hyperparameter vectors are optimized using an error function fused from fall detection accuracy, false alarm rate, and alarm delay rate, along with an improved Grey Wolf optimization algorithm, to obtain the target hyperparameter vector, including: Construct candidate hyperparameter vectors; the candidate hyperparameter vectors include attention weights for millimeter-wave radar features, attention weights for camera visual features, attention weights for temporal features, human centroid height threshold, instantaneous velocity change threshold, and human body angle threshold with respect to the horizontal plane. By repeatedly constructing multiple different candidate hyperparameter vectors, multiple individual gray wolves can be obtained; For any individual gray wolf, based on the sample data and the corresponding labels, the loss function value for each individual gray wolf is obtained by fusing the fall detection accuracy, false alarm rate and alarm delay rate using an error function. Based on the loss function value, the improved gray wolf optimization algorithm is used to optimize the individual gray wolves, resulting in optimized gray wolf individuals; Repeat the optimization steps for individual gray wolves until the maximum number of training iterations is reached, and obtain the target hyperparameter vector.

[0007] Furthermore, for any individual gray wolf, based on the sample data and the corresponding labels, a loss function value is obtained for each individual gray wolf using an error function that fuses fall detection accuracy, false alarm rate, and alarm delay rate, including: For any individual gray wolf, based on the sample data, the probability of falling is obtained, and based on the probability of falling and the label corresponding to the sample data, the accuracy of fall detection, false alarm rate and alarm delay rate are obtained. Based on the fall detection accuracy, false alarm rate, and alarm delay rate, the loss function value corresponding to the individual gray wolf is obtained as follows: ; in, This refers to an individual gray wolf. This represents the loss function value of an individual gray wolf. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the third weighting coefficient, and + + =1; This indicates the accuracy rate of fall detection. Indicates the false alarm rate. Indicates alarm delay rate, This indicates the system delay time from the occurrence of a fall to the time the system alarm is triggered. This indicates the maximum allowed delay time.

[0008] Furthermore, based on the loss function value, the improved gray wolf optimization algorithm is used to optimize the individual gray wolves, resulting in optimized gray wolf individuals, including: Based on the loss function value, the individual gray wolves are classified into individual identities to obtain optimized gray wolf individuals after individual identity classification; Employing improved prey encirclement strategies and introducing The gray wolf optimization algorithm based on the wolf neighborhood adaptive exploration strategy optimizes gray wolf individuals after individual identity classification, and obtains optimized gray wolf individuals.

[0009] Further, real-time point cloud data is acquired via millimeter-wave radar, and features are extracted from the real-time point cloud data using a pre-set point cloud spatiotemporal coding network to obtain point cloud geometric feature vectors and point cloud kinematic feature vectors. Based on the point cloud geometric feature vectors, point cloud kinematic feature vectors, and target dynamic threshold parameters, single-frame wake-up information is obtained, and a sliding window wake-up flag is obtained based on the single-frame wake-up information, including: Real-time point cloud data is acquired using millimeter-wave radar; the real-time point cloud data includes the three-dimensional spatial coordinates of the point cloud, the radial velocity of the point cloud, and the signal-to-noise ratio. Using the real-time point cloud data as input, a point cloud geometric feature vector consisting of the centroid height of the point cloud cluster, the width ratio of the cloud cluster, and the rate of change of the ground projection area of ​​the point cloud cluster is obtained through a pre-set point cloud spatiotemporal coding network. Additionally, a point cloud kinematic feature vector consisting of the instantaneous horizontal velocity of the centroid of the point cloud cluster and the acceleration of the centroid of the point cloud cluster in the vertical direction is obtained. If the height of the centroid of the point cloud cluster is less than the threshold of the height of the human centroid and the instantaneous horizontal velocity of the centroid of the point cloud cluster is greater than the threshold of the instantaneous velocity change, then the single-frame wake-up information is determined to be 1; otherwise, the single-frame wake-up information is determined to be 0. If the single-frame wake-up information is 1 for three consecutive frames and the acceleration of the cloud cluster centroid in the vertical direction is greater than a preset threshold, then the sliding window wake-up flag is determined to be a wake-up flag; otherwise, the sliding window wake-up flag is determined to be a non-wake-up flag.

[0010] Further, a pre-set skeletal point extraction model is used to extract a set of human skeletal key points from the real-time image data; the angle between the human body and the horizontal plane is obtained based on the set of human skeletal key points, and visual confidence is obtained based on the angle and the target dynamic threshold parameter, including: A pre-set skeletal point extraction model is used to extract the set of human skeletal key points from the real-time image data; the set of human skeletal key points includes the coordinates of the head key points, the coordinates of the hip key points, the coordinates of the left ankle key points, and the coordinates of the right ankle key points. Based on the coordinates of the head key points and the hip key points, the angle between the human body and the horizontal plane is obtained. If the angle between the human body and the horizontal plane is less than the threshold value for the angle between the human body and the horizontal plane, then the visual confidence level is determined to be the first preset value; otherwise, the visual confidence level is determined to be the second preset value.

[0011] Further, based on the point cloud geometric feature vector, point cloud kinematic feature vector, included angle, and visual confidence, millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector are constructed respectively. These feature vectors are then weighted and fused according to the target multimodal attention weight to obtain a fused feature vector, including: The point cloud geometric feature vector and the point cloud kinematic feature vector are used to construct a millimeter-wave radar feature vector; The included angle and visual confidence level are used to form a camera visual feature vector; The descent amplitude and the static time of the human body are obtained from the geometric feature vector of the point cloud, and then a vector is formed to obtain the temporal feature vector. The attention weights of millimeter-wave radar features, camera visual features, and temporal features are used to weight the millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector respectively, and then fused to obtain a fused feature vector.

[0012] Furthermore, a pre-set feature recognition model is used to analyze the fused feature vector to obtain the fall probability, and a fall determination result is determined based on the fall probability and the target dynamic threshold parameter. Finally, a fall classification warning is performed based on the fall determination result, including: The fused feature vector is used as input to the pre-set feature recognition model to obtain the fall probability; The fall determination result is determined based on a pre-set fall probability threshold range, fall probability, visual confidence level, and human center of gravity height threshold; the fall determination result includes fall, suspected fall, or no fall; Execute pre-set warnings based on fall assessment results to achieve graded fall warnings.

[0013] Furthermore, the method also includes: Respiratory rate and heart rate variability are extracted using the micro-Doppler effect of millimeter-wave radar, and enhanced early warning is provided based on the respiratory rate and heart rate variability.

[0014] This invention provides a method for detecting and warning of falls in the elderly based on millimeter-wave radar and a camera. This invention provides a fall detection and early warning method for the elderly based on millimeter-wave radar and a camera. First, it collects daily activity data of the elderly to construct a labeled dataset. Then, it optimizes the hyperparameter vector using an improved gray wolf optimization algorithm to obtain the target multimodal attention weights and target dynamic threshold parameters. Next, it uses millimeter-wave radar to acquire real-time point cloud data, extracts features through a point cloud spatiotemporal coding network, and obtains a sliding window wake-up flag. When the sliding window wake-up flag is active, it activates the camera to collect real-time image data, extracts the set of key points of the human skeleton, and calculates visual confidence. Finally, it fuses the millimeter-wave radar feature vector, the camera visual feature vector, and the temporal feature vector, and analyzes the fall probability through a feature recognition model to provide tiered early warnings. This effectively improves the accuracy of fall detection for the elderly, reduces the false alarm rate and alarm delay rate, and solves the problems of low accuracy, high false alarm rate, and alarm delay in existing fall detection methods for the elderly, providing a more reliable guarantee for the safety of the elderly. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0016] Figure 1 A flowchart illustrating a method for monitoring and warning of falls in the elderly based on millimeter-wave radar and a camera, provided as an embodiment of the present invention.

[0017] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0018] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] like Figure 1 As shown, this embodiment of the invention provides a method for detecting and warning of falls in the elderly based on millimeter-wave radar and a camera, including: S101. Collect daily activity data of the target elderly person and construct a labeled dataset based on the daily activity data; the labeled dataset includes sample data and the label corresponding to the sample data; S102. Construct candidate hyperparameter vectors, and based on the sample data and the corresponding labels, optimize the candidate hyperparameter vectors using an error function that fuses fall detection accuracy, false alarm rate, and alarm delay rate, and an improved gray wolf optimization algorithm to obtain target hyperparameter vectors; the target hyperparameter vectors include at least target multimodal attention weights and target dynamic threshold parameters; S103. Real-time point cloud data is acquired through millimeter-wave radar. Feature extraction is performed on the real-time point cloud data through a pre-set point cloud spatiotemporal coding network to obtain point cloud geometric feature vectors and point cloud kinematic feature vectors. Based on the point cloud geometric feature vectors, point cloud kinematic feature vectors, and target dynamic threshold parameters, single-frame wake-up information is acquired, and a sliding window wake-up flag is acquired based on the single-frame wake-up information. The sliding window wake-up flag includes a wake-up flag or a non-wake-up flag. S104. When the sliding window wake-up flag is the wake-up flag, the camera is woken up to perform real-time image acquisition, obtain real-time image data, and use a pre-set skeletal point extraction model to extract the set of human skeletal key points in the real-time image data; obtain the angle between the human body and the horizontal plane based on the set of human skeletal key points, and obtain the visual confidence level based on the angle and the target dynamic threshold parameter. S105. Based on the point cloud geometric feature vector, point cloud kinematic feature vector, included angle, and visual confidence, construct millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector respectively. Then, perform weighted fusion of the millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector according to the target multimodal attention weight to obtain a fused feature vector. S106. The fused feature vector is analyzed using a pre-set feature recognition model to obtain the fall probability. The fall determination result is determined based on the fall probability and the target dynamic threshold parameter. Finally, a fall classification warning is given based on the fall determination result, realizing fall monitoring and warning for the elderly based on millimeter-wave radar and camera.

[0021] In fall detection scenarios within non-sleeping areas for elderly residents, this invention improves the accuracy, real-time performance, privacy protection, and personalized adaptation of fall detection by deeply fusing the Gray Wolf optimization algorithm with multimodal features. It can adaptively adjust to different body types and home environments. The Gray Wolf optimization algorithm automatically searches for optimal fusion weights and decision thresholds, enhancing fall detection accuracy. An architecture combining millimeter-wave radar constant-on monitoring with camera event triggering, along with a sliding window judgment mechanism, controls system latency, ensuring the camera only wakes up when the radar detects a suspicious event, remaining in standby mode otherwise. This allows for safe deployment in highly privacy-sensitive areas such as bathrooms and bedrooms. Furthermore, the system integrates respiratory rate and heart rate variability monitoring functions, supporting tiered alerts and effectively ensuring the safety of the elderly.

[0022] In some possible embodiments, daily activity data of the target elderly person are collected, and a labeled dataset is constructed based on the daily activity data, including: The system collects millimeter-wave radar data and camera image data of elderly people in non-fall states, as well as millimeter-wave radar data and camera image data of elderly people in simulated fall and / or fall states. Using millimeter-wave radar data and camera image data as sample data, non-fall states are constructed as non-fall labels, and simulated falls and / or fall states are constructed as fall labels, thus obtaining the labeling labels corresponding to the sample data.

[0023] For example, daily activity data (walking, sitting, lying down, bending over, simulating falls) of the target elderly person can be collected in this living environment to construct a labeled dataset: ;in, The millimeter-wave radar data represents the i-th sample data, i.e., the millimeter-wave radar point cloud data. The dimension is ; This represents the number of points in the point cloud at frame t, and can take values ​​in [50, 500]. Each dimension includes the x-axis coordinate, y-axis coordinate, z-axis coordinate, radial velocity, and signal-to-noise ratio for each point. This represents the camera image of the i-th sample data, with dimension . , W represents the image height, C represents the image width, and C represents the RGB color channels. Let represent the sample label of the i-th sample data, and =1 indicates falling down. =0 indicates that the person did not fall, and N represents the total number of sample data.

[0024] In some possible embodiments, a candidate hyperparameter vector is constructed, and based on the sample data and the corresponding labels, the candidate hyperparameter vector is optimized using an error function fused from fall detection accuracy, false alarm rate, and alarm delay rate, along with an improved gray wolf optimization algorithm, to obtain a target hyperparameter vector, including: Construct candidate hyperparameter vectors; the candidate hyperparameter vectors include attention weights for millimeter-wave radar features, attention weights for camera visual features, attention weights for temporal features, human centroid height threshold, instantaneous velocity change threshold, and human body angle threshold with respect to the horizontal plane. The candidate hyperparameter vector is: ; in, The attention weight represents the characteristics of millimeter-wave radar; it is dimensionless and ranges from [0,1]. The attention weights represent the visual features of the camera; they are dimensionless and range from [0,1]. The attention weights represent temporal features, are dimensionless, and range from [0,1]; and satisfy the following conditions: + + =1. During optimization, if the attention weight constraint is not met, the two largest attention weights are retained, and then 1 is subtracted from the two largest attention weights to obtain the third attention weight. Alternatively, the sum of the largest attention weight and any other attention weight may be greater than 1. In this case, the largest attention weight is retained, and the remaining weights are evenly distributed among the other two attention weights. The remaining weight is 1 minus the largest attention weight. This weight constraint and normalization mechanism improves the system's robustness, ensuring that even if abnormal weight allocation occurs during training, the algorithm can automatically adjust to maintain a total weight sum of 1, avoiding performance degradation due to improper weight allocation. This represents the threshold height of the human body's center of mass, expressed in meters (m), with a value range of [0.2, 1.0]. This represents the threshold for instantaneous velocity change, expressed in meters per second (m / s), with a value range of [1.0, 3.0]. This represents the threshold angle between the human body and the horizontal plane, in degrees (°), with a range of [15, 60].

[0025] By repeatedly constructing multiple different candidate hyperparameter vectors, multiple individual gray wolves can be obtained; For any individual gray wolf, based on the sample data and the corresponding labels, the loss function value for each individual gray wolf is obtained by fusing the fall detection accuracy, false alarm rate and alarm delay rate using an error function. Based on the loss function value, the improved gray wolf optimization algorithm is used to optimize the individual gray wolves, resulting in optimized gray wolf individuals; Repeat the optimization steps for individual gray wolves until the maximum number of training iterations is reached, and obtain the target hyperparameter vector.

[0026] In some possible embodiments, for any individual gray wolf, based on the sample data and the corresponding labels, a loss function value is obtained for each individual gray wolf using an error function that fuses fall detection accuracy, false alarm rate, and alarm delay rate, including: For any individual gray wolf, based on the sample data, the probability of falling is obtained, and based on the probability of falling and the label corresponding to the sample data, the accuracy of fall detection, false alarm rate and alarm delay rate are obtained. Based on the fall detection accuracy, false alarm rate, and alarm delay rate, the loss function value corresponding to the individual gray wolf is obtained as follows: ; in, This refers to an individual gray wolf. This represents the loss function value of an individual gray wolf. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the third weighting coefficient, and + + =1; This represents the fall detection accuracy, is dimensionless, and takes values ​​[0,1]. ; This represents the number of correctly detected fall samples, that is, the number of samples whose true label is "fall" and whose system output is "fall". This represents the number of correctly detected non-fall samples, that is, the number of samples whose true label is "non-fall" and whose system output is "non-fall". This indicates the number of false alarms, that is, the number of samples whose true label is not "fall" but whose system output is "fall". This indicates the number of missed samples, that is, the number of samples whose actual label is "fall" but whose system output is "not fall". represents the false alarm rate, dimensionless, taking values ​​[0,1], and = ; Indicates alarm delay rate, This indicates the system delay time from the occurrence of a fall to the system alarm, in seconds (s). This indicates the maximum allowed delay time, which is set to 5 seconds by default. , Set them to 0.6, 0.3, and 0.1 respectively; Optionally, the system delay time from the occurrence of a fall to the system alarm typically refers to: for real-time detection, this system delay time usually refers to the time from acquiring millimeter-wave radar point cloud data to achieving tiered early warning. For sample data, it can be based on the sample data acquisition time, plus radar feature extraction time, preset camera wake-up and shooting time, visual inference time, and tiered early warning trigger time. The tiered early warning trigger time refers to the time from the completion of visual inference to the triggering of the tiered early warning. It is worth noting that the loss function value is obtained through a large amount of sample data; therefore, This can represent the average delay time of all sample data. Alternatively, the alarm delay rate can be directly removed. As the loss function, at this time + =1.

[0027] In some possible embodiments, based on the loss function value, an improved gray wolf optimization algorithm is used to optimize individual gray wolves, resulting in optimized gray wolf individuals, including: Based on the loss function value, the individual gray wolves are classified into individual identities to obtain optimized gray wolf individuals after individual identity classification; Employing improved prey encirclement strategies and introducing The gray wolf optimization algorithm based on the wolf neighborhood adaptive exploration strategy optimizes gray wolf individuals after individual identity classification, and obtains optimized gray wolf individuals.

[0028] Existing gray wolf optimization algorithms are prone to getting trapped in local optima during the optimization process, resulting in poor optimization performance and an inability to achieve precise adjustment. Therefore, embodiments of this application employ an improved prey-encircling strategy and introduce... The gray wolf optimization algorithm of the wolf neighborhood adaptive exploration strategy optimizes gray wolf individuals after individual identity classification, improves the ability to find the optimal solution in the high-dimensional solution space, thereby improving the adaptability and monitoring accuracy of the technical solution described in the embodiments of this application.

[0029] For example, an improved prey-encircling strategy is as follows: ; in, Represents the group enclosing factor. This represents the maximum value of the group enclosing factor. This represents the minimum value of the group's bounding factor. This indicates the current number of optimizations in the Grey Wolf optimization algorithm. This indicates the preset maximum number of optimization attempts; Based on the aforementioned group encirclement factor and all individual gray wolves, the method for encircling prey is as follows: ; in, This represents the nth individual gray wolf. dimensional elements, This indicates the number of individual gray wolves obtained by surrounding prey. dimensional elements, Indicates the first Upper limit of dimensional elements, Indicates the first Lower bound of dimensional elements This represents the total number of individual gray wolves. Indicates the first The first gray wolf individual The element iter represents the current optimization iteration of the Grey Wolf optimization algorithm. This indicates that the nth gray wolf individual is related to the nth individual. The Euclidean distance between individual gray wolves This represents the intensity factor and can be set to 0.5; Let C represent the natural constant, and let C represent the constant term, which can be set to 1.5. Let represent the fitness of the solution space for the nth gray wolf individual. This represents the fitness of the worst-case solution space for all individual gray wolves. This represents the fitness of the optimal solution space for all individual gray wolves, where the fitness of the optimal solution space = 1 / ( +0.001).

[0030] This improved prey-encircling strategy, by introducing a group encirclement factor, dynamically adjusts the search strategy of individual wolves, effectively enhancing the algorithm's optimization accuracy and convergence speed. During the global search phase, a larger group encirclement factor strengthens the algorithm's global exploration capability, preventing it from getting trapped in local optima. During the local exploration phase, a smaller group encirclement factor prompts the algorithm to focus on a refined search near the optimal solution, significantly improving optimization accuracy. Furthermore, through the rational design of the strength factor and fitness function, the algorithm further balances the relationship between global search and local exploration, enabling it to quickly find the global optimum in practical applications such as complex multimodal function optimization and engineering parameter design, demonstrating excellent performance and stability.

[0031] The wolf neighborhood adaptive exploration strategy is as follows: right The wolf performs neighborhood information collection, obtaining the first neighborhood information and the second neighborhood information as follows: ; ; in, Represents the first neighborhood information. Indicates the second neighborhood information. express Wolf, express The optimization speed of the wolf in the iter optimization process. This represents the range of neighborhood information collection and is set as a constant term between (0.01, 0.1). Based on the first neighborhood information and the second neighborhood information, The wolf performs neighborhood adaptive exploration as follows: ; ; ; in, The expression obtained by adaptive exploration of the neighborhood Wolf, express The optimization speed of the wolf during the iter+1 optimization process. Indicates the adaptive weighting factor. This represents the inertia weight, set to decrease linearly from 0.9 to 0.4 as the number of optimization iterations increases; Let represent the first learning factor, and set it to a constant in (0.5, 2); Represents the first random number between (0,1). express The historical optimal value corresponding to the wolf, that is, the value with the minimum loss function value; This represents the neighborhood exploration step size, and is set to 0.01; Represents a symbolic function. express The fitness of the solution space. express The fitness of the solution space. This represents the maximum value of the adaptive weighting factor, and is set to 0.9; This represents the minimum value of the adaptive weighting factor, and is set to 0.2; This represents the attenuation coefficient, and is set to 0.9 or 0.95; Should The wolf neighborhood adaptive exploration strategy effectively improves the search accuracy of the optimization algorithm by accurately collecting neighborhood information. Its adaptive weighting factor and dynamic adjustment mechanism enhance the algorithm's adaptability to different solution spaces, allowing for flexible adjustment of the search strategy according to the optimization process. Simultaneously, the rationally set neighborhood information collection range and exploration step size accelerate the algorithm's convergence speed, enabling it to more efficiently approach the optimal solution during iteration.

[0032] Employing improved prey encirclement strategies and introducing The gray wolf optimization algorithm with a wolf neighborhood adaptive exploration strategy can improve local search capabilities and ensure diversity in the early and mid-stages of the algorithm. It can also improve the global search and convergence capabilities of the gray wolf optimization algorithm, thereby ensuring the accuracy and adaptability of obtaining the target hyperparameter vector, and making it suitable for different elderly people.

[0033] In some possible embodiments, real-time point cloud data is acquired via millimeter-wave radar. Feature extraction is performed on the real-time point cloud data using a pre-set point cloud spatiotemporal coding network to obtain point cloud geometric feature vectors and point cloud kinematic feature vectors. Based on the point cloud geometric feature vectors, point cloud kinematic feature vectors, and target dynamic threshold parameters, single-frame wake-up information is obtained, and a sliding window wake-up flag is obtained based on the single-frame wake-up information, including: Real-time point cloud data is acquired using millimeter-wave radar; the real-time point cloud data includes the three-dimensional spatial coordinates of the point cloud, the radial velocity of the point cloud, and the signal-to-noise ratio. Using the real-time point cloud data as input, a point cloud geometric feature vector consisting of the centroid height of the point cloud cluster, the width ratio of the cloud cluster, and the rate of change of the ground projection area of ​​the point cloud cluster is obtained through a pre-set point cloud spatiotemporal coding network. Additionally, a point cloud kinematic feature vector consisting of the instantaneous horizontal velocity of the centroid of the point cloud cluster and the acceleration of the centroid of the point cloud cluster in the vertical direction is obtained. If the height of the centroid of the point cloud cluster is less than the threshold of the height of the human centroid and the instantaneous horizontal velocity of the centroid of the point cloud cluster is greater than the threshold of the instantaneous velocity change, then the single-frame wake-up information is determined to be 1; otherwise, the single-frame wake-up information is determined to be 0. If the single-frame wake-up information is 1 for three consecutive frames and the acceleration of the cloud cluster centroid in the vertical direction is greater than a preset threshold, then the sliding window wake-up flag is determined to be a wake-up flag; otherwise, the sliding window wake-up flag is determined to be a non-wake-up flag.

[0034] For example, deploying a millimeter-wave radar (77GHz FMCW, i.e., frequency-modulated continuous wave millimeter-wave radar), the raw point cloud data generated through range-Doppler transformation is as follows: ; in, This represents the millimeter-wave radar point cloud dataset at time t, i.e., the raw point cloud data; t represents the time frame index, the frame rate is 20 frames / second, and the interval between adjacent frames is Δt=0.05 seconds; This represents the X-axis coordinate of the m-th point in the t-th frame of the original point cloud data. This represents the Y-axis coordinate of the m-th point in the t-th frame of the original point cloud data. This represents the Z-axis coordinate of the m-th point in the t-th frame of the original point cloud data, in meters (m). This represents the radial velocity at the m-th point. This represents the signal-to-noise ratio at the m-th point. This represents the number of points in the original point cloud data of frame t, with a value of [50, 500]. The pre-configured point cloud spatiotemporal coding network can be set up as an encoder-decoder architecture. For example, the point cloud spatiotemporal coding network can include sequentially connected PointNet++, an architecture based on Transformer Encoder, and fully connected layers. However, it is worth noting that other existing techniques can also be used to extract point cloud geometric feature vectors and point cloud kinematic feature vectors.

[0035] Optionally, the pre-defined point cloud spatiotemporal coding network can also be defined as other software or network models and perform subordinate calculations, which may include: ; ; ; ; ; ; ; in, Represents the geometric eigenvectors of a point cloud. Let T denote the kinematic eigenvectors of the point cloud, and let T denote the transpose. This indicates the height of the centroid of a point cloud cluster, in meters (m). This represents the Z-axis coordinate of the j-th point in the t-th frame of the original point cloud data; This indicates the aspect ratio (the ratio of height to width) of a point cloud cluster. This represents the maximum value of the point cloud cluster along the Z-axis. This represents the minimum value of the point cloud cluster along the Z-axis. This represents the maximum value of the point cloud cluster along the X-axis. This represents the minimum value of the point cloud cluster along the X-axis. This represents the rate of change of the ground projection area of ​​a point cloud cluster; The area of ​​the point cloud cluster in frame t is the projected area on the ground (i.e., the XY plane), which is obtained by calculating the polygon area of ​​the convex hull of the point cloud in the XY plane. This represents the projected area of ​​the previous frame; This represents the instantaneous horizontal velocity of the centroid of a point cloud cluster, expressed in meters per second (m / s). , ) represents the planar coordinates of the centroid of the point cloud in frame t. , () represents the planar coordinates of the centroid of the point cloud in frame t-1. For frame interval, This represents the acceleration of the centroid of the point cloud cluster in the vertical direction (Z direction), expressed in meters per second squared (m / s²). This represents the height coordinates of the centroid of the point cloud in frame t; Indicates the t-th The elevation coordinates of the centroid of a point cloud in frame 1; Indicates the t-th The elevation coordinates of the centroids of the two point cloud frames. It's worth noting that, to reduce data complexity and enhance data consistency, all of the above parameters can be dedimensionalized.

[0036] Based on the parameters in the point cloud geometric feature vector and the point cloud kinematic feature vector, the single-frame wake-up information can be obtained as follows: ; in, The single-frame wake-up information of frame t is represented by a Boolean variable, where 1 indicates that the condition is met and 0 indicates that the condition is not met. The AND operator is represented. This represents the target human body centroid height threshold in the target dynamic threshold parameters; This represents the threshold for the instantaneous velocity change in the target dynamic threshold parameters. It is worth noting that if the various parameters being calculated have been dimensionless, then the threshold data also needs to be dimensionless to ensure that the data can be calculated accurately.

[0037] The sliding window wake-up flag is obtained based on the single-frame wake-up information as follows: ; in, This represents the sliding window wake-up flag. A value of 1 indicates wake-up, and a value of 0 indicates no wake-up. When both parentheses contain true, the condition is met. The value is 1 if it is set to 1, otherwise the value is 0. [ ] indicates an indicator function. The function takes the value 1 when the value inside the [ ] is 1, and takes the value 0 otherwise. Indicates the sliding window frame index. Indicates the length of the sliding window; This represents the frame count threshold and can be set to 3. This represents the vertical acceleration of the centroid of the point cloud cluster in frame t. It represents the acceleration due to gravity.

[0038] This application's embodiments acquire point cloud data in real time using millimeter-wave radar and accurately extract geometric and kinematic features using a point cloud spatiotemporal coding network. Combined with a multi-frame sliding window judgment mechanism, it achieves high-precision wake-up detection of human targets. Compared with traditional camera-based solutions, it has the advantages of being unaffected by lighting conditions and having strong privacy protection capabilities. Compared with single threshold judgment methods, through multi-dimensional feature fusion and continuous frame verification, it effectively reduces the false wake-up rate and improves the reliability and stability of detection. It can be widely applied to human presence perception and device wake-up systems in smart home, smart office, and other scenarios.

[0039] In some possible embodiments, a pre-set skeletal point extraction model is used to extract a set of human skeletal key points from the real-time image data; the angle between the human body and the horizontal plane is obtained based on the set of human skeletal key points, and visual confidence is obtained based on the angle and the target dynamic threshold parameter, including: A pre-set skeletal point extraction model is used to extract the set of human skeletal key points from the real-time image data; the set of human skeletal key points includes the coordinates of the head key points, the coordinates of the hip key points, the coordinates of the left ankle key points, and the coordinates of the right ankle key points. Based on the coordinates of the head key points and the hip key points, the angle between the human body and the horizontal plane is obtained. If the angle between the human body and the horizontal plane is less than the threshold value for the angle between the human body and the horizontal plane, then the visual confidence level is determined to be the first preset value; otherwise, the visual confidence level is determined to be the second preset value.

[0040] For example, the pre-set skeletal point extraction model can be the Mobilenetv3 model to extract key points of the human body. However, it is worth noting that other existing technologies can also be used as skeletal point extraction models.

[0041] After acquiring real-time image data via a camera, the set of key points for the human skeleton is extracted using a pre-set skeletal point extraction model: ; in, This represents the set of key points in the human skeleton. Indicates the coordinates of key points in the head. Indicates the coordinates of key points in the hip. Indicates the coordinates of key points on the left ankle. This indicates the coordinates of key points on the right ankle.

[0042] Based on the coordinates of the head key points and the hip key points, the angle between the human body and the horizontal plane is obtained as follows: ; in, This indicates the angle between the human body and the horizontal plane. This indicates the Y-axis coordinate of the hip key points in the pixel coordinate system. The pixel coordinate system is defined as follows: the origin is the top left corner of the real-time image data, the positive direction of the X-axis is horizontal to the right, and the positive direction of the Y-axis is vertical downward. The Y-axis coordinates of the key points in the head are represented. The X-axis coordinates of key points on the hip are represented. The X-axis coordinates of key points in the head; Based on the angle between the human body and the horizontal plane, the visual confidence level is obtained as follows: ; in, Indicates visual confidence level. This represents the threshold of the angle between the target human body and the horizontal plane in the target dynamic threshold parameters. Optionally, considering the existence of rest areas, machine vision can be used to detect resting devices (such as sofas or beds) that allow lying down in the camera image data. When an elderly person is detected to be on a resting device, the visual confidence level can be directly set to 0, thereby avoiding misidentifying the elderly person resting on the resting device as a fall.

[0043] This application embodiment extracts a set of key human skeletal points from real-time image data using a skeletal point extraction model, providing a data foundation for subsequent angle calculation and confidence level determination. Then, it determines the visual confidence level based on the angle between the human body and the horizontal plane, as well as the target dynamic threshold parameter. This method is highly adaptable and flexible, allowing for flexible setting of the visual confidence level judgment criteria by adjusting the threshold of the angle between the human body and the horizontal plane according to different application scenarios and needs, thereby better meeting diverse practical application requirements.

[0044] In some possible embodiments, millimeter-wave radar feature vectors, camera visual feature vectors, and temporal feature vectors are constructed based on the point cloud geometric feature vectors, point cloud kinematic feature vectors, included angles, and visual confidence, respectively. These feature vectors are then weighted and fused according to the target multimodal attention weights to obtain a fused feature vector, including: The point cloud geometric feature vector and the point cloud kinematic feature vector are used to construct a millimeter-wave radar feature vector; The included angle and visual confidence level are used to form a camera visual feature vector; The descent amplitude and the static time of the human body are obtained from the geometric feature vector of the point cloud, and then a vector is formed to obtain the temporal feature vector. The attention weights of millimeter-wave radar features, camera visual features, and temporal features are used to weight the millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector respectively, and then fused to obtain a fused feature vector.

[0045] For example, the feature vector of millimeter-wave radar can be: ; The camera's visual feature vector can be: ; Based on the geometric feature vectors of the point cloud, the descent amplitude and the time the human body remained stationary are obtained respectively, and then combined into a vector to obtain the temporal feature vector: ; in, Represents the temporal feature vector. This indicates the degree of descent of the human body, and ; This represents the centroid height of the point cloud cluster 5 frames ago. This represents the centroid height of the point cloud cluster in frame t; This represents the time the human body remains still; it is calculated by multiplying the number of consecutive frames in which the centroid displacement of the detected point cloud cluster is less than 0.05 or 0.1 meters by the frame interval Δt = 0.05 seconds. = ; This represents the number of consecutive frames that satisfy the displacement condition. Indicates the frame interval.

[0046] Therefore, the attention weights of millimeter-wave radar features, camera visual features, and temporal features are used to weight the millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector respectively, and then fused into: ; in, Represents the fused feature vector. This represents the target attention weights representing millimeter-wave radar features in the target multimodal attention weights. This represents the target attention weights in the target multimodal attention weights, which are the target attention weights based on camera visual features. The target attention weights represent the temporal features in the target multimodal attention weights; This indicates a zero-padding operation, which expands the dimension to 5 by padding the end of the input vector with zeros, thus making the millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector have the same dimension.

[0047] This application embodiment, by weighted fusion of millimeter-wave radar, camera vision, and temporal features, can fully leverage the advantages of different modal data, effectively improve the accuracy of human fall detection, reduce the misjudgment rate of single-modal data in complex environments, and provide a more reliable decision-making basis for subsequent fall warning and rescue response.

[0048] In some possible embodiments, a pre-set feature recognition model is used to analyze the fused feature vector to obtain the fall probability, and a fall determination result is determined based on the fall probability and the target dynamic threshold parameter. Finally, a fall classification warning is performed based on the fall determination result, including: The fused feature vector is used as input to the pre-set feature recognition model to obtain the fall probability; The fall determination result is determined based on a pre-set fall probability threshold range, fall probability, visual confidence level, and human center of gravity height threshold; the fall determination result includes fall, suspected fall, or no fall; Execute pre-set warnings based on fall assessment results to achieve graded fall warnings.

[0049] For example, the pre-set feature recognition model can be set as a gated recurrent unit network (GRU) or a backpropagation neural network. The hidden state update process and classification process of GRU are relatively conventional, and will not be described in detail in the embodiments of this application.

[0050] After the pre-set feature recognition model outputs the fall probability between [0,1], the fall determination result can be determined based on the fall probability: ; in, This indicates the result of the fall assessment. Indicates the probability of falling. For the pre-set first threshold, This represents a pre-set second threshold. However, it is worth noting that the first and second thresholds can be set to other values, as long as the first threshold is greater than the second threshold. Indicates other situations.

[0051] A pre-set warning strategy can be configured for each category: fall, suspected fall, and no fall. For example, when a fall is determined, the system immediately triggers a Level 1 warning: the local device emits a high-decibel audio-visual alarm to attract the attention of those nearby, and simultaneously sends an SMS or mobile application push notification containing the time, location, and real-time image snapshot of the fall to pre-bound emergency contacts (such as family members or community healthcare workers) via the IoT module. If the fall is suspected, a Level 2 warning is activated: the local device emits a low-volume alert and continuously collects millimeter-wave radar data and camera images for the next 3 seconds. If multiple consecutive frames still meet the suspected fall criteria (a specific frame threshold can be set), the warning is automatically upgraded to Level 1; otherwise, it is deactivated. For a no-fall determination, the system only records the feature data and determination result of the current frame to local storage for subsequent model iteration optimization and threshold parameter adjustment. Furthermore, the warning mechanism allows users to customize alarm methods and contact lists according to actual scenario needs. For example, in a nursing home setting, it can be linked to the monitoring system of the nursing station; in a home setting, it can be linked to a smart speaker for voice broadcasting, further enhancing the flexibility and practicality of the warning response. This application embodiment uses a tiered early warning mechanism to ensure timely response to fall incidents while avoiding excessive alarms for suspected cases, effectively balancing the sensitivity and accuracy of early warnings and providing more humane technical support for elderly safety monitoring.

[0052] In some possible embodiments, the method may further include: Respiratory rate and heart rate variability are extracted using the micro-Doppler effect of millimeter-wave radar, and enhanced early warning is provided based on the respiratory rate and heart rate variability.

[0053] For example, the respiratory rate can be extracted using the micro-Doppler effect of millimeter-wave radar: ; in, This indicates respiratory rate, measured in breaths per minute (breaths / min). This represents the respiratory fundamental frequency extracted from the phase changes of millimeter-wave radar echoes. The specific extraction method is as follows: Select a point in the point cloud corresponding to the human chest (Z coordinate within the range of 0.3 meters to 0.8 meters, and X and Y coordinates located in the torso region, defined as a circular area with a radius of 0.3 meters centered on the centroid of the point cloud). Extract the phase change sequence over a continuous 10 seconds. Perform a Fast Fourier Transform (FFT) on this phase change sequence; the frequency component with the largest amplitude in the spectrum is the respiratory frequency.

[0054] Heart rate variability was obtained as follows: ; in, Indicates heart rate variability. Indicates the first The heartbeat interval, measured in milliseconds (ms), is obtained by extracting the peak intervals of the heartbeat signal from millimeter-wave radar echoes. Specifically, the extraction method involves bandpass filtering the phase difference sequence of the point cloud echoes from the chest region (passband frequency range 0.8Hz to 3Hz, corresponding to heart rates of 48 to 180 beats per minute). After filtering, the heartbeat signal is obtained, and the peak value of the heartbeat signal is detected. The time interval between adjacent peak values ​​is the heartbeat interval. This represents the average value of L heartbeat intervals; Therefore, enhanced early warning can be provided based on the respiratory rate and heart rate variability: ; Among them, AlertLevel represents the enhanced warning level. A value of 0 means no enhanced warning is issued; a value of 1 means a Level 1 enhanced warning is issued; a value of 2 means a Level 2 enhanced warning is issued; and a value of 3 means a Level 3 warning is issued.

[0055] For different enhanced warning levels, the system executes corresponding refined response strategies: When AlertLevel=1, based on the original Level 1 warning, a continuous vital sign monitoring mode is activated, updating respiratory rate and heart rate variability data every 0.5 seconds and simultaneously uploading it to the cloud server for storage; when AlertLevel=2, while sending emergency contact notifications, the system automatically pushes details of the fall incident and current vital sign data to the emergency medical center platform in the jurisdiction, requesting professional medical personnel to intervene and assess the situation; when AlertLevel=3, the highest priority response is immediately triggered. In addition to the above actions, the local device activates the emergency call function, guiding nearby personnel to perform basic first aid measures (such as keeping the patient supine and checking breathing) through voice prompts. At the same time, the cloud platform links with the 120 emergency medical system to automatically generate an emergency dispatch instruction containing precise location information.

[0056] Furthermore, to enhance the long-term reliability and adaptability of the system, this method also supports a self-optimization mechanism for dynamic thresholds and model parameters: the system periodically (e.g., weekly) summarizes and analyzes historical fall event data, suspected fall misjudgment records, and user feedback information, readjusting target dynamic threshold parameters (e.g., the threshold for the angle between the human body and the horizontal plane, and the threshold for the height of the human center of mass) and multimodal attention weights, enabling the model to better adapt to different users' physical characteristics (e.g., height, weight) and scene changes (e.g., lighting conditions, furniture layout). Simultaneously, the system provides an open API interface, allowing third-party platforms (e.g., nursing home management systems, home health monitoring platforms) to access fall monitoring data, achieving cross-system data sharing and collaborative response, further expanding the coverage of application scenarios.

[0057] In practical deployments, the system also features a low-power operating mode: when the millimeter-wave radar detects no human activity in the area for more than 30 minutes, it automatically switches to sleep mode, retaining only the radar's passive monitoring function, while the camera stops collecting data to reduce energy consumption; once the radar detects human movement, it immediately wakes up the camera to enter normal operating mode, ensuring that energy saving does not affect the real-time monitoring. This low-power design is particularly suitable for home scenarios, effectively extending device battery life and reducing user maintenance costs.

[0058] The system also supports user-defined configuration: users can adjust warning thresholds (such as the first and second thresholds for the probability of falling), select warning methods (such as whether to turn on the sound and light alarm, whether to link with smart devices), and set the sensitivity of vital sign monitoring through the mobile application, so that the system can be personalized according to individual needs and usage habits, further improving the user experience and monitoring effect.

[0059] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. It should be understood that the invention is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for detecting and warning of falls in the elderly based on millimeter-wave radar and a camera, characterized in that, include: Collect daily activity data of the target elderly and construct a labeled dataset based on the daily activity data; The labeled dataset includes sample data and the corresponding labels for the sample data; Candidate hyperparameter vectors are constructed, and based on the sample data and the corresponding labels, the candidate hyperparameter vectors are optimized using an error function that fuses fall detection accuracy, false alarm rate, and alarm delay rate, as well as an improved gray wolf optimization algorithm, to obtain a target hyperparameter vector; the target hyperparameter vector includes at least target multimodal attention weights and target dynamic threshold parameters; Real-time point cloud data is acquired using millimeter-wave radar. Features are extracted from the real-time point cloud data using a pre-set point cloud spatiotemporal coding network to obtain point cloud geometric feature vectors and point cloud kinematic feature vectors. Based on the point cloud geometric feature vectors, point cloud kinematic feature vectors, and target dynamic threshold parameters, single-frame wake-up information is obtained, and a sliding window wake-up flag is obtained based on the single-frame wake-up information. The sliding window wake-up flag includes a wake-up flag or a non-wake-up flag. When the sliding window wake-up flag is set to wake-up, the camera is woken up to perform real-time image acquisition, obtain real-time image data, and use a pre-set skeletal point extraction model to extract the set of human skeletal key points in the real-time image data; obtain the angle between the human body and the horizontal plane based on the set of human skeletal key points, and obtain the visual confidence level based on the angle and the target dynamic threshold parameter. Based on the point cloud geometric feature vector, point cloud kinematic feature vector, included angle, and visual confidence, millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector are constructed respectively. The millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector are then weighted and fused according to the target multimodal attention weight to obtain a fused feature vector. The fused feature vector is analyzed using a pre-set feature recognition model to obtain the fall probability. The fall determination result is determined based on the fall probability and the target dynamic threshold parameter. Finally, a fall classification warning is given based on the fall determination result, realizing fall monitoring and warning for the elderly based on millimeter-wave radar and camera.

2. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 1, characterized in that, Collect daily activity data of the target elderly, and construct a labeled dataset based on the daily activity data, including: The system collects millimeter-wave radar data and camera image data of elderly people in non-fall states, as well as millimeter-wave radar data and camera image data of elderly people in simulated fall and / or fall states. Using millimeter-wave radar data and camera image data as sample data, non-fall states are constructed as non-fall labels, and simulated falls and / or fall states are constructed as fall labels, thus obtaining the labeling labels corresponding to the sample data.

3. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 1, characterized in that, Candidate hyperparameter vectors are constructed, and based on the sample data and the corresponding labels, the candidate hyperparameter vectors are optimized using an error function fused from fall detection accuracy, false alarm rate, and alarm delay rate, along with an improved gray wolf optimization algorithm, to obtain the target hyperparameter vector, including: Construct candidate hyperparameter vectors; the candidate hyperparameter vectors include attention weights for millimeter-wave radar features, attention weights for camera visual features, attention weights for temporal features, human centroid height threshold, instantaneous velocity change threshold, and human body angle threshold with respect to the horizontal plane. By repeatedly constructing multiple different candidate hyperparameter vectors, multiple individual gray wolves can be obtained; For any individual gray wolf, based on the sample data and the corresponding labels, the loss function value for each individual gray wolf is obtained by fusing the fall detection accuracy, false alarm rate and alarm delay rate using an error function. Based on the loss function value, the improved gray wolf optimization algorithm is used to optimize the individual gray wolves, resulting in optimized gray wolf individuals. Repeat the optimization steps for individual gray wolves until the maximum number of training iterations is reached, and obtain the target hyperparameter vector.

4. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 3, characterized in that, For any individual gray wolf, based on the sample data and its corresponding labels, a loss function value is obtained for each individual gray wolf using an error function that fuses fall detection accuracy, false alarm rate, and alarm delay rate. This includes: For any individual gray wolf, based on the sample data, the probability of falling is obtained, and based on the probability of falling and the label corresponding to the sample data, the accuracy of fall detection, false alarm rate and alarm delay rate are obtained. Based on the fall detection accuracy, false alarm rate, and alarm delay rate, the loss function value corresponding to the individual gray wolf is obtained as follows: ； in, This refers to an individual gray wolf. This represents the loss function value of an individual gray wolf. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the third weighting coefficient, and + + =1; This indicates the accuracy rate of fall detection. Indicates the false alarm rate. Indicates alarm delay rate, This indicates the system delay time from the occurrence of a fall to the time the system alarm is triggered. Indicates the maximum allowed delay time.

5. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 3, characterized in that, Based on the loss function value, the improved gray wolf optimization algorithm is used to optimize the individual gray wolves, resulting in optimized gray wolf individuals, including: Based on the loss function value, the individual gray wolves are classified into individual identities to obtain optimized gray wolf individuals after individual identity classification; Employing improved prey encirclement strategies and introducing The gray wolf optimization algorithm based on the wolf neighborhood adaptive exploration strategy optimizes gray wolf individuals after individual identity classification, and obtains optimized gray wolf individuals.

6. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 3, characterized in that, Real-time point cloud data is acquired using millimeter-wave radar. Feature extraction is performed on the real-time point cloud data using a pre-set point cloud spatiotemporal coding network to obtain point cloud geometric feature vectors and point cloud kinematic feature vectors. Based on the point cloud geometric feature vectors, point cloud kinematic feature vectors, and target dynamic threshold parameters, single-frame wake-up information is obtained, and a sliding window wake-up flag is obtained based on the single-frame wake-up information, including: Real-time point cloud data is acquired using millimeter-wave radar; the real-time point cloud data includes the three-dimensional spatial coordinates of the point cloud, the radial velocity of the point cloud, and the signal-to-noise ratio. Using the real-time point cloud data as input, a point cloud geometric feature vector consisting of the centroid height of the point cloud cluster, the width ratio of the cloud cluster, and the rate of change of the ground projection area of ​​the point cloud cluster is obtained through a pre-set point cloud spatiotemporal coding network. Additionally, a point cloud kinematic feature vector consisting of the instantaneous horizontal velocity of the centroid of the point cloud cluster and the acceleration of the centroid of the point cloud cluster in the vertical direction is obtained. If the height of the centroid of the point cloud cluster is less than the threshold of the height of the human centroid and the instantaneous horizontal velocity of the centroid of the point cloud cluster is greater than the threshold of the instantaneous velocity change, then the single-frame wake-up information is determined to be 1; otherwise, the single-frame wake-up information is determined to be 0. If the single-frame wake-up information is 1 for three consecutive frames and the acceleration of the cloud cluster centroid in the vertical direction is greater than a preset threshold, then the sliding window wake-up flag is determined to be a wake-up flag; otherwise, the sliding window wake-up flag is determined to be a non-wake-up flag.

7. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 3, characterized in that, A pre-set skeletal point extraction model is used to extract the set of key points of the human skeleton in the real-time image data; The angle between the human body and the horizontal plane is obtained based on the set of key points of the human skeleton, and visual confidence is obtained based on the angle and the target dynamic threshold parameter, including: A pre-set skeletal point extraction model is used to extract the set of human skeletal key points from the real-time image data; the set of human skeletal key points includes the coordinates of the head key points, the coordinates of the hip key points, the coordinates of the left ankle key points, and the coordinates of the right ankle key points. Based on the coordinates of the head key points and the hip key points, the angle between the human body and the horizontal plane is obtained. If the angle between the human body and the horizontal plane is less than the threshold value for the angle between the human body and the horizontal plane, then the visual confidence level is determined to be the first preset value; otherwise, the visual confidence level is determined to be the second preset value.

8. The method for monitoring and early warning of falls in the elderly based on millimeter-wave radar and camera according to claim 1, characterized in that, Based on the point cloud geometric feature vector, point cloud kinematic feature vector, included angle, and visual confidence, millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector are constructed respectively. These feature vectors are then weighted and fused according to the target multimodal attention weight to obtain a fused feature vector, including: The point cloud geometric feature vector and the point cloud kinematic feature vector are used to construct a millimeter-wave radar feature vector; The included angle and visual confidence level are used to form a camera visual feature vector; The descent amplitude and the static time of the human body are obtained from the geometric feature vector of the point cloud, and then a vector is formed to obtain the temporal feature vector. The attention weights of millimeter-wave radar features, camera visual features, and temporal features are used to weight the millimeter-wave radar feature vector, camera visual feature vector, and temporal feature vector respectively, and then fused to obtain a fused feature vector.

9. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 7, characterized in that, The fused feature vector is analyzed using a pre-set feature recognition model to obtain the fall probability. A fall determination result is then determined based on the fall probability and the target dynamic threshold parameter. Finally, a graded fall warning is issued based on the fall determination result, including: The fused feature vector is used as input to the pre-set feature recognition model to obtain the fall probability; The fall determination result is determined based on a pre-set fall probability threshold range, fall probability, visual confidence level, and human center of gravity height threshold; the fall determination result includes fall, suspected fall, or no fall; Execute pre-set warnings based on fall assessment results to achieve graded fall warnings.

10. The method for monitoring and early warning of elderly falls based on millimeter-wave radar and camera according to claim 1, characterized in that, Also includes: Respiratory rate and heart rate variability are extracted using the micro-Doppler effect of millimeter-wave radar, and enhanced early warning is provided based on the respiratory rate and heart rate variability.

Images