Cooperative tumble detection method based on Bayesian cognitive routing and space-time rhythm prior

Abstract

The invention relates to the technical field of intelligent security and protection and intelligent old-age care monitoring, in particular to a collaborative tumble detection method based on Bayesian cognitive routing and spatio-temporal rhythm prior, which comprises the following steps: a dual-mode visual acquisition unit acquires a video stream; constructing a multi-mode tumble detection model, namely extracting video features, preliminary behavior probability distribution and statistical features from a video stream by utilizing a lightweight time sequence sensing network, calculating rhythm surprise by utilizing a neural state space rhythm model, calculating a routing activation probability by utilizing a Bayesian gating network and judging whether to intercept or not, if so, triggering the visual language large model; constructing a training mechanism of teacher-student alignment until the network converges; and outputting an identification result and executing a modal isolation data response strategy. The method has the beneficial effects that a'lightweight perception-large model rechecking 'bionic architecture is constructed, the detection precision is improved, the problems of infrared night vision and shielding are solved, false alarm and missing alarm caused by similar postures are reduced, and family privacy protection is also realized.

Description

Technical Field

[0001] This invention relates to the fields of intelligent security and smart elderly care monitoring technology, and in particular to a collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm priors. Background Technology

[0002] Fall detection is a core task in smart elderly care and home security in an aging society, and its accuracy and response speed are directly related to the life safety of those under guardianship. Although various fall detection solutions exist in academia and industry, a review and analysis of existing technologies reveals that mainstream methods still face insurmountable technical bottlenecks in complex real-world home applications, such as lack of robustness, difficulty in resolving semantic ambiguities, and long-term perception blind spots.

[0003] Wearable fall detection methods rely on contact sensors such as accelerometers and gyroscopes to collect motion data and match fall signal waveforms through thresholding or shallow machine learning (such as SVM). The fundamental defects of this type of method are: (1) a blind spot in the perception of "non-violent emergency situations". For "slow slips" common in the elderly, such as slowly sliding down a wall or "pathological flaccid paralysis", due to the lack of significant impact signals, they are easily filtered out by the algorithm, resulting in missed detections; (2) a serious "semantic gap". The sensor can only perceive physical signals and lacks understanding of environmental semantics. It is easily interfered with by high-frequency daily actions such as "quickly sitting down" and "the bracelet slipping off", resulting in a very high false alarm rate; (3) this scheme relies heavily on the wearing compliance of the elderly. During high-risk periods such as resting at night or taking a bath, users often take off the device, making the monitoring ineffective.

[0004] The skeletal key point detection method relies on algorithms such as YOLO-Pose and OpenPose to extract human skeletal key points through video surveillance, and then calculates joint angle or speed changes. However, it has serious anti-interference defects in non-ideal environments: (1) Occlusion failure: In the home environment, furniture occlusion or body self-occlusion (such as curling up and falling to the ground) is common. Once key parts such as legs or spine are occluded, the skeletal point extraction network often outputs incorrect coordinates or directly drops frames, causing the detection logic to collapse; (2) Cross-domain generalization failure: When night monitoring switches to infrared mode, image texture and color information are severely lost. The skeletal extraction network trained on the RGB dataset shows a sharp decline in performance in the infrared domain, causing key points to drift or disappear, making this type of technology unable to work during high-risk nighttime periods.

[0005] The "spatiotemporal rhythm blind spot" based on deep learning models suffers from a serious "temporal shortsightedness" problem when relying on recognition technologies such as I3D and SlowFast through video surveillance, resulting in the inability to handle long-tail semantic ambiguity: (1) It cannot identify "static emergency state". Existing models can usually only process short video clips of a few seconds and can only identify "the posture at the current moment", with a high false alarm rate; (2) The computational power wall of long sequence modeling. If the model is to "remember" the user's behavior pattern in the past week to establish a personalized benchmark, the self-attention mechanism of the mainstream Transformer architecture faces the bottleneck of quadratic computational complexity. On edge devices with limited computing power, it is impossible to perform real-time reasoning on historical sequences of tens of thousands of frames. This forces existing technologies to abandon the modeling of "long-cycle living habits" and remain at the stage of "only looking at the present and ignoring the past". Summary of the Invention

[0006] To overcome the aforementioned shortcomings, this invention aims to provide a collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm priors. By constructing a lightweight perception-large model kernel biomimetic architecture and introducing a neural state space model with linear computational complexity, it solves the problem of cross-domain recognition between day and night. Under the edge-end collaborative architecture, it balances computing power and accuracy, reduces false alarm rate, and achieves privacy protection by ensuring that the original video does not leave the domain.

[0007] This invention achieves the above objective through the following scheme: a collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm priors, comprising the following steps:

[0008] (1) The dual-mode vision acquisition unit acquires the video stream;

[0009] (2) Construct a multimodal fall detection model, including:

[0010] (2.1) Extracting video features from video streams using a lightweight temporal-aware network Output the initial behavioral probability distribution And calculate the statistical features characterizing the cognitive uncertainty of the model, including prediction entropy E and confidence boundary M;

[0011] (2.2) The user's historical behavior sequence is modeled using a neural state-space rhythm model, and the rhythmic surprise of the current behavior relative to the historical pattern is calculated. ;

[0012] (2.3) Using Bayesian gating networks combined with video features Statistical characteristics and rhythmic surprise Generate route activation probability And based on the route activation probability Dynamically determine whether to intercept the current sample: if it is determined not to intercept, output the classification result of the lightweight temporal perception network; if it is determined to intercept, trigger the large visual language model to perform a review and inference on the current sample.

[0013] (3) Construct a “teacher-student alignment” training mechanism, freeze the parameters of the lightweight temporal perception network and the visual language large model, use the offline dataset to perform binary classification supervision training on the Bayesian gated network, balance positive and negative samples by weighted sampling until the network converges, and obtain the trained Bayesian gated network.

[0014] (4) Input the real-time video stream into the trained multimodal fall detection model, output the recognition result, and execute the modal isolation data response strategy based on the recognition result.

[0015] Preferably, the video stream is acquired by a dual-mode visual acquisition unit, which includes a visible light mode and an infrared night vision mode. The visible light mode acquires RGB images, and the infrared night vision mode acquires single-channel grayscale images.

[0016] Preferably, step (2.1) specifically includes the following steps:

[0017] (2.1.1) Perform sliding window sampling on the video stream to obtain the video sequence. ;

[0018] (2.1.2) Video sequence The input is a pre-trained lightweight spatiotemporal feature extraction network. This network employs a video encoder based on a weakly supervised vision-language pre-training architecture and undergoes data augmentation training including random grayscale to extract high-dimensional spatiotemporal feature vectors across the illumination domain. Softmax is used to generate a preliminary behavioral probability distribution. ;

[0019] (2.1.3) Based on the preliminary behavioral probability distribution The statistical characteristics of the uncertainty index predicted by the parallel computing model are characterized, including the prediction entropy. and confidence boundary ,in The first in the preliminary behavioral probability distribution dimensional components, and These represent the values ​​of the maximum probability and the second highest probability in the initial behavior probability distribution, respectively.

[0020] Preferably, step (2.2) specifically includes the following steps:

[0021] (2.2.1) Treat the user's historical behavior tag stream as a discrete time series Introducing time scale parameters Based on the zero-order hold principle, the parameters of continuous systems are... Convert to discrete parameters :

[0022]

[0023] ,

[0024] Where I is the identity matrix;

[0025] (2.2.2) Real-time update of hidden behavioral states using discretized state equations And based on the hidden state Predict the probability distribution of prior actions at the current moment. ;

[0026] (2.2.3) Rhythmic Surprise Preliminary behavioral probability distribution With prior behavior probability distribution The Kourbak-Leibler divergence between them is calculated using the following formula: .

[0027] Preferably, step (2.3) specifically includes the following steps:

[0028] (2.3.1) Prediction entropy is obtained through positional encoding Confidence Boundary and rhythmic surprise Mapped to a low-dimensional embedding vector, then combined with video features Concatenation yields the enhanced input vector ,in These are weighting coefficients;

[0029] (2.3.2) Keep MC Dropout enabled in the last fully connected layer of the lightweight temporal-aware network, perform T random forward propagations on the same input, and obtain the distribution set. Calculate the variance of cognitive uncertainty ,in The L2 norm of a vector This represents the Softmax probability score obtained during the i-th forward propagation. This represents the probability distribution of the mean of T predictions;

[0030] (2.3.3) Construct a multilayer perceptron containing a normalization layer and an activation function as a Bayesian gated network, and calculate the route activation probability. :

[0031]

[0032]

[0033] in, It is a learnable weight matrix of a two-layer MLP. It is Batch Normalization, used for feature standardization; It is the Rectified Linear Unit activation function; Logit is the log-probability of routes that have not passed through the Sigmoid function, output by the MLP. For the Sigmoid function, This is the visual uncertainty compensation coefficient. This is the rhythm surprise compensation coefficient;

[0034] (2.3.4) Set threshold Based on route activation probability Dynamically determine whether to block the current sample:

[0035] like If the sample is identified as high-risk, proceed to step (2.3.5).

[0036] like If a sample is identified as low-risk, the classification result of a lightweight temporal-aware network is output.

[0037] (2.3.5) Intercept and trigger the visual large language model review, specifically including the following steps:

[0038] (2.3.5.1) The quantized visual large language model weights are preloaded into the local video memory of the edge device and kept in a hot resident state to wait for inference instructions;

[0039] (2.3.5.2) Use image processing algorithms to traverse the video sequence, superimpose a prominent red number at a fixed position in the pixel layer of each frame, and stitch the images after superimposing the numbers into a high-resolution temporal grid diagram in time sequence, and achieve explicit alignment of spatiotemporal information at the input end.

[0040] (2.3.5.3) Perform cross-modal reasoning locally and construct a general structured thought chain prompt template. The prompt template includes the generation of the final judgment text containing the reasoning process according to the changes in the posture and position of the person in the time-series grid diagram, and reserves an environmental constraint slot in the preceding position of the template. The system detects the domain distribution characteristics of the video stream. If it is determined to be a standard visible light mode, the slot remains empty and the model performs general reasoning. If it is determined to be an infrared night vision mode, environmental correction instructions are dynamically injected into the slot to generate an accurate judgment.

[0041] Preferably, step (3) specifically includes the following steps:

[0042] (3.1) Constructing an asymmetric distillation inference environment: Define an offline video dataset including daytime, nighttime infrared, and occluded multimodal scenes; define a lightweight temporal perception network as the student model, a large visual language model as the teacher model, and real labels as... Freeze all weight parameters of the student model and the teacher model, and only update the gradient of the Bayesian gated network; use the student model and the teacher model to perform full dual-stream inference on the offline video dataset, and obtain their respective prediction class distribution results and confidence scores.

[0043] (3.2) Define positive sample data for routing: filter out the set of sample data that the student model cannot process but the teacher model can correct. The sample is labeled as 1, indicating that it belongs to the sample data that needs to be intercepted;

[0044] (3.3) Define invalid or routed negative sample data: filter out the sample data set that does not require the intervention of the teacher model. The sample label is marked as 0, which means that the sample data does not need to be intercepted;

[0045] (3.4) Class balancing and parameter optimization: Introducing dynamic weights ,use Calculate the loss and backpropagate the gradient, adjusting the focusing parameters. Optimize the in-distribution samples at the decision boundary, where For category weights, To focus parameters, This represents the current number of positive samples. This represents the current number of negative samples. This represents the total number of samples in the current category.

[0046] Preferably, the modal isolation data response strategy includes physical blocking and semantic firewall; the physical blocking is to block the video stream outgoing port of the camera and store the original video locally in a closed loop; the semantic firewall is to only allow the local visual large language model to generate de-identified semantic information metadata and encrypt and upload it to the cloud.

[0047] As a preferred approach, the collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm priors also includes a hierarchical semantic response and a human-machine co-training mechanism. The hierarchical semantic response is: the system dynamically adjusts the generation strategy of the cloud-based large model (LLM API) according to the confidence and risk level of fall detection. The human-machine co-training mechanism is: a weakly supervised update mechanism based on federated learning is constructed, and specific real samples that are false positives or false negatives are marked as high-value difficult examples and stored in a dynamic retraining pool, and the Bayesian gated network is periodically fine-tuned online.

[0048] Preferably, T is 5 times.

[0049] Preferably, the classification results of the lightweight temporal sensing network include the probability distribution of 12 types of actions, of which six are falls and six are normal daily actions.

[0050] The beneficial effects of this invention are as follows: This invention constructs a biomimetic architecture of "lightweight perception-large model verification". Through the Bayesian cognitive routing mechanism, it effectively balances edge computing resources and detection accuracy. In particular, it utilizes "prediction variance" to keenly perceive OOD scenarios such as infrared night vision, adaptively triggering large model verification. By designing environmental correction instructions, it uses powerful semantic reasoning capabilities to solve the problems of infrared night vision and occlusion, and solves the texture dependency problem of large models in infrared mode. This biomimetic architecture avoids the computing power bottleneck of a single large model and solves the lack of generalization of lightweight models, achieving a perfect balance between accuracy and efficiency. The neural state space model with linear computational complexity introduced in this invention achieves lossless modeling of user behavior sequences over several weeks at the edge, effectively distinguishing between "rest" (low surprise) that conforms to the biological clock and "abnormal lingering" (high surprise) that violates life rhythms, effectively reducing false alarms and false negatives caused by similar postures. This invention innovatively proposes a "modal isolation" privacy protection strategy, solving family privacy concerns. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the steps of the method of the present invention;

[0052] Figure 2 This is a schematic diagram of the construction process of the multimodal fall detection model of the present invention;

[0053] Figure 3 This is a dynamic relationship diagram between rhythm surprise and routing interception threshold in one embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of a timing mesh diagram in one embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of an offline training process based on "teacher-student alignment" in one embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the edge-end collaborative architecture and the "modal isolation" privacy mechanism of the present invention. Detailed Implementation

[0057] The present invention will be further described below with reference to specific implementation examples, but the scope of protection of the present invention is not limited thereto:

[0058] Example: This example uses a dual-mode visual acquisition unit with automatic day and night switching function as the system front end. The unit has a built-in photosensitive sensor. When the ambient light intensity is lower than a certain threshold of 0.1 Lux, it automatically switches from RGB visible light mode to NIR infrared night vision mode.

[0059] Visible light domain: Video streams contain rich RGB colors and texture details, such as clothing colors and facial features. Lightweight networks typically exhibit high confidence and low uncertainty in this domain.

[0060] Infrared night vision: The video stream degrades to a single-channel grayscale image, color information is completely lost, clothing textures are blurred, and only the geometric contours of the human body edges and skeleton are retained. This drastic change in data distribution can lead to cognitive dissonance in lightweight models pre-trained only on RGB datasets, manifested as increased prediction entropy and drastic fluctuations in prediction variance.

[0061] For the two image data modalities mentioned above, this invention designs a collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm priors, specifically designed to capture cognitive uncertainty in infrared mode, such as... Figure 1 As shown, RGB images and single-channel grayscale images are first processed into a multimodal video stream using a lightweight temporal-aware network and a neural state-space rhythm model. Then, a Bayesian gating network is used to determine whether to perform a review. The specific steps include:

[0062] (1) The dual-mode vision acquisition unit acquires the video stream;

[0063] (2) Construct a multimodal fall detection model, such as Figure 2 As shown, it includes:

[0064] (2.1) Extracting video features from video streams using a lightweight temporal-aware network Output the initial behavioral probability distribution The statistical features that characterize the cognitive uncertainty of the model are calculated, including prediction entropy E and confidence boundary M.

[0065] This embodiment uses a lightweight temporal-aware network (ActionCLIP) architecture based on ViT (Vision Transformer) as the visual backbone, specifically including the following steps:

[0066] (2.1.1) Perform sliding window sampling on the video stream, Window Size=8, Stride=4, to obtain the video sequence. Where H and W represent the height H and width W of the video frame, respectively, and 3 represents the RGB three channels;

[0067] (2.1.2) Video sequence The input is a pre-trained lightweight spatiotemporal feature extraction network. This network employs a video encoder based on a weakly supervised vision-language pre-training architecture, undergoes data augmentation training including random grayscale, and takes the output of the last layer of the Transformer, CLS Token, with dimensions... Extracting high-dimensional spatiotemporal feature vectors across illumination domains Softmax is used to generate a preliminary behavioral probability distribution. ;

[0068] (2.1.3) Based on the preliminary behavioral probability distribution The statistical characteristics of the uncertainty index predicted by the parallel computing model are characterized, including the prediction entropy. and confidence boundary ,in The first in the preliminary behavioral probability distribution The dimension component represents the dispersion of the distribution. and These represent the values ​​of the maximum and second highest probabilities in the initial behavioral probability distribution, respectively, which are the prediction confidence scores of the most likely and second most likely categories of the model.

[0069] (2.2) The user's historical behavior sequence is modeled using a neural state-space rhythm model, and the rhythmic surprise of the current behavior relative to the historical pattern is calculated. .

[0070] This embodiment utilizes the linear complexity of the Mamba architecture to solve the problem of long-period rhythm modeling, specifically including the following steps:

[0071] (2.2.1) Treat the user's historical behavior tag stream as a discrete time series Introducing time scale parameters Based on the zero-order hold principle, the parameters of continuous systems are... Convert to discrete parameters :

[0072]

[0073] ,

[0074] Where I is the identity matrix, i.e., the diagonal is 1 and the rest are 0, which is used to represent the identity operator in the discretization derivation;

[0075] (2.2.2) Real-time update of hidden behavioral states using discretized state equations And based on the hidden state Predict the probability distribution of prior actions at the current moment. ;

[0076] (2.2.3) Rhythmic Surprise Preliminary behavioral probability distribution With prior behavior probability distribution The Kourbak-Leibler divergence between them is calculated using the following formula: .

[0077] The value quantifies the degree of anomalousness of the current behavior: when A surge in activity indicates a violation of the user's long-term lifestyle patterns, and the system will dynamically adjust the routing bias based on this. For example... Figure 3 As shown, an abnormal fall occurred at 3 AM, with a rhythmic surprise level. The value spikes, the threshold drops, and the visual language big model is forced to undergo a review and inference.

[0078] (2.3) Using Bayesian gating networks combined with video features Statistical characteristics and rhythmic surprise Generate route activation probability And based on the route activation probability Dynamically determine whether to intercept the current sample: if it is determined not to intercept, output the classification result of the lightweight temporal perception network; if it is determined to intercept, trigger the visual language large model to perform verification reasoning on the current sample.

[0079] In this embodiment, the classification results of the lightweight temporal sensing network include the probability distribution of 12 types of actions, of which six are falls and six are normal daily actions.

[0080] Bayesian gated networks are not simply classifiers, but are constructed as uncertain rectified perceptrons, specifically including the following steps:

[0081] (2.3.1) Due to It is a high-dimensional vector of 512 dimensions, and As scalars, direct concatenation would obscure statistical features; therefore, positional encoding is used to reduce the prediction entropy. Confidence Boundary and rhythmic surprise Mapped to a low-dimensional embedding vector, then combined with video features Concatenation yields the enhanced input vector ,in These are the weighting coefficients.

[0082] (2.3.2) Keep MC Dropout enabled in the last fully connected layer of the lightweight temporal-aware network, with a dropout rate of 0.2, and then perform 5 random forward propagations on the same input to obtain the distribution set. Calculate the variance of cognitive uncertainty ,in The L2 norm of a vector This represents the Softmax probability score obtained during the i-th forward propagation. Let T represent the probability distribution of the mean of the predictions.

[0083] Cognitive uncertainty variance When used for detecting out-of-distribution (OOD) samples, such as in infrared night vision or severely occluded scenarios, even small perturbations in the model's weight parameters can cause drastic oscillations in the output. Significantly increased.

[0084] (2.3.3) Construct a multilayer perceptron containing a normalization layer and an activation function as a Bayesian gated network, and calculate the route activation probability. :

[0085]

[0086]

[0087] in, It is a learnable weight matrix of a two-layer MLP. It is Batch Normalization, used for feature standardization; It is the Rectified Linear Unit activation function; Logit is the log-probability of routes that have not passed through the Sigmoid function, output by the MLP. For the Sigmoid function, This is the visual uncertainty compensation coefficient. This is the rhythm surprise compensation coefficient.

[0088] (2.3.4) Set threshold Based on route activation probability Dynamically determine whether to block the current sample:

[0089] like If the sample is identified as high-risk, proceed to step (2.3.5).

[0090] like If a sample is identified as low-risk, the classification result of a lightweight temporal-aware network is output.

[0091] Router activation probability The calculation formula shows that even video features The probability of falling is inherently low, but if the variance... Extremely high (unseen) or rhythmic surprise Extremely high (abnormal behavior), the compensation item will be forcibly increased. This triggers the intervention of a large model, thereby avoiding missed reports.

[0092] (2.3.5) Intercept and trigger the visual large language model review, specifically including the following steps:

[0093] (2.3.5.1) The quantized visual large language model weights are preloaded into the local video memory of the edge device and kept in a hot resident state to wait for inference instructions;

[0094] (2.3.5.2) Traverse the video sequence using image processing algorithms ,like Figure 4 As shown, prominent red numbers 1-8 are superimposed at fixed positions in the pixel layer of each frame, and the eight frames of images after superimposing the numbers are stitched together in time sequence into a high-resolution temporal grid map, and the spatiotemporal information is explicitly aligned at the input end; this stitching method enables the visual big language model to read spatial features (person's posture) and temporal features (number changes) simultaneously in the form of a single large image.

[0095] (2.3.5.3) Perform cross-modal inference locally and construct a general structured thought chain prompt template. The prompt template includes a final judgment text containing the inference process generated according to the changes in the posture and position of the person in the temporal mesh diagram, and reserves an environmental constraint slot in the preceding position of the template. The system detects the domain distribution characteristics of the video stream. If it is determined to be a standard visible light mode, the slot remains empty and the model performs general inference. If it is determined to be an infrared night vision mode, an environmental correction instruction is dynamically injected into the slot to generate an accurate judgment. In one embodiment, the environmental correction instruction is: "Currentinput is Infrared Night Vision. Ignore texture loss. Focus strictly on skeletal geometry and posture changes." This instruction significantly improves the robustness of night detection.

[0096] (3) Construct a “teacher-student alignment” training mechanism, freeze the parameters of the lightweight temporal perception network and the visual language large model, use the offline dataset to perform binary classification supervision training on the Bayesian gated network, balance the positive and negative samples through weighted sampling until the network converges, and obtain the trained Bayesian gated network.

[0097] like Figure 5 As shown, this training mechanism trains a Bayesian gating network through hard example mining, specifically including the following steps:

[0098] (3.1) Constructing an asymmetric distillation inference environment: Define an offline video dataset including daytime, nighttime infrared, and occluded multimodal scenes; define a lightweight temporal perception network as the student model, a large visual language model as the teacher model, and real labels as... Freeze all weight parameters of the student and teacher models, and only update the gradient of the Bayesian gated network; perform full two-stream inference on the offline video dataset using both the student and teacher models respectively, and obtain the predicted class distribution results from the student model. With confidence level The teacher model obtains the predicted category distribution results. ;

[0099] (3.2) Define positive sample data for routing: filter out the set of sample data that the student model cannot process but the teacher model can correct. The judgment logic is as follows: the sample label is marked as 1 if and only if the student model predicts incorrectly and the teacher model predicts correctly, or if the prediction confidence of the student model is lower than the preset safety threshold. This is considered a sample data that needs to be intercepted.

[0100] (3.3) Define invalid or routed negative sample data: filter out the sample data set that does not require the intervention of the teacher model. The specific judgment logic is as follows: when the student model predicts correctly (simple sample), or when both the student model and the teacher model predict incorrectly (judged as invalid dirty data such as completely black or extremely blurry data, which is also ineffective when a large model is involved), the sample label is marked as 0, which is a sample data that does not need to be intercepted.

[0101] (3.4) Class balancing and parameter optimization: In real-world scenarios Training is prone to getting stuck in local optima. This invention introduces dynamic weights. ,use Calculate the loss and backpropagate the gradient, adjusting the focusing parameters. Optimize the in-distribution samples at the decision boundary, where For category weights, To focus parameters, This represents the current number of positive samples. This represents the current number of negative samples. This represents the total number of samples in the current category.

[0102] By utilizing a weighted random sampler to reorganize the training batch to address the problem of scarce positive samples, the gradient contribution of positive samples (difficult examples) is significantly increased, forcing the Bayesian gating network to focus on identifying hidden samples that, although the confidence level is acceptable, have actually been misclassified.

[0103] (4) Input the real-time video stream into the trained multimodal fall detection model, output the recognition result, and execute the modal isolation data response strategy based on the recognition result.

[0104] like Figure 6 As shown, the modal isolation data response strategy includes physical blocking and semantic firewall.

[0105] The physical blocking is achieved by the edge gateway configuring network policies via iptables to block the outgoing ports of the camera's RTSP / RTMP video streams, ensuring that the original video data does not leave the domain at the physical layer.

[0106] The semantic firewall allows only the upload of de-identified semantic information metadata generated by the local visual large language model through an HTTPS encrypted channel.

[0107] In one embodiment, the original metadata uploaded by the semantic firewall is {"timestamp": 1715023200, "event_type": "fall_detected", "confidence": 0.98, "privacy_mask": true}. When the cloud-based big data model receives this de-identified semantic information metadata, it generates suggestive text based on the risk level, such as "A suspected fall has been detected, please contact us immediately...", thus upgrading the monitoring mode from "watching videos" to "reading reports".

[0108] In one embodiment, the collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm priors also includes a hierarchical semantic response and human-machine co-training mechanism.

[0109] The hierarchical semantic response is as follows: the system dynamically adjusts the generation strategy of the large cloud model based on the confidence and risk level of fall detection. Specifically, the strategy is as follows:

[0110] For low-risk, mild circadian rhythm abnormalities (such as frequent nighttime activity), the tone parameters of "gentle, caring, and suggestive" are injected to generate caring health reminders. For example, if frequent nighttime out-of-bed activity is detected, it is recommended to pay attention to sleep quality.

[0111] For high-risk falls, inject tone parameters of "serious, directive, high priority" to generate a brief and objective first aid report, such as "Time: 03:00, Location: Bedroom, Event: Fall, Please respond immediately".

[0112] The human-machine co-training involves constructing a weakly supervised update mechanism based on federated learning, marking specific real samples that are falsely reported or missed as high-value difficult examples, storing them in a dynamic retraining pool, and periodically performing online incremental fine-tuning of the Bayesian gating network.

[0113] The above description describes specific embodiments of the present invention and the technical principles employed. Any changes made in accordance with the concept of the present invention that do not exceed the spirit of the specification and drawings should still fall within the protection scope of the present invention.

Claims

1. A collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior, characterized in that... Includes the following steps: (1) The dual-mode vision acquisition unit acquires the video stream; (2) Construct a multimodal fall detection model, including: (2.1) Extracting video features from video streams using a lightweight temporal-aware network Output the initial behavioral probability distribution And calculate the statistical features characterizing the cognitive uncertainty of the model, including prediction entropy E and confidence boundary M; (2.2) The user's historical behavior sequence is modeled using a neural state-space rhythm model, and the rhythmic surprise of the current behavior relative to the historical pattern is calculated. ; (2.3) Using Bayesian gating networks combined with video features Statistical characteristics and rhythmic surprise Generate route activation probability And based on the route activation probability Dynamically determine whether to intercept the current sample: if it is determined not to intercept, output the classification result of the lightweight temporal-aware network; If an interception is detected, the visual language big model is triggered to perform a review and reasoning on the current sample. (3) Construct a "teacher-student alignment" training mechanism, freeze the parameters of the lightweight temporal perception network and the visual language large model, use the offline dataset to perform binary classification supervision training on the Bayesian gated network, balance positive and negative samples through weighted sampling until the network converges, and obtain the trained Bayesian gated network. (4) Input the real-time video stream into the trained multimodal fall detection model, output the recognition result, and execute the data response strategy of modal isolation based on the recognition result.

2. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1, characterized in that, The dual-mode vision acquisition unit includes a visible light mode and an infrared night vision mode. The visible light mode acquires RGB images, and the infrared night vision mode acquires single-channel grayscale images.

3. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1, characterized in that, Step (2.1) specifically includes the following steps: (2.1.1) Perform sliding window sampling on the video stream to obtain the video sequence. ; (2.1.2) Video sequence The input is a pre-trained lightweight spatiotemporal feature extraction network. This network employs a video encoder based on a weakly supervised vision-language pre-training architecture and undergoes data augmentation training including random grayscale to extract high-dimensional spatiotemporal feature vectors across the illumination domain. Softmax is used to generate a preliminary behavioral probability distribution. ; (2.1.3) Based on the preliminary behavioral probability distribution The statistical characteristics of the uncertainty index predicted by the parallel computing model are characterized, including the prediction entropy. and confidence boundary ,in The first in the preliminary behavioral probability distribution dimensional components, and These represent the values ​​of the maximum probability and the second highest probability in the initial behavior probability distribution, respectively.

4. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1, characterized in that, Step (2.2) specifically includes the following steps: (2.2.1) Treat the user's historical behavior tag stream as a discrete time series Introducing time scale parameters Based on the zero-order hold principle, the parameters of continuous systems are... Convert to discrete parameters : , ， Where I is the identity matrix; (2.2.2) Real-time update of hidden behavioral states using discretized state equations And based on the hidden state Predict the probability distribution of prior actions at the current moment. ; (2.2.3) Rhythmic Surprise Preliminary behavioral probability distribution With prior behavior probability distribution The Kourbak-Leibler divergence between them is calculated using the following formula: .

5. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1, characterized in that, Step (2.3) specifically includes the following steps: (2.3.1) Prediction entropy is obtained through positional encoding Confidence Boundary and rhythmic surprise Mapped to a low-dimensional embedding vector, then combined with video features Concatenation yields the enhanced input vector ,in These are weighting coefficients; (2.3.2) Keep MC Dropout enabled in the last fully connected layer of the lightweight temporal-aware network, perform T random forward propagations on the same input, and obtain the distribution set. Calculate the variance of cognitive uncertainty ,in The L2 norm of a vector This represents the Softmax probability score obtained during the i-th forward propagation. This represents the probability distribution of the mean of T predictions; (2.3.3) Construct a multilayer perceptron containing a normalization layer and an activation function as a Bayesian gated network, and calculate the route activation probability. : , , in, It is a learnable weight matrix of a two-layer MLP. It is Batch Normalization, used for feature standardization; It is the Rectified Linear Unit activation function; Logit is the log-probability of the route that has not passed through the Sigmoid function, output by the MLP. For the Sigmoid function, This is the visual uncertainty compensation coefficient. This is the rhythm surprise compensation coefficient; (2.3.4) Set threshold Based on route activation probability Dynamically determine whether to block the current sample: like If the sample is identified as high-risk, proceed to step (2.3.5). like If a sample is identified as low-risk, the classification result of a lightweight temporal-aware network is output. (2.3.5) Intercept and trigger the visual large language model review, specifically including the following steps: (2.3.5.1) The quantized visual large language model weights are preloaded into the local video memory of the edge device and kept in a hot-resident state to wait for inference instructions; (2.3.5.2) Use image processing algorithms to traverse the video sequence, superimpose a prominent red number at a fixed position in the pixel layer of each frame, and stitch the images after superimposing the numbers into a high-resolution temporal grid diagram in time sequence, and achieve explicit alignment of spatiotemporal information at the input end. (2.3.5.3) Perform cross-modal reasoning locally and construct a general structured thought chain prompt template. The prompt template includes the generation of the final judgment text containing the reasoning process according to the changes in the posture and position of the person in the time-series grid diagram, and reserves an environmental constraint slot in the preceding position of the template. The system detects the domain distribution characteristics of the video stream. If it is determined to be a standard visible light mode, the slot remains empty and the model performs general reasoning. If it is determined to be an infrared night vision mode, environmental correction instructions are dynamically injected into the slot to generate an accurate judgment.

6. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1, characterized in that, Step (3) specifically includes the following steps: (3.1) Constructing an asymmetric distillation inference environment: Define an offline video dataset including daytime, nighttime infrared, and occluded multimodal scenes; define a lightweight temporal perception network as the student model, a large visual language model as the teacher model, and real labels as... Freeze all weight parameters of the student model and the teacher model, and only update the gradient of the Bayesian gated network; use the student model and the teacher model to perform full two-stream inference on the offline video dataset, and obtain the prediction class distribution results and confidence scores of their respective models; (3.2) Define positive sample data for routing: filter out the set of sample data that the student model cannot process but the teacher model can correct. The sample is labeled as 1, indicating that it belongs to the sample data that needs to be intercepted; (3.3) Define invalid or routed negative sample data: filter out the sample data set that does not require the intervention of the teacher model. The sample label is marked as 0, which means that the sample data does not need to be intercepted; (3.4) Class balancing and parameter optimization: Introducing dynamic weights ,use Calculate the loss and backpropagate the gradient, adjusting the focusing parameters. Optimize the in-distribution samples at the decision boundary, where For category weights, To focus parameters, This represents the current number of positive samples. This represents the current number of negative samples. This represents the total number of samples in the current category.

7. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1, characterized in that, The modal isolation data response strategy includes physical blocking and semantic firewall; the physical blocking is to block the external transmission port of the camera's video stream and store the original video locally in a closed loop; the semantic firewall is to only allow the local visual large language model to generate de-identified semantic information metadata and encrypt it for uploading to the cloud.

8. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1, characterized in that, It also includes a hierarchical semantic response and a human-machine co-training mechanism. The hierarchical semantic response is: the system dynamically adjusts the generation strategy of the cloud-based large language model according to the confidence and risk level of fall detection. The human-machine co-training is: a weakly supervised update mechanism based on federated learning is constructed, and specific real samples that are falsely reported or missed are marked as high-value difficult examples and stored in a dynamic retraining pool, and the Bayesian gating network is periodically fine-tuned online.

9. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 5, characterized in that, The value of T is 5.

10. The collaborative fall detection method based on Bayesian cognitive routing and spatiotemporal rhythm prior as described in claim 1 or 5, characterized in that, The classification results of the lightweight temporal sensing network include the probability distribution of 12 types of actions, of which six are falls and six are normal daily actions.

Images