Foggy day self-adaptive pedestrian trajectory prediction method and system for old-age care robot

Abstract

The invention discloses a foggy day self-adaptive pedestrian trajectory prediction method for an old-age care robot. The method comprises the following steps: S1, extracting foggy day physical parameters and fog invariant features; s2, based on a multi-scale Mama space-time sequence module, extracting fusion features in the foggy weather; s3, constructing a fog sensing dynamic heterogeneous network; and S4, trajectory modeling is carried out by using the fusion features. The invention also discloses a foggy day self-adaptive pedestrian trajectory prediction system for the old-age care robot, which is applied to the foggy day self-adaptive pedestrian trajectory prediction method for the old-age care robot. Comprising a physical prior fusion fog sign decoupling unit used for estimating foggy day physical parameters and extracting fog invariant features, and a multi-scale Mamba space-time sequence module used for extracting space-time fusion features in a foggy day environment. The fog perception dynamic heterogeneous network is used for modeling the social relation of pedestrians in foggy days and fusing the spatio-temporal features; and the trajectory generation module is used for generating a prediction trajectory based on the fused features.

Description

[Technical Field]

[0001] This invention belongs to the field of navigation and safety monitoring technology for intelligent elderly care robots, and relates to an adaptive pedestrian trajectory prediction method for elderly care robots in foggy weather. [Background Technology]

[0002] As society ages, the importance of elderly care service robots in assisting seniors with daily travel, caregiving, and intelligent health management is becoming increasingly prominent. In low-visibility environments such as smog and fog, these robots need stable environmental perception and safe navigation capabilities to ensure the safety of seniors. For example, when guiding seniors on foot or performing emergency assistance tasks in foggy weather, accurate pedestrian trajectory prediction is crucial to avoiding collisions.

[0003] Currently, existing pedestrian trajectory prediction technologies face the following main problems when used in foggy environments to assist elderly care robots and other intelligent systems:

[0004] (1) Visual feature degradation: Optical scattering in foggy environments leads to a significant decrease in the quality of features extracted by the visual perception system. Traditional image dehazing methods such as DCP and AOD-Net have problems such as inaccurate parameter estimation or neglect of physical consistency of features.

[0005] (2) Changes in behavior patterns: Foggy weather can alter pedestrian behavior patterns, such as reduced speed, tighter group structure, and narrower social interaction range. Existing models lack adaptability to these changes.

[0006] (3) Low efficiency of time series modeling: The time series modeling method based on Transformer has problems such as high computational complexity and memory bottleneck when processing long sequences, which makes it difficult to meet the real-time requirements of elderly care robots.

[0007] (4) Insufficient social relationship modeling: Existing social relationship modeling methods lack environmental adaptability, cannot accurately capture the unique pedestrian interaction patterns in foggy weather, and are not targeted enough for the application scenarios of elderly care robots.

[0008] Therefore, the present invention was developed based on the above-mentioned shortcomings. [Summary of the Invention]

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide an adaptive pedestrian trajectory prediction method and system for elderly care robots in foggy weather. It integrates atmospheric scattering physics priors and the Mamba architecture to solve the problems of physical degradation and interaction modeling in pedestrian trajectory prediction in foggy weather scenarios, and fully considers the travel safety requirements of elderly care robots for pedestrians and the elderly in low visibility environments.

[0010] This invention is achieved through the following technical solution:

[0011] A method for predicting pedestrian trajectories in foggy weather using an elderly care robot, characterized by the following steps:

[0012] S1. Extract the physical parameters of fog and fog-invariant features. Simultaneously estimate the atmospheric scattering coefficient, atmospheric illumination value and scene depth map of the image through the physical parameter estimation network. Construct the physical prior feature path and the fog adaptive feature path, and fuse them through the adaptive attention mechanism to obtain the fog-invariant features.

[0013] S2. Based on the multi-scale Mamba spatiotemporal sequence module, extract the fusion features under foggy conditions. Utilize the multi-layer Mamba module, cross-layer attention connection and feature pyramid mechanism to model the multi-scale spatiotemporal dependency of pedestrian trajectories and obtain temporal features.

[0014] S3. Construct a fog perception dynamic heterogeneous network, dynamically adjust the attention weights through the fog perception modulation function, distinguish three relationship types: individual-individual, individual-group, and group-group, perform group recognition and spatiotemporal feature fusion, and obtain fused features.

[0015] S4. Use the fused features to model the trajectory, optimize it through a multi-objective loss function, and generate future trajectory predictions by combining a variational inference framework.

[0016] The above-described method for predicting pedestrian trajectories in foggy weather using an elderly care robot is characterized in that: the physical parameter estimation network in step S1 includes a depth estimation branch, an atmospheric illumination estimation branch, and a scattering coefficient estimation branch. In the depth estimation branch, a pixel-level depth map d(x) is output based on a U-Net network. In the atmospheric illumination estimation branch, the atmospheric illumination value A is predicted by combining global feature pooling and a multilayer perceptron with a prior-guided attention mechanism. In the scattering coefficient estimation branch, the scattering coefficient β of the current scene is predicted by combining global scene features and local pedestrian region features.

[0017] The fog-adaptive pedestrian trajectory prediction method for elderly care robots described above is characterized in that: in step S1, the physical prior feature path recovers the fog-free image I0 through a scattering model, and then extracts features F from the recovered image through a feature extraction network. phys The fog adaptation feature path extracts feature F from the original foggy image I. adapt And the feature F′ is adjusted through a learnable scattering modulation layer. adapt The physical prior feature path and the fog adaptive feature path are fused through an adaptive attention mechanism to obtain a comprehensive fog-invariant feature representation:

[0018]

[0019] Where α is the fusion weight generated by the dynamic attention network, and f α It is a multilayer perceptron consisting of three fully connected networks, with the fusion weight α ranging from [0,1].

[0020] The above-described method for predicting pedestrian trajectories in foggy weather using an elderly care robot is characterized by the following: the processing steps of the multi-scale Mamba spatiotemporal sequence module in step S2 include:

[0021] S21. Map the input pedestrian historical trajectory sequence X to a high-dimensional feature space:

[0022]

[0023] in It is a projection matrix. This is the bias term, and D is the model's hidden dimension, D = 256;

[0024] S22. Processing the state-space model through the Mamba block, including generating the state transition matrix A(x), input projection matrix B(x), output projection matrix C(x), and time step parameter Δ(x), and obtaining the output feature Y through a scanning operation, the process is as follows:

[0025]

[0026] in It is a gated weight matrix;

[0027] S23. Apply residual connections and layer normalization to stabilize the training process;

[0028] S24. Achieving feature interaction across different time scales through cross-layer attention connections:

[0029] F l+1 =Mamba*l(F l +α l Attention(F) l ,F*l-1))

[0030] Where Mamba*l represents the complete spatiotemporal Mamba block processing of the l-th layer, α l It is the balance coefficient;

[0031] S25. Obtain multi-scale features F by using attention-based multi-scale feature pyramid fusion. multi .

[0032] The fog-adaptive pedestrian trajectory prediction method for elderly care robots described above is characterized in that: the fog perception modulation function in step S3 is:

[0033]

[0034] Where β ranges from [0,2], and a larger value indicates denser fog; d ij This is the actual distance between pedestrian i and pedestrian j;

[0035] Expression for dynamically adjusting attention weights:

[0036]

[0037] in This represents the original attention weights.

[0038] The above-described method for predicting pedestrian trajectories in foggy weather using elderly care robots is characterized in that the group identification in step S3 includes:

[0039] S31. Calculate the pedestrian similarity matrix using the following formula:

[0040]

[0041] Where sim(T) i ,T j ) represents trajectory similarity, w ij It represents the weights of the edges in the social graph, and α is the dynamic balance factor.

[0042] S32. Based on the density clustering algorithm, dynamically adjust the clustering parameters according to the fog intensity.

[0043] S33. Construct virtual nodes for the group and extract internal structural features through a graph convolutional network.

[0044] The above-described method for predicting pedestrian trajectories in foggy weather using an elderly care robot is characterized in that step S4 includes:

[0045] S41. Construct a multi-objective loss function system;

[0046] S42. Transform the spatiotemporal feature map into a pedestrian-level representation vector through adaptive pooling operations.

[0047] Then, conditional latent variable mapping is constructed using multi-level nonlinear transformations;

[0048] S43. Based on the variational inference framework, a conditional variational autoencoder is used to parametrically model the trajectory distribution and generate future trajectory predictions.

[0049] The above-described method for predicting pedestrian trajectories in foggy weather using elderly care robots is characterized in that: the multi-objective loss function in step S41 includes:

[0050] Depth loss is calculated using the following formula:

[0051] Ldepth =λ1L1(d,d gt )+λ2L edge (d,d gt )+λ3L grad (d,d gt )

[0052] Where L1 represents the depth map prediction value d and the standard depth d. gt The mean absolute error between them, L edge For the edge-preserving loss term, L grad This is the gradient consistency loss. Weight coefficients λ1, λ2, and λ3 are used to balance the contributions of each loss component;

[0053] The physical reconstruction loss is calculated using the following formula:

[0054] L recon =||I-(I0·e -βd(x) +A·(1-e -βd(x) ))||1

[0055] The trajectory prediction loss is calculated using the following formula:

[0056]

[0057] Where T obs Indicates the observation time step. These are the corresponding actual trajectory coordinates;

[0058] The social interaction consistency loss is calculated using the following formula:

[0059]

[0060] The overall loss function is calculated using the following formula:

[0061] L total =α1L depth +α2L recon +α3L traj +α4L social

[0062] Among them, α1, α2, α3 and α4 are loss weights.

[0063] A fog-adaptive pedestrian trajectory prediction system for elderly care robots, applied to the aforementioned fog-adaptive pedestrian trajectory prediction method for elderly care robots, is characterized by comprising: a physical prior fusion fog feature decoupling unit for estimating fog physical parameters and extracting fog-invariant features; a multi-scale Mamba spatiotemporal sequence module for extracting spatiotemporal fusion features in foggy environments; a fog-aware dynamic heterogeneous network for modeling social relationships among pedestrians in foggy conditions and fusing spatiotemporal features; and a trajectory generation module based on fusion features for generating predicted trajectories.

[0064] Compared with the prior art, the present invention has the following advantages:

[0065] 1. This invention constructs a differentiable atmospheric scattering model and decouples fog concentration and light degradation through a physical parameter estimation network, achieving end-to-end training from physical perception to behavior prediction. This effectively solves the problem of visual feature degradation in foggy environments and provides high reliability for pedestrian trajectory prediction under foggy conditions. For elderly care robots, this fusion can significantly improve their accuracy in perceiving the surrounding environment in foggy weather, ensuring the safety of guidance and assistance work.

[0066] 2. This invention innovatively introduces the Mamba state-space model for efficient temporal modeling, and uses its linear complexity characteristics to construct the spatiotemporal sequence modeling backbone, which solves the computational complexity bottleneck of traditional attention mechanisms. While maintaining the ability to model long-range dependencies, it significantly improves the inference speed, enabling the elderly care robot to observe the trajectories of multiple pedestrians in multi-pedestrian scenarios and provide stable and efficient prediction capabilities, thus meeting the needs of the elderly care robot for large-scale, multi-target collaborative prediction.

[0067] 3. The present invention further designs a fog-sensing dynamic heterogeneous network, which accurately models the complex interactions between pedestrians through a dynamic adjacency matrix and captures the collaborative evolution law of motion trends by a spatiotemporal fusion module. This can better capture interactive behaviors in different scenarios. For applications such as elderly care robots that need to interact closely with humans, it can significantly improve interaction prediction and safety planning under foggy conditions.

[0068] 4. This invention lays a solid foundation for the perception and interaction of elderly care robots and other intelligent systems in foggy environments. It can not only improve the robustness and accuracy of pedestrian trajectory prediction in foggy environments in intelligent transportation scenarios, but also be used in elderly care robots and other intelligent platforms that require safety assistance functions. It provides advanced solutions for pedestrian behavior recognition and trajectory prediction under low visibility conditions, such as path planning and safety assistance for elderly care robots in foggy environments, to ensure the travel safety of the elderly. [Attached Image Description]

[0069] Figure 1 This is a schematic diagram of the physical parameter estimation network of the present invention;

[0070] Figure 2 This is a schematic diagram of the multi-scale Mamba spatiotemporal sequence module of the present invention;

[0071] Figure 3 This is a schematic diagram of the fog-sensing dynamic heterogeneous network of the present invention;

[0072] Figure 4 This is a system block diagram of the present invention.

Detailed Implementation Methods

[0073] The present invention will now be further described with reference to the accompanying drawings:

[0074] like Figures 1 to 4 As shown, an adaptive pedestrian trajectory prediction method for elderly care robots in foggy weather includes the following steps:

[0075] S1. Extract the physical parameters of fog and fog-invariant features. Simultaneously estimate the atmospheric scattering coefficient, atmospheric illumination value and scene depth map of the image through the physical parameter estimation network. Construct the physical prior feature path and the fog adaptive feature path, and fuse them through the adaptive attention mechanism to obtain the fog-invariant features.

[0076] S2. Based on the multi-scale Mamba spatiotemporal sequence module, extract the fusion features under foggy conditions. Utilize the multi-layer Mamba module, cross-layer attention connection and feature pyramid mechanism to model the multi-scale spatiotemporal dependency of pedestrian trajectories and obtain temporal features.

[0077] S3. Construct a fog perception dynamic heterogeneous network, dynamically adjust the attention weights through the fog perception modulation function, distinguish three relationship types: individual-individual, individual-group, and group-group, perform group recognition and spatiotemporal feature fusion, and obtain fused features.

[0078] S4. Use the fused features to model the trajectory, optimize it through a multi-objective loss function, and generate future trajectory predictions by combining a variational inference framework.

[0079] After inputting the foggy video sequence and extracted pedestrian historical trajectory data, step S1 proceeds with the operation. To effectively estimate the atmospheric scattering physical parameters of an image, this invention designs an end-to-end Physical Parameter Estimation Network (PPEN) with three branches: a depth estimation branch, an atmospheric illumination estimation branch, and a scattering coefficient estimation branch. This network can simultaneously estimate the atmospheric scattering coefficient β, atmospheric illumination value A, and scene depth map d(x) of the image. The specific branches are as follows:

[0080] Depth estimation branch: A pure U-Net network is used to predict pixel-level depth maps d(x) from the input image. The network first downsamples the input image through a five-layer encoder to capture multi-scale features, and then recovers the pixel-level depth map through a five-layer decoder.

[0081] Atmospheric illumination estimation branch: Predicts the global atmospheric illumination value A using global feature pooling and a multilayer perceptron. Atmospheric illumination typically corresponds to the pixel values ​​of the brightest region in an image, but simply selecting the brightest pixel is susceptible to noise and highly reflective objects. To avoid this problem, a prior-guided attention mechanism is employed.

[0082]

[0083] Where f is the scoring function, v i It is the image pixel value, p i It is the normalized pixel spatial position coordinate, d i This is the estimated depth value. The function contains four convolutional layers, followed by global average pooling and three fully connected layers, outputting the atmospheric illumination values ​​A∈[0,1] in RGB three-channel format. 3 .

[0084] Scattering coefficient estimation branch: Combining global scene features and local pedestrian region features, predict the scattering coefficient β of the current scene, β = g(F global ,F pedestrian ), where F global These are global scene features, extracted using a ResNet-50 network; F pedestrian is the pedestrian region feature, extracted from the pedestrian bounding box through ROI pooling; g is the prediction function, implemented as a three-layer fully connected network, outputting a single scalar scattering coefficient β∈[0,3].

[0085] In PPEN, the observed foggy image is defined as I(x), and the true reflectance of the scene (fog-free image) is defined as I0(x). The goal is to accurately estimate key physical parameters in foggy environments through dual guidance from supervised signals and physical reconstruction constraints.

[0086]

[0087] In step S1, a clear image is reconstructed using the estimated physical parameters, and two complementary feature extraction paths are constructed to efficiently provide high-quality features to the downstream feature extractor. These two paths are the physical prior feature path and the fog adaptation feature path. The physical prior feature path recovers the fog-free image I0 using a scattering model, and then extracts features F from the recovered image through the feature extraction network. phys :

[0088] F phys =CNN*θ*phys(I0)

[0089] Where CNN*θ*phys is the feature extraction network. The fog-adaptive feature path directly extracts features from the original foggy image I, and adjusts the features through a learnable scattering modulation layer:

[0090] F adapt =CNN*θ*adapt(I)

[0091]

[0092] Where CNN*θ*adapt is a feature extraction network that shares weights with the physical path, ⊙ represents element-wise multiplication, and d i γ is the average depth, representing the depth information of the scene. γ and δ are learnable parameters that adjust for the scattering coefficient and the influence of atmospheric illumination. The features extracted from the two paths are fused through an adaptive attention mechanism to obtain a comprehensive fog-invariant feature representation.

[0093]

[0094] Where α is the fusion weight generated by the dynamic attention network, and f α It is a multilayer perceptron consisting of three fully connected layers, with the fusion weight α ranging from [0,1], controlling the relative importance of the contributions of the two paths. This allows the system to employ different fusion strategies in different regions of the same image.

[0095] Given a sequence of pedestrian historical trajectories X = x1, x2, ..., x T ,in Including two-dimensional coordinates and F-dimensional additional features (velocity, acceleration, direction, etc.), we designed a multi-scale spatiotemporal feature extraction framework based on Mamba, such as... Figure 2 As shown, through the combined action of multi-layer Mamba blocks, cross-layer attention connections, and feature pyramid mechanisms, the multi-scale spatiotemporal dependencies of pedestrian trajectories are efficiently modeled. The processing steps of the multi-scale Mamba spatiotemporal sequence module in step S2 include:

[0096] S21. Map the input pedestrian historical trajectory sequence X to a high-dimensional feature space:

[0097]

[0098] in It is a projection matrix. This is the bias term, and D is the model's hidden dimension, which is set to 256 here.

[0099] S22. Processing the state-space model through the Mamba block, including generating the state transition matrix A(x), input projection matrix B(x), output projection matrix C(x), and time step parameter Δ(x), and obtaining the output feature Y through a scanning operation, the process is as follows:

[0100]

[0101] Where A(x), B(x), C(x), and Δ(x) represent the state transition matrix, input projection matrix, output projection matrix, and time step parameter of the continuous-time system, respectively. It is the gated weight matrix. By mapping the input features to the SSM parameter space, and then efficiently updating the state through the Scan operation, the output projection C(x) finally maps the state to the output space Y.

[0102] S23. Apply residual connections and layer normalization to stabilize the training process. This achieves the effect of stabilizing the training process.

[0103] Output=LayerNorm(X+Dropout(YW out +b out ))

[0104] in It is the output projection matrix. This is the output bias term. Setting Dropout to 0.1 provides a regularization effect.

[0105] S24. Achieving feature interaction across different time scales through cross-layer attention connections: F l+1 =Mamba*l(F l +α l Attention(F) l ,F*l-1)), where Mamba*l represents the complete spatiotemporal Mamba block processing of the l-th layer, α l It is the balance coefficient.

[0106] In this way, the Mamba block at layer l+1 can utilize relevant spatiotemporal information from layer l-1 during processing, thereby establishing a more direct connection between different time scales and strengthening information exchange between different time scales.

[0107] S25. Obtain multi-scale features F by using attention-based multi-scale feature pyramid fusion. multi .

[0108] After processing all network layers, we need to fuse the captured multi-scale features into the final representation. Therefore, we introduce an attention-based multi-scale feature pyramid, and obtain the feature F through the feature pyramid mechanism. multi .

[0109] In step S3, the fog-sensing dynamic heterogeneous network receives temporal features as input and adaptively adjusts the social relationship modeling according to the fog intensity. Based on this, as... Figure 3 As shown, this invention designs a heterogeneous graph attention layer to handle different types of social relationships. It distinguishes three main relationship types: individual-to-individual (P2P) interactions between independent pedestrians, individual-to-group (P2G) interactions between independent pedestrians and groups, and group-to-group (G2G) interactions between different groups. For pedestrian node i, its feature update is achieved through a heterogeneous graph attention mechanism.

[0110] To adapt to foggy conditions, a fog-sensing modulation function g was designed in step S3. r Weights used to dynamically adjust attention in the network:

[0111]

[0112] Where β ranges from [0,2], and a larger value indicates denser fog; d ij This represents the actual distance between pedestrians i and j. As the fog intensity increases or the pedestrian distance increases, the function g... r It will output a smaller value, thus reducing the original attention weights. This perfectly simulates the phenomenon of limited visibility and reduced social interaction in real-world foggy conditions.

[0113] Node feature update mechanism of graph neural network in pedestrian trajectory prediction system in foggy weather

[0114]

[0115] Where R represents the relation type set P2P, P2G, G2G. It is the set of neighboring nodes that have a relationship of type r with node i. β is the weight of the m-th attention head under foggy conditions. r These are the skip connection weights of relation r, controlling the proportion of original features retained. It is a linear transformation matrix of the jump connection.

[0116] Next, group identification was performed, the core of which was quantifying the similarity between pedestrians. Group identification included:

[0117] S31. Calculate the pedestrian similarity matrix using the following formula:

[0118]

[0119] Where sim(T) i ,T j The similarity score is calculated as a weighted combination of direction, velocity, acceleration, and distance similarity. ij The weights are the edges in the social graph, and α is a dynamic balancing factor that adaptively adjusts under foggy conditions. This pedestrian similarity matrix combines trajectory features and social relationships.

[0120] S32. Based on the density clustering algorithm, the clustering parameters are dynamically adjusted according to the fog intensity.

[0121] S33. Construct virtual nodes for the group and extract internal structural features through a graph convolutional network.

[0122] Step S32 uses the density-based clustering algorithm (DBSCAN) for group identification, dynamically adjusting the clustering parameter β based on the fog intensity. After group identification, virtual nodes are created for each group, and weighted pooling is implemented using an attention mechanism. To capture the internal structure of the groups, subgraph representations of the groups are implemented. For group G, the similarity matrix S is used... ij The weights of the edges are used to construct the internal subgraph G. sub Furthermore, structural information is extracted through a graph convolutional network (GCN), and by combining pooling and structural representation, end-to-end training of the balance coefficient γ is achieved.

[0123]

[0124] By integrating Mamba time series features Graph attention space features Spatiotemporal feature fusion is performed, and a gating fusion formula is designed to achieve adaptive feature integration:

[0125] F fused =FC([[F Mamba ,F Graph ]])+γ·F Mamba ⊙F Graph

[0126] Furthermore, a cross-modal attention mechanism is introduced to establish a fine-grained correspondence between temporal and spatial features:

[0127]

[0128] Q Mamba =W q F Mamba K Graph =W k F Graph

[0129] V Graph =W v F Graph V Mamba =W v′ F Mamba

[0130] Among them W q W k W v W v′ It is a linear projection matrix, Q Mamba K Graph V Graph and V Mamba It is a query, key, and value matrix obtained by linear projection of the corresponding features. Finally, the features are fused through weighted combination and fog-adaptive enhancement is introduced:

[0131] F final =(λF fused +(1-λ)F cross )·(1+δ·tanh(W β β))

[0132] Where λ is the equilibrium parameter, initially set to 0.5, and the fog enhancement layer is constructed using the learnable parameter δ and the projection matrix W. β Adaptive feature adjustment compensates for information degradation in foggy conditions. Especially under heavy fog conditions, the ability to predict foggy trajectories is significantly improved by effectively integrating spatiotemporal information.

[0133] Pedestrian trajectory prediction is essentially a multimodal problem—given the same historical trajectory, pedestrians may take multiple reasonable but different future paths. This multimodal characteristic is more pronounced in foggy environments because reduced visibility increases the uncertainty of behavior. To address this, we use the acquired feature map F... final The feature post-processing network is used to model the trajectory and generate diverse future trajectories. Step S4 includes:

[0134] S41. Construct a multi-objective loss function system;

[0135] To improve the accuracy of depth estimation, multi-scale supervision and edge-aware loss are used to constrain the optimization process:

[0136] L depth =λ1L1(d,d gt )+λ2L edge (d,d gt )+λ3L grad (d,d gt )

[0137] Where L1 represents the depth map prediction value d and the standard depth d. gt Mean absolute error (MAE) between L edge For the edge-preserving loss term, L grad This is the gradient consistency loss. The weight coefficients λ1, λ2, and λ3 are used to balance the contributions of each loss component.

[0138] Introducing a reconstruction loss based on a physics model ensures the accuracy and consistency of physical parameter estimation:

[0139] L recon =||I-(I0·e -βd(x) +A·(1-e -βd(x) ))||1

[0140] Basic trajectory prediction loss:

[0141]

[0142] Where T obs Indicates the observation time step. These are the corresponding actual trajectory coordinates. The first term is the Euclidean distance loss, which quantifies the predicted trajectory. With the actual trajectory The deviation between them, the second term is the KL divergence regularization term, to ensure the posterior distribution q φ (z|X,Y gt ) and prior distribution p θ The proximity between (z|X) prevents overfitting and enhances prediction diversity.

[0143] Simultaneously, a social interaction consistency loss was introduced to accurately model the phenomenon of social interaction range contraction in foggy conditions:

[0144]

[0145] Finally, all the above constraints are integrated to construct the overall loss function:

[0146] L total =α1L depth +α2L recon +α3L traj +α4L social

[0147] Among them, α1, α2, α3 and α4 are loss weights. Extensive experiments have shown that the proposed model achieves the best performance when the parameters are set to 0.2, 0.3, 0.5 and 0.4, respectively.

[0148] S42. The spatiotemporal feature map is converted into a pedestrian-level representation vector through adaptive pooling, and then a conditional latent variable mapping is constructed using multi-layer nonlinear transformation:

[0149]

[0150] Where Ψ is the feature projection function, which maps the feature map to the D-dimensional pedestrian representation space.

[0151] S43. Based on the variational inference framework, a conditional variational autoencoder is used to parametrically model the trajectory distribution and generate future trajectory predictions. During the training phase, the encoder learns the posterior distribution from historical observations and feature mappings to the latent space; during the inference phase, the decoder network generates future trajectory predictions by sampling from the prior distribution and combining it with conditional features.

[0152]

[0153] The final decoder outputs the predicted trajectory of the pedestrian path. Where N represents the number of pedestrians in the scene, T predTo predict the time step, i.e. the pedestrian's two-dimensional spatial coordinates contained in each time step.

[0154] like Figure 2 As shown, the present invention also discloses a fog-adaptive pedestrian trajectory prediction system for elderly care robots, which is applied to the above-mentioned fog-adaptive pedestrian trajectory prediction method for elderly care robots. The system includes: a physical prior fusion fog feature decoupling unit for estimating fog physical parameters and extracting fog-invariant features; a multi-scale Mamba spatiotemporal sequence module for extracting spatiotemporal fusion features in foggy environments; a fog-aware dynamic heterogeneous network for modeling social relationships of pedestrians in foggy weather and fusing spatiotemporal features; and a trajectory generation module for generating predicted trajectories based on fusion features.

Claims

1. A method for predicting pedestrian trajectories in foggy weather using an elderly care robot, characterized in that, Includes the following steps: S1. Extract the physical parameters of fog and fog-invariant features. Simultaneously estimate the atmospheric scattering coefficient, atmospheric illumination value and scene depth map of the image through the physical parameter estimation network. Construct the physical prior feature path and the fog adaptive feature path, and fuse them through the adaptive attention mechanism to obtain the fog-invariant features. S2. Based on the multi-scale Mamba spatiotemporal sequence module, extract the fusion features under foggy conditions. Utilize the multi-layer Mamba module, cross-layer attention connection and feature pyramid mechanism to model the multi-scale spatiotemporal dependency of pedestrian trajectories and obtain temporal features. S3. Construct a fog perception dynamic heterogeneous network, dynamically adjust the attention weights through the fog perception modulation function, distinguish three relationship types: individual-individual, individual-group, and group-group, perform group recognition and spatiotemporal feature fusion, and obtain fused features. S4. Use the fused features to model the trajectory, optimize it through a multi-objective loss function, and generate future trajectory predictions by combining a variational inference framework.

2. The method for predicting pedestrian trajectories in foggy weather for elderly care robots according to claim 1, characterized in that: The physical parameter estimation network in step S1 includes a depth estimation branch, an atmospheric illumination estimation branch, and a scattering coefficient estimation branch. The depth estimation branch outputs a pixel-level depth map d(x) based on the U-Net network. The atmospheric illumination estimation branch predicts the atmospheric illumination value A through global feature pooling and a multilayer perceptron, combined with a prior-guided attention mechanism. The scattering coefficient estimation branch predicts the scattering coefficient β of the current scene by combining global scene features and local pedestrian region features.

3. The method for predicting pedestrian trajectories in foggy weather for elderly care robots according to claim 1, characterized in that: In step S1, the physical prior feature path recovers the haze-free image I0 through a scattering model, and then extracts features F from the recovered image through a feature extraction network. phys The fog adaptation feature path extracts feature F from the original foggy image I. adapt And the feature F′ is adjusted through a learnable scattering modulation layer. adapt The physical prior feature path and the fog adaptive feature path are fused through an adaptive attention mechanism to obtain a comprehensive fog-invariant feature representation: Where α is the fusion weight generated by the dynamic attention network, and f α It is a multilayer perceptron consisting of three fully connected networks, with the fusion weight α ranging from [0,1].

4. The method for predicting pedestrian trajectories in foggy weather for elderly care robots according to claim 1, characterized in that: The processing procedure of the multi-scale Mamba spatiotemporal sequence module in step S2 includes: S21. Map the input pedestrian historical trajectory sequence X to a high-dimensional feature space: in It is a projection matrix. This is the bias term, and D is the model's hidden dimension, D = 256; S22. Processing the state-space model through the Mamba block, including generating the state transition matrix A(x), input projection matrix B(x), output projection matrix C(x), and time step parameter Δ(x), and obtaining the output feature Y through a scanning operation, the process is as follows: in It is a gated weight matrix; S23. Apply residual connections and layer normalization to stabilize the training process; S24. Achieving feature interaction across different time scales through cross-layer attention connections: F l+1 Mamba*l(F l +α l ·Attention(F l ,F*l-1)) Where Mamba*l represents the complete spatiotemporal Mamba block processing of the l-th layer, α l It is the balance coefficient; S25. Obtain multi-scale features F by using attention-based multi-scale feature pyramid fusion. multi .

5. The method for predicting pedestrian trajectories in foggy weather for elderly care robots according to claim 1, characterized in that: The fog sensing modulation function in step S3 is: Where β ranges from [0,2], and a larger value indicates denser fog; d ij This is the actual distance between pedestrian i and pedestrian j; Expression for dynamically adjusting attention weights: in This represents the original attention weights.

6. The method for predicting pedestrian trajectories in foggy weather for elderly care robots according to claim 1, characterized in that, The group identification in step S3 includes: S31. Calculate the pedestrian similarity matrix using the following formula: Where sim(T) i ,T j ) represents trajectory similarity, w ij It represents the weights of the edges in the social graph, and α is the dynamic balance factor. S32. Based on the density clustering algorithm, dynamically adjust the clustering parameters according to the fog intensity. S33. Construct virtual nodes for the group and extract internal structural features through a graph convolutional network.

7. The method for predicting pedestrian trajectories in foggy weather for elderly care robots according to claim 1, characterized in that: Step S4 includes: S41. Construct a multi-objective loss function system; S42. Transform the spatiotemporal feature map into a pedestrian-level representation vector through adaptive pooling operations. Then, conditional latent variable mapping is constructed using multi-level nonlinear transformations; S43. Based on the variational inference framework, a conditional variational autoencoder is used to parametrically model the trajectory distribution and generate future trajectory predictions.

8. The method for predicting pedestrian trajectories in foggy weather for elderly care robots according to claim 7, characterized in that: The multi-objective loss function in step S41 includes: Depth loss is calculated using the following formula: L depth =λ1L1(d,d gt )+λ2L edge (d,d gt )+λ3L grad (d,d gt ) Where L1 represents the depth map prediction value d and the standard depth d. gt The mean absolute error between them, L edge For the edge-preserving loss term, L grad This is the gradient consistency loss. Weight coefficients λ1, λ2, and λ3 are used to balance the contributions of each loss component; The physical reconstruction loss is calculated using the following formula: L recon =||I-(I0·e -βd(x) +A·(1-e -βd(x) ))||1 The trajectory prediction loss is calculated using the following formula: Where T obs Indicates the observation time step. These are the corresponding actual trajectory coordinates; The social interaction consistency loss is calculated using the following formula: The overall loss function is calculated using the following formula: L total =α1L depth +α2L recon +α3L traj +α4L social Among them, α1, α2, α3 and α4 are loss weights.

9. A fog-adaptive pedestrian trajectory prediction system for an elderly care robot, applied to the fog-adaptive pedestrian trajectory prediction method for an elderly care robot as described in any one of claims 1 to 8, characterized in that, include: A physical prior fusion fog feature decoupling unit for estimating fog physical parameters and extracting fog-invariant features; a multi-scale Mamba spatiotemporal sequence module for extracting spatiotemporal fusion features under fog conditions; a fog-aware dynamic heterogeneous network for modeling pedestrian social relationships in fog and fusing spatiotemporal features; and a trajectory generation module based on fusion features for generating predicted trajectories.

Images