Intelligent parking space management system based on multi-source data fusion

Abstract

The invention discloses an intelligent parking space management system based on multi-source data fusion. The intelligent parking space management system comprises a data acquisition and processing module used for acquiring and preprocessing multi-source original data; the multi-modal fusion module is used for carrying out fusion modeling through an improved DynMM structure; the initial state recognition module is used for generating an initial occupancy state label and an initial occupancy confidence score; the space consistency modeling module is used for generating a space consistency state label and a space consistency confidence score through an improved GraphSAGE model; the comparison training module is used for comparing and updating the improved GraphSAGE model; the visualization module is used for generating a state distribution diagram; and the feedback updating module is used for collecting interaction feedback data and incrementally updating the improved DynMM structure and the improved GraphSAGE model. According to the method, the state identification precision, the space consistency and the dynamic adaptive capacity of parking space management under multi-source data fusion are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and urban parking management technology, and in particular to an intelligent parking space management system based on multi-source data fusion. Background Technology

[0002] With the continuous advancement of smart city construction and refined urban traffic management, the problem of parking resource shortages and low management efficiency in core urban areas has received widespread attention. Existing smart parking systems typically rely on single-modal sensor data for parking space status identification, but in actual deployment, they generally suffer from the following problems:

[0003] Single-modal sensors exhibit poor robustness in specific scenarios, such as interference from steel bars in geomagnetic sensors, changes in lighting conditions in video, and weather interference in infrared sensors, leading to decreased accuracy in parking space recognition. Furthermore, data from multiple sensors lacks a synchronization mechanism in the temporal dimension, resulting in inconsistent sampling frequencies and uncontrollable signal time differences. Existing fusion methods often employ static weighting or simple splicing, making it difficult to adaptively adjust modal contributions based on scene changes. Traditional fusion models lack characterization of parking space topology, failing to leverage the collaborative states of adjacent parking spaces to enhance prediction accuracy. In situations where edge computing resources are limited, deploying complex deep models is difficult, and real-time inference capabilities are insufficient, impacting overall system response efficiency and practicality.

[0004] Therefore, how to provide a smart parking management system based on multi-source data fusion is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a smart parking management system based on multi-source data fusion. This invention fully utilizes an improved DynMM structure and an improved GraphSAGE model, detailing multi-source sensing data acquisition, cross-modal feature fusion, spatial consistency inference, and model comparison and incremental updates. The improved DynMM structure achieves modal path memory, structural prior constraints, asynchronous temporal modeling, and causal gating fusion, significantly improving the adaptability and stability of multi-source heterogeneous data fusion. The improved GraphSAGE model introduces structural heterogeneity sampling and contrastive learning mechanisms, enabling spatial consistency inference and high-confidence recognition of parking space states. This invention possesses advantages such as high fusion accuracy, strong robustness, good interpretability, and edge deployment friendliness.

[0006] A smart parking space management system based on multi-source data fusion according to an embodiment of the present invention includes the following steps: Data acquisition and processing module: used to collect multi-source raw data of the target parking area and preprocess it to generate a standardized multimodal sample set; Multimodal fusion module: Used to perform fusion modeling on standardized multimodal sample sets through an improved DynMM structure to generate cross-modal fusion feature vectors; Initial state identification module: used to generate initial occupancy state labels and initial occupancy confidence scores; Spatial Consistency Modeling Module: Used to generate spatial consistency state labels and spatial consistency confidence scores using the improved GraphSAGE model; The comparative training module is used to perform comparative training and updates on the improved GraphSAGE model. Visualization module: Used to generate a status distribution map of the target parking area; Feedback Update Module: Used to collect user interaction feedback data and perform incremental updates on the improved DynMM structure and the improved GraphSAGE model.

[0007] Optionally, modules can be integrated using the following methods: Step 1: Collect multi-source raw data of the target parking area and preprocess the multi-source raw data to generate a standardized multimodal sample set; Step 2: Input the standardized multimodal sample set into the improved DynMM structure, update the modal path memory, construct the modal structure prior graph, perform asynchronous modeling and causal gating adjustment, and generate cross-modal fusion feature vectors through cross-modal attention; Step 3: Based on the cross-modal fusion feature vector, perform parking space status recognition operation to generate the initial occupancy status label and initial occupancy confidence score of the target parking space; Step 4: Input the cross-modal fusion feature vectors within the target parking area into the improved GraphSAGE model to perform spatial consistency modeling, and generate spatial consistency state labels and spatial consistency confidence scores; Step 5: Based on the spatially consistent state labels, construct positive sample pairs and negative sample pairs, and perform comparative training and updates on the improved GraphSAGE model; Step 6: Based on the spatial consistency status label and spatial consistency confidence score, generate a visual status layer of the target parking space and a status distribution map of the target parking area; Step 7: Collect user interaction feedback data and incrementally update the improved DynMM structure and the improved GraphSAGE model.

[0008] Optionally, step one specifically includes: Image acquisition equipment, geomagnetic sensors, infrared sensors, millimeter-wave radar, brightness sensors, temperature and humidity sensors, and rain and snow sensing sensors are deployed in the target parking area to simultaneously collect raw data from multiple sources. The multi-source raw data includes image frame sequences, magnetic field strength data, infrared reflection signals, distance echo data, light intensity data, temperature data, relative humidity data, and precipitation intensity data; The multi-source raw data undergoes preprocessing operations, including time alignment, data normalization, outlier removal, and feature standardization, to generate a standardized multimodal sample set.

[0009] Optionally, step two specifically includes: Based on a standardized multimodal sample set, an independent feature extraction function is constructed for each modality to extract the instantaneous state feature vector of the current modality; The feature extraction functions are divided into image feature extraction functions, geomagnetic feature extraction functions, infrared feature extraction functions, millimeter wave feature extraction functions, and environmental perception feature extraction functions. For each mode, the memory state vector of the current time step is generated by gating update formula based on the memory state vector of the previous time step and the current instantaneous state vector; if the current time step is 1, then the memory state vector of the previous time step is the current instantaneous state vector. At consecutive time steps, the memory state vectors of each mode are stacked sequentially to form a modal memory trajectory matrix. The memory trajectory matrices of all modes are then vertically concatenated according to the modal dimension to construct a historical path state matrix. The historical path state matrix contains the modal memory trajectory matrices of all time steps before the current time step. Constructing a modal structure prior graph involves: using the memory state vector of each modality at the current time step as graph nodes, and using the structural dependency between modalities as the edge weights of the graph. The edge weights of all graphs form an adjacency matrix according to the modality index; the structural dependency is the cosine similarity between the memory state vectors of different modalities at the current time step. The adjacency matrix is ​​normalized by adding an identity matrix to generate a normalized adjacency matrix. The normalized adjacency matrix and the historical path state matrix are jointly input into the residual graph convolutional network to perform graph structure-aware propagation between modes and output the fused state vector of each mode at the current time step. The fused state vector of each modality at the current time step is nonlinearly fused with the memory state vector to obtain the structure-aware fused vector; Set the maximum historical perception length for each modality and define the asynchronous modeling window length for each modality, wherein the asynchronous modeling window length is the minimum value between the current time step and the maximum historical perception length; Based on the asynchronous modeling window length, a sliding window approach is used to extract continuous time series segments of the current modality from the standardized multimodal sample set to construct an asynchronous modeling sequence; The asynchronous modeling sequence of each mode is processed by an intramodal asynchronous modeling operation through a one-dimensional gated recurrent unit to generate the asynchronous modeling feature vector of the current time step; The asynchronous modeling feature vector of the current time step is concatenated with the structure-aware fusion vector of the corresponding modality, and a nonlinear transformation is performed to generate a time-aware feature vector. The time-aware feature vector is transformed into a causal gated weight vector through a linear transformation and a sigmoid function. Based on the causal gated weight vector, the asynchronous modeling feature vector and the structure-aware fusion vector are weighted and fused to obtain the gated-aware feature vector. At the current time step, perform cross-modal average pooling on the gated sensing feature vectors of all modalities to generate cross-modal semantic vectors; The cross-modal semantic vector is further concatenated with the fused feature vector from the previous time step along the feature dimension, and a shared modality query vector is generated through a multilayer perceptron structure. A linear mapping matrix is ​​introduced to transform the dimension of the gated sensing feature vector, generating a projection vector, and the dot product similarity score between the projection vector and the shared modality query vector is calculated. The dot product similarity scores of all modalities are normalized using the Softmax function to obtain the modal attention weights for each modality. Based on modal attention weights, the gated perception feature vectors of all modalities are weighted and fused to generate the cross-modal fusion feature vector of the current time step.

[0010] Optionally, step three specifically includes: The cross-modal fusion feature vector is used to generate a category logits vector through a fully connected layer; The category logits vector is used to calculate the occupancy state prediction probability vector using the softmax function; During the inference and verification phase, a temperature calibration factor is introduced to calibrate the occupancy state prediction probability vector; The occupancy status prediction probability vector has a dimension of 3, with each dimension corresponding to the probability that the target parking space is in an vacant state, the probability that the target parking space is in an occupied state, and the probability that the target parking space is in an uncertain state. The index corresponding to the maximum probability value in the occupancy status prediction probability vector is used as the initial occupancy status label of the target parking space at the current time step, and the corresponding probability value is used as the initial occupancy confidence score.

[0011] Optionally, step four specifically includes: Construct a parking space topology graph, in which each parking space in the target parking area is taken as a graph node, the cross-modal fusion feature vector corresponding to each parking space is taken as the embedding feature vector of the graph node, and the initial occupancy status label is taken as the pseudo label of the graph node. If the Euclidean distance between any two parking spaces is less than a set threshold, an adjacency edge is established. The weight of the adjacent edge is determined by weighted fusion based on whether the pseudo-labels of the two graph nodes are consistent and the initial occupancy confidence score; The graph nodes that have adjacent edges to the target parking space node constitute the first-order neighbor set of the target parking space; Set the maximum number of iterations, at the [number]th iteration. In the iterative layer, the Jensen-Shannon divergence function is used to calculate the first... Structural heterogeneity scoring between the target node and its neighboring nodes in the iterative layer; The first The structural heterogeneity scores of all neighbor nodes in the first-order neighbor node set are summed to obtain the total heterogeneity score; The ratio of the structural heterogeneity score of each neighbor node to the total heterogeneity score is used as the sampling probability of the neighbor node; Based on the sampling probability, for the first Perform random sampling with replacement on the set of neighbor nodes to generate the first neighbor node. The set of neighbor nodes; The first The embedded feature vector of the target node in the iterative layer and the first The embedded feature vectors of all neighbor nodes in the first-order neighbor node set are concatenated along the feature dimension and then aggregated by mean to generate the second-order neighbor node. Embedded feature vectors of target nodes in the iterative layer; The embedded feature vector of the maximum iteration layer is used as the spatial consistency feature vector; The spatial consistency feature vector is input into the multilayer perceptron structure to perform linear mapping and Softmax activation, generating a spatial consistency state prediction vector. The index corresponding to the maximum probability value in the spatial consistency state prediction vector is used as the spatial consistency state label of the target parking space in the target parking area, and the corresponding probability value is used as the spatial consistency confidence score. The spatial consistency status labels include 0: idle state, 1: occupied state, and 2: uncertain state.

[0012] Optionally, step five specifically includes: Combine the spatial consistency feature vectors corresponding to two nodes with the same spatial consistency state label into a positive sample pair; Combine the spatial consistency feature vectors corresponding to two nodes with different spatial consistency state labels into a negative sample pair; Based on positive and negative sample pairs, a contrastive learning loss is constructed using the InfoNCE function. The optimization objective of the contrastive learning loss is to reduce the distance between positive sample pairs and increase the distance between negative sample pairs. The cross-entropy between the spatial consistency feature vector and the initial occupancy state label is used as the classification loss. The contrastive learning loss and classification loss are weighted and combined to form a joint loss function, which is then used to conduct contrastive training and updates on the improved GraphSAGE model.

[0013] Optionally, step six specifically includes: The visualization status layer includes a parking space number layer, a status label layer, a confidence heat map layer, and a status prediction boundary layer; The visualization status layer is rendered into a visualization status map of the target parking space based on the status labels, and the spatial consistency confidence score is marked. Among them, "0: Idle status" corresponds to green, "1: Occupied status" corresponds to red, and "2: Uncertain status" corresponds to yellow. Based on the visualized status diagrams of multiple target parking spaces, a status distribution map of the target parking area is constructed, and the status distribution map of the target parking area is transmitted to the user terminal.

[0014] Optionally, step seven specifically includes: The user interaction feedback data includes spatial consistency status label correction information, spatial consistency confidence score correction information, visualization layer annotation information, and abnormal scene description information. The user interaction feedback data is aligned and matched with the original sample set to form a feedback sample set, and incremental updates are performed on the improved DynMM structure and the improved GraphSAGE model based on the feedback sample set.

[0015] The beneficial effects of this invention are: First, this invention collects multi-source sensor data including video image modes, geomagnetic signal modes, infrared thermal imaging modes, ultrasonic modes, and other auxiliary modes, and generates a standardized multi-modal sample set through modal alignment, time step unification, scale normalization, and structural encoding, providing a high-quality and low-noise data foundation for dynamic fusion and parking space status modeling.

[0016] Secondly, this invention introduces an improved DynMM structure as the core of multimodal data fusion. This improved DynMM structure integrates a modal path memory mechanism, a modal structure prior graph constraint mechanism, and an asynchronous temporal modeling branching and causal inference-assisted gating mechanism, effectively realizing the dynamic fusion of multi-source modal information in the spatiotemporal dimension. The improved DynMM structure overcomes the problem of weak weight adaptation in complex scenarios of traditional static weighted fusion methods, significantly improving robustness to modal missingness and intermodal noise differences.

[0017] Furthermore, this invention utilizes an improved GraphSAGE model to model the spatial consistency of cross-modal fusion feature vectors. This improved GraphSAGE model introduces an adaptive sampling mechanism for structural heterogeneity and employs Jensen-Shannon divergence to measure the structural differences between neighboring nodes, effectively enhancing the model's ability to perceive and infer the topological relationships of parking spaces. Simultaneously, the improved GraphSAGE model, combined with a contrastive learning mechanism, improves the discriminability of node features and the clarity of state classification boundaries, thereby enhancing overall recognition performance.

[0018] In summary, this invention combines the dynamic modal fusion capability of the improved DynMM structure with the spatial consistency modeling capability of the improved GraphSAGE model, enabling high-precision identification and stable management of parking space status in complex urban environments. It has the advantages of strong fusion adaptability, reasonable spatial modeling, high identification accuracy, and edge deployment friendliness. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0020] Figure 1 This is a schematic diagram of a smart parking management system based on multi-source data fusion proposed in this invention; Figure 2 This is a flowchart of an improved DynMM structure in a smart parking management system based on multi-source data fusion proposed in this invention. Figure 3 This is a flowchart of the modeling process for an improved GraphSAGE model in a smart parking management system based on multi-source data fusion, as proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figures 1-3 A smart parking space management system based on multi-source data fusion includes the following steps: Data acquisition and processing module: used to collect multi-source raw data of the target parking area and preprocess it to generate a standardized multimodal sample set; Multimodal fusion module: Used to perform fusion modeling on standardized multimodal sample sets through an improved DynMM structure to generate cross-modal fusion feature vectors; Initial state identification module: used to generate initial occupancy state labels and initial occupancy confidence scores; Spatial Consistency Modeling Module: Used to generate spatial consistency state labels and spatial consistency confidence scores using the improved GraphSAGE model; The comparative training module is used to perform comparative training and updates on the improved GraphSAGE model. Visualization module: Used to generate a status distribution map of the target parking area; Feedback Update Module: Used to collect user interaction feedback data and perform incremental updates on the improved DynMM structure and the improved GraphSAGE model.

[0023] In this embodiment, the modules are interconnected using the following method: Step 1: Collect multi-source raw data of the target parking area and preprocess the multi-source raw data to generate a standardized multimodal sample set; Step 2: Input the standardized multimodal sample set into the improved DynMM structure, update the modal path memory, construct the modal structure prior graph, perform asynchronous modeling and causal gating adjustment, and generate cross-modal fusion feature vectors through cross-modal attention; Step 3: Based on the cross-modal fusion feature vector, perform parking space status recognition operation to generate the initial occupancy status label and initial occupancy confidence score of the target parking space; Step 4: Input the cross-modal fusion feature vectors within the target parking area into the improved GraphSAGE model to perform spatial consistency modeling, and generate spatial consistency state labels and spatial consistency confidence scores; Step 5: Based on the spatially consistent state labels, construct positive sample pairs and negative sample pairs, and perform comparative training and updates on the improved GraphSAGE model; Step 6: Based on the spatial consistency status label and spatial consistency confidence score, generate a visual status layer of the target parking space and a status distribution map of the target parking area; Step 7: Collect user interaction feedback data and incrementally update the improved DynMM structure and the improved GraphSAGE model.

[0024] In this embodiment, step one specifically includes: Image acquisition equipment, geomagnetic sensors, infrared sensors, millimeter-wave radar, brightness sensors, temperature and humidity sensors, and rain and snow sensing sensors are deployed in the target parking area to simultaneously collect raw data from multiple sources. The multi-source raw data includes image frame sequences, magnetic field strength data, infrared reflection signals, distance echo data, light intensity data, temperature data, relative humidity data, and precipitation intensity data; The multi-source raw data undergoes preprocessing operations, including time-series alignment, data normalization, outlier removal, and feature standardization, to generate a standardized multimodal sample set. Specifically: The image frame sequence is subjected to pixel grayscale conversion, illumination equalization and distortion correction operations, and the image frame sequence is sliced ​​and resampled through a sliding window of a set size to generate a normalized image tensor sequence. The magnetic field strength data is filtered by moving average to remove DC offset, and transient noise is removed by first-order difference operation. After normalization, a geomagnetic feature sequence is generated. Bandpass filtering and amplitude normalization are performed on the infrared reflection signal and range echo data to generate infrared feature sequences and millimeter wave feature sequences; The light intensity data, temperature data, relative humidity data, and precipitation intensity data are averaged and normalized based on a sliding time window of a set length, and then spliced ​​into a four-dimensional environmental perception feature vector. The environmental perception feature vector sequence is then spliced ​​together on a continuous time step. The standardized image tensor sequence, geomagnetic feature sequence, infrared feature sequence, millimeter wave feature sequence and environmental perception feature vector sequence are aligned according to time steps to form a standardized multimodal sample set.

[0025] In this embodiment, step two specifically includes: Based on a standardized multimodal sample set, an independent feature extraction function is constructed for each modality to extract the instantaneous state feature vector of the current modality; The feature extraction functions are divided into image feature extraction functions, geomagnetic feature extraction functions, infrared feature extraction functions, millimeter wave feature extraction functions, and environmental perception feature extraction functions. The image feature extraction function performs three layers of 3×3 convolutional coding, spatial max pooling and Flatten flattening operations on the standardized image tensor sequence in sequence, and generates an instantaneous image state feature vector through a fully connected layer. The geomagnetic feature extraction function performs one-dimensional convolution and GELU activation on the geomagnetic feature sequence, and generates a geomagnetic instantaneous state feature vector through a fully connected layer. The infrared feature extraction function extracts high-frequency and low-frequency features from the infrared feature sequence using one-dimensional convolution with kernel sizes of 3 and 11, respectively. After concatenating the high-frequency and low-frequency features in the channel dimension, the number of channels is compressed by 1×1 convolution, and an infrared instantaneous state feature vector is generated through a fully connected layer. The millimeter-wave feature extraction function performs a short-time Fourier transform on the millimeter-wave feature sequence and generates a millimeter instantaneous state feature vector through frequency domain convolution and average pooling operations. The environmental perception feature extraction function extracts the cross-time step semantics of the environmental perception feature vector sequence through a long short-term memory network, and generates the environmental perception instantaneous state feature vector by passing the hidden state at the last moment through a fully connected layer. For each mode, the memory state vector of the current time step is generated by gating update formula based on the memory state vector of the previous time step and the current instantaneous state vector; if the current time step is 1, then the memory state vector of the previous time step is the current instantaneous state vector. At consecutive time steps, the memory state vectors of each mode are stacked sequentially to form a modal memory trajectory matrix. The memory trajectory matrices of all modes are then vertically concatenated according to the modal dimension to construct a historical path state matrix. The historical path state matrix contains the modal memory trajectory matrices of all time steps before the current time step. Constructing a modal structure prior graph involves: using the memory state vector of each modality at the current time step as graph nodes, and using the structural dependency between modalities as the edge weights of the graph. The edge weights of all graphs form an adjacency matrix according to the modality index; the structural dependency is the cosine similarity between the memory state vectors of different modalities at the current time step. The adjacency matrix is ​​normalized by adding an identity matrix to generate a normalized adjacency matrix. The normalized adjacency matrix and the historical path state matrix are jointly input into the residual graph convolutional network to perform graph structure-aware propagation between modes and output the fused state vector of each mode at the current time step. The fused state vector of each modality at the current time step is nonlinearly fused with the memory state vector to obtain the structure-aware fused vector; Set the maximum historical perception length for each modality and define the asynchronous modeling window length for each modality, wherein the asynchronous modeling window length is the minimum value between the current time step and the maximum historical perception length; Based on the asynchronous modeling window length, a sliding window approach is used to extract continuous time series segments of the current modality from the standardized multimodal sample set to construct an asynchronous modeling sequence; The asynchronous modeling sequence of each mode is processed by an intramodal asynchronous modeling operation through a one-dimensional gated recurrent unit to generate the asynchronous modeling feature vector of the current time step; The asynchronous modeling feature vector of the current time step is concatenated with the structure-aware fusion vector of the corresponding modality, and a nonlinear transformation is performed to generate a time-aware feature vector. The time-aware feature vector is transformed into a causal gated weight vector through a linear transformation and a sigmoid function. Based on the causal gated weight vector, the asynchronous modeling feature vector and the structure-aware fusion vector are weighted and fused to obtain the gated-aware feature vector. At the current time step, perform cross-modal average pooling on the gated sensing feature vectors of all modalities to generate cross-modal semantic vectors; The cross-modal semantic vector is further concatenated with the fused feature vector from the previous time step along the feature dimension, and a shared modality query vector is generated through a multilayer perceptron structure. A linear mapping matrix is ​​introduced to transform the dimension of the gated sensing feature vector, generating a projection vector, and the dot product similarity score between the projection vector and the shared modality query vector is calculated. The dot product similarity scores of all modalities are normalized using the Softmax function to obtain the modal attention weights for each modality. Based on modal attention weights, the gated perception feature vectors of all modalities are weighted and fused to generate the cross-modal fusion feature vector of the current time step.

[0026] In this invention, by updating the modal path memory mechanism, the continuous expression of information and dynamic state perception of each modality in the temporal dimension are realized, effectively alleviating the semantic misalignment problem caused by asynchronous acquisition of multimodal data. Constructing a modal structure prior graph and introducing a residual graph convolutional network can capture the structural dependencies between different modalities at the current moment, improving the spatial adaptability of cross-modal collaborative modeling and the semantic consistency of feature fusion. By adopting asynchronous modeling and causal gating adjustment strategies, each modality can fully model the historical state evolution process within its perception range, and the gating mechanism suppresses invalid historical disturbances, enhancing the adaptability to sudden state changes and non-uniform sampling time series. Finally, by automatically mining the semantic contribution differences between modalities through a cross-modal attention mechanism, and dynamically weighting and generating cross-modal fusion feature vectors, the comprehensive perception ability and recognition robustness of parking state information in complex scenarios are improved.

[0027] In this embodiment, step three specifically includes: The cross-modal fusion feature vector is used to generate a category logits vector through a fully connected layer; The category logits vector is used to calculate the occupancy state prediction probability vector using the softmax function; During the inference and verification phase, a temperature calibration factor is introduced to calibrate the occupancy state prediction probability vector; The temperature calibration factor is determined by minimizing the cross-entropy loss between the temperature-calibrated occupancy state prediction probability vector and the actual parking space status label vector, and the temperature calibration factor corresponding to the minimum cross-entropy loss is used as a fixed parameter in the inference and verification stage; the temperature calibration factor does not participate in the training process, but only calibrates the occupancy state prediction probability vector in the inference and verification stage. The occupancy status prediction probability vector has a dimension of 3, with each dimension corresponding to the probability that the target parking space is in an vacant state, the probability that the target parking space is in an occupied state, and the probability that the target parking space is in an uncertain state. The index corresponding to the maximum probability value in the occupancy status prediction probability vector is used as the initial occupancy status label of the target parking space at the current time step, and the corresponding probability value is used as the initial occupancy confidence score.

[0028] In this invention, a temperature calibration factor is introduced during the inference and verification stage. This effectively alleviates the overconfidence prediction problem in complex or modally incomplete scenarios. By calibrating the occupancy state prediction probability vector with temperature, the output confidence level is made closer to the true distribution, thereby improving the reliability and interpretability of the prediction results in engineering applications. The temperature calibration factor, as a non-training parameter, is adaptively determined by minimizing the cross-entropy loss after calibration. This enables probability adjustment and confidence redistribution of occupancy state label predictions, helping to improve the recognition capability of this invention under occlusion or modality loss conditions.

[0029] In this embodiment, step four specifically includes: Construct a parking space topology graph, in which each parking space in the target parking area is taken as a graph node, the cross-modal fusion feature vector corresponding to each parking space is taken as the embedding feature vector of the graph node, and the initial occupancy status label is taken as the pseudo label of the graph node. If the Euclidean distance between any two parking spaces is less than a set threshold, an adjacency edge is established. The weight of the adjacent edge is determined by weighted fusion based on whether the pseudo-labels of the two graph nodes are consistent and the initial occupancy confidence score; The graph nodes that have adjacent edges to the target parking space node constitute the first-order neighbor set of the target parking space; Set the maximum number of iterations, at the [number]th iteration. In the iterative layer, the Jensen-Shannon divergence function is used to calculate the first... Structural heterogeneity scoring between the target node and its neighboring nodes in the iterative layer: ; in, Represents the target node In the Embedded feature vectors of iterative layers Neighboring nodes In the Embedded feature vectors of the iterative layer; The first The structural heterogeneity scores of all neighbor nodes in the first-order neighbor node set are summed to obtain the total heterogeneity score; The ratio of the structural heterogeneity score of each neighbor node to the total heterogeneity score is used as the sampling probability of the neighbor node; Based on the sampling probability, for the first Perform random sampling with replacement on the set of neighbor nodes to generate the first neighbor node. The set of neighbor nodes; The first The embedded feature vector of the target node in the iterative layer and the first The embedded feature vectors of all neighbor nodes in the first-order neighbor node set are concatenated along the feature dimension and then aggregated by mean to generate the second-order neighbor node. Embedded feature vectors of target nodes in the iterative layer; The embedded feature vector of the maximum iteration layer is used as the spatial consistency feature vector; The spatial consistency feature vector is input into the multilayer perceptron structure to perform linear mapping and Softmax activation, generating a spatial consistency state prediction vector. The index corresponding to the maximum probability value in the spatial consistency state prediction vector is used as the spatial consistency state label of the target parking space in the target parking area, and the corresponding probability value is used as the spatial consistency confidence score. The spatial consistency status labels include 0: idle state, 1: occupied state, and 2: uncertain state.

[0030] In this invention, a refined modeling of the spatial distribution relationship of parking spaces within a target parking area is achieved by constructing a parking space spatial topology graph and introducing a structural heterogeneity measurement mechanism based on Jensen-Shannon divergence. The improved GraphSAGE model not only considers the spatial distance constraints between adjacent parking spaces but also integrates pseudo-labels and confidence information, making the graph structure more consistent with the spatial characteristics of real parking scenarios. By probabilistically sampling and hierarchically aggregating the structural heterogeneity scores of neighboring nodes, neighboring nodes with high structural similarity are adaptively selected, enhancing the relevance and noise resistance of feature propagation while maintaining the sparsity of the graph structure.

[0031] In this embodiment, step five specifically includes: Combine the spatial consistency feature vectors corresponding to two nodes with the same spatial consistency state label into a positive sample pair; Combine the spatial consistency feature vectors corresponding to two nodes with different spatial consistency state labels into a negative sample pair; Based on positive and negative sample pairs, a contrastive learning loss is constructed using the InfoNCE function. The optimization objective of the contrastive learning loss is to reduce the distance between positive sample pairs and increase the distance between negative sample pairs. The cross-entropy between the spatial consistency feature vector and the initial occupancy state label is used as the classification loss. The contrastive learning loss and classification loss are weighted and combined to form a joint loss function, which is then used to conduct contrastive training and updates on the improved GraphSAGE model.

[0032] In this embodiment, step six specifically includes: The visualization status layer includes a parking space number layer, a status label layer, a confidence heat map layer, and a status prediction boundary layer; The visualization status layer is rendered into a visualization status map of the target parking space based on the status labels, and the spatial consistency confidence score is marked. Among them, "0: Idle status" corresponds to green, "1: Occupied status" corresponds to red, and "2: Uncertain status" corresponds to yellow. Based on the visualized status diagrams of multiple target parking spaces, a status distribution map of the target parking area is constructed, and the status distribution map of the target parking area is transmitted to the user terminal.

[0033] In this embodiment, step seven specifically includes: The user interaction feedback data includes spatial consistency status label correction information, spatial consistency confidence score correction information, visualization layer annotation information, and abnormal scene description information. The user interaction feedback data is aligned and matched with the original sample set to form a feedback sample set, and incremental updates are performed on the improved DynMM structure and the improved GraphSAGE model based on the feedback sample set.

[0034] Example 1: To verify the feasibility of this invention in practice, it was applied to a smart parking management system in an underground parking lot in a city. This underground parking lot is surrounded by dense commercial, office, and residential resources, with a daily traffic flow exceeding 4,500 vehicles. It suffers from problems such as inaccurate parking space status recognition, complex deployment environment of sensing equipment, data susceptibility to obstruction and interference, and untimely system response, severely restricting the parking lot's turnover efficiency and the level of intelligent operation and management. The original smart parking management system used a traditional method based on a single geomagnetic sensor and video image recognition to determine parking space status. When ambient light was insufficient, vehicles severely obstructed the view, or geomagnetic signals drifted, the system's recognition accuracy dropped significantly, with frequent misjudgments and omissions, and delays in updating parking space status information.

[0035] By employing this invention, multi-source sensing data is collected through the deployment of multi-source sensing devices, and time-step synchronization, modal alignment, and normalized encoding operations are performed to construct a standardized multimodal sample set. Dynamic feature fusion is then performed on the standardized multimodal sample set based on an improved DynMM structure to generate cross-modal fusion feature vectors. These cross-modal fusion feature vectors are input into an improved GraphSAGE model, where Jensen-Shannon divergence is used to measure structural differences between neighboring nodes, guiding the spatial propagation of parking space node state information. Furthermore, a contrastive learning mechanism is introduced to optimize spatial consistency judgment, making the parking space state recognition results more robust and spatially logically consistent.

[0036] After deploying the system of this invention, a 30-day continuous practical application test was conducted. The performance of the system of this invention was compared with three commonly used parking space status recognition schemes: Scheme A: using only a geomagnetic sensor and employing a threshold-based method for parking space status recognition; Scheme B: performing independent recognition of images and geomagnetic signals, and then determining the status based on set rules; Scheme C: stitching together features from multiple sensors and inputting them into an XGBoost classifier to output parking space status labels. The performance metrics compared included: average recognition accuracy, status update latency, occlusion robustness score (0-1), interference resistance score (0-1), edge inference time, deployment resource utilization, and system stability score (0-1). The experimental results are shown in Table 1.

[0037] Table 1 Performance Evaluation Comparison Table of the Invention and the Comparative Solution ; As shown in Table 1, this invention outperforms the comparative schemes in several key performance indicators. In terms of average recognition accuracy, this invention achieves 97.8%, representing improvements of 11.5%, 7.1%, and 5.3% compared to comparative schemes A, B, and C, respectively. Regarding state update latency, this invention controls the latency at 282ms, a significant reduction of 45.8% compared to comparative scheme A's 520ms, resulting in a faster response speed. This invention achieves scores of 0.84 and 0.88 in occlusion robustness and interference resistance, respectively, significantly outperforming the three comparative schemes, indicating strong adaptability to complex environmental factors such as occlusion, poor lighting, and signal noise. While the edge inference time is slightly higher than comparative scheme A, it is significantly lower than comparative schemes B and C, balancing model complexity with real-time edge computing. In terms of deployment resource utilization, this invention maintains a utilization rate of 38.9%, 13.2 percentage points lower than comparative scheme C, which is beneficial for long-term stable operation on edge terminal devices with limited computing power. In terms of system stability score, the present invention achieved 0.91, which is significantly better than the comparative scheme, verifying that the present invention has good recognition accuracy, real-time response capability and deployment stability in complex urban parking environments.

[0038] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A smart parking space management system based on multi-source data fusion, characterized in that, include: Data acquisition and processing module: used to collect multi-source raw data of the target parking area and preprocess it to generate a standardized multimodal sample set; Multimodal fusion module: Used to perform fusion modeling on standardized multimodal sample sets through an improved DynMM structure to generate cross-modal fusion feature vectors; Initial state identification module: used to generate initial occupancy state labels and initial occupancy confidence scores; Spatial Consistency Modeling Module: Used to generate spatial consistency state labels and spatial consistency confidence scores using the improved GraphSAGE model; The comparative training module is used to perform comparative training and updates on the improved GraphSAGE model. Visualization module: Used to generate a status distribution map of the target parking area; Feedback Update Module: Used to collect user interaction feedback data and perform incremental updates on the improved DynMM structure and the improved GraphSAGE model.

2. The intelligent parking space management system based on multi-source data fusion according to claim 1, characterized in that, The modules are connected in the following way: Step 1: Collect multi-source raw data of the target parking area and preprocess the multi-source raw data to generate a standardized multimodal sample set; Step 2: Input the standardized multimodal sample set into the improved DynMM structure, update the modal path memory, construct the modal structure prior graph, perform asynchronous modeling and causal gating adjustment, and generate cross-modal fusion feature vectors through cross-modal attention; Step 3: Based on the cross-modal fusion feature vector, perform parking space status recognition operation to generate the initial occupancy status label and initial occupancy confidence score of the target parking space; Step 4: Input the cross-modal fusion feature vectors within the target parking area into the improved GraphSAGE model to perform spatial consistency modeling, and generate spatial consistency state labels and spatial consistency confidence scores; Step 5: Based on the spatially consistent state labels, construct positive sample pairs and negative sample pairs, and perform comparative training and updates on the improved GraphSAGE model; Step 6: Based on the spatial consistency status label and spatial consistency confidence score, generate a visual status layer of the target parking space and a status distribution map of the target parking area; Step 7: Collect user interaction feedback data and incrementally update the improved DynMM structure and the improved GraphSAGE model.

3. The intelligent parking space management system based on multi-source data fusion according to claim 2, characterized in that, Step one specifically includes: Image acquisition equipment, geomagnetic sensors, infrared sensors, millimeter-wave radar, brightness sensors, temperature and humidity sensors, and rain and snow sensing sensors are deployed in the target parking area to simultaneously collect raw data from multiple sources. The multi-source raw data includes image frame sequences, magnetic field strength data, infrared reflection signals, distance echo data, light intensity data, temperature data, relative humidity data, and precipitation intensity data; The multi-source raw data undergoes preprocessing operations, including time alignment, data normalization, outlier removal, and feature standardization, to generate a standardized multimodal sample set.

4. The intelligent parking space management system based on multi-source data fusion according to claim 2, characterized in that, Step two specifically includes: Based on a standardized multimodal sample set, an independent feature extraction function is constructed for each modality to extract the instantaneous state feature vector of the current modality; The feature extraction functions are divided into image feature extraction functions, geomagnetic feature extraction functions, infrared feature extraction functions, millimeter wave feature extraction functions, and environmental perception feature extraction functions. For each mode, the memory state vector of the current time step is generated by gating update formula based on the memory state vector of the previous time step and the current instantaneous state vector; if the current time step is 1, then the memory state vector of the previous time step is the current instantaneous state vector. At consecutive time steps, the memory state vectors of each mode are stacked sequentially to form a modal memory trajectory matrix. The memory trajectory matrices of all modes are then vertically concatenated according to the modal dimension to construct a historical path state matrix. The historical path state matrix contains the modal memory trajectory matrices of all time steps before the current time step. Constructing a modal structure prior graph involves: using the memory state vector of each modality at the current time step as graph nodes, and using the structural dependency between modalities as the edge weights of the graph. The edge weights of all graphs form an adjacency matrix according to the modality index; the structural dependency is the cosine similarity between the memory state vectors of different modalities at the current time step. The adjacency matrix is ​​normalized by adding an identity matrix to generate a normalized adjacency matrix. The normalized adjacency matrix and the historical path state matrix are jointly input into the residual graph convolutional network to perform graph structure-aware propagation between modes and output the fused state vector of each mode at the current time step. The fused state vector of each modality at the current time step is nonlinearly fused with the memory state vector to obtain the structure-aware fused vector; Set the maximum historical perception length for each modality and define the asynchronous modeling window length for each modality, wherein the asynchronous modeling window length is the minimum value between the current time step and the maximum historical perception length; Based on the asynchronous modeling window length, a sliding window approach is used to extract continuous time series segments of the current modality from the standardized multimodal sample set to construct an asynchronous modeling sequence; The asynchronous modeling sequence of each mode is processed by an intramodal asynchronous modeling operation through a one-dimensional gated recurrent unit to generate the asynchronous modeling feature vector of the current time step; The asynchronous modeling feature vector of the current time step is concatenated with the structure-aware fusion vector of the corresponding modality, and a nonlinear transformation is performed to generate a time-aware feature vector. The time-aware feature vector is transformed into a causal gated weight vector through a linear transformation and a sigmoid function. Based on the causal gated weight vector, the asynchronous modeling feature vector and the structure-aware fusion vector are weighted and fused to obtain the gated-aware feature vector. At the current time step, perform cross-modal average pooling on the gated sensing feature vectors of all modalities to generate cross-modal semantic vectors; The cross-modal semantic vector is further concatenated with the fused feature vector from the previous time step along the feature dimension, and a shared modality query vector is generated through a multilayer perceptron structure. A linear mapping matrix is ​​introduced to transform the dimension of the gated sensing feature vector, generating a projection vector, and the dot product similarity score between the projection vector and the shared modality query vector is calculated. The dot product similarity scores of all modalities are normalized using the Softmax function to obtain the modal attention weights for each modality. Based on modal attention weights, the gated perception feature vectors of all modalities are weighted and fused to generate the cross-modal fusion feature vector of the current time step.

5. A smart parking space management system based on multi-source data fusion according to claim 2, characterized in that, Step three specifically includes: The cross-modal fusion feature vector is used to generate a category logits vector through a fully connected layer; The category logits vector is used to calculate the occupancy state prediction probability vector using the softmax function; During the inference and verification phase, a temperature calibration factor is introduced to calibrate the occupancy state prediction probability vector; The occupancy status prediction probability vector has a dimension of 3, with each dimension corresponding to the probability that the target parking space is in an vacant state, the probability that the target parking space is in an occupied state, and the probability that the target parking space is in an uncertain state. The index corresponding to the maximum probability value in the occupancy status prediction probability vector is used as the initial occupancy status label of the target parking space at the current time step, and the corresponding probability value is used as the initial occupancy confidence score.

6. A smart parking space management system based on multi-source data fusion according to claim 2, characterized in that, Step four specifically includes: Construct a parking space topology graph, in which each parking space in the target parking area is taken as a graph node, the cross-modal fusion feature vector corresponding to each parking space is taken as the embedding feature vector of the graph node, and the initial occupancy status label is taken as the pseudo label of the graph node. If the Euclidean distance between any two parking spaces is less than a set threshold, an adjacency edge is established. The weight of the adjacent edge is determined by weighted fusion based on whether the pseudo-labels of the two graph nodes are consistent and the initial occupancy confidence score; The graph nodes that have adjacent edges to the target parking space node constitute the first-order neighbor set of the target parking space; Set the maximum number of iterations, at the [number]th iteration. In the iterative layer, the Jensen-Shannon divergence function is used to calculate the first... Structural heterogeneity scoring between the target node and its neighboring nodes in the iterative layer; The first The structural heterogeneity scores of all neighbor nodes in the first-order neighbor node set are summed to obtain the total heterogeneity score; The ratio of the structural heterogeneity score of each neighbor node to the total heterogeneity score is used as the sampling probability of the neighbor node; Based on the sampling probability, for the first Perform random sampling with replacement on the set of neighbor nodes to generate the first neighbor node. The set of neighbor nodes; The first The embedded feature vector of the target node in the iterative layer and the first The embedded feature vectors of all neighbor nodes in the first-order neighbor node set are concatenated along the feature dimension and then aggregated by mean to generate the second-order neighbor node. Embedded feature vectors of target nodes in the iterative layer; The embedded feature vector of the maximum iteration layer is used as the spatial consistency feature vector; The spatial consistency feature vector is input into the multilayer perceptron structure to perform linear mapping and Softmax activation, generating a spatial consistency state prediction vector. The index corresponding to the maximum probability value in the spatial consistency state prediction vector is used as the spatial consistency state label of the target parking space in the target parking area, and the corresponding probability value is used as the spatial consistency confidence score. The spatial consistency status labels include 0: idle state, 1: occupied state, and 2: uncertain state.

7. A smart parking space management system based on multi-source data fusion according to claim 2, characterized in that, Step five specifically includes: Combine the spatial consistency feature vectors corresponding to two nodes with the same spatial consistency state label into a positive sample pair; Combine the spatial consistency feature vectors corresponding to two nodes with different spatial consistency state labels into a negative sample pair; Based on positive and negative sample pairs, a contrastive learning loss is constructed using the InfoNCE function. The optimization objective of the contrastive learning loss is to reduce the distance between positive sample pairs and increase the distance between negative sample pairs. The cross-entropy between the spatial consistency feature vector and the initial occupancy state label is used as the classification loss. The contrastive learning loss and classification loss are weighted and combined to form a joint loss function, which is then used to conduct contrastive training and updates on the improved GraphSAGE model.

8. A smart parking space management system based on multi-source data fusion according to claim 2, characterized in that, Step six specifically includes: The visualization status layer includes a parking space number layer, a status label layer, a confidence heat map layer, and a status prediction boundary layer; The visualization status layer is rendered into a visualization status map of the target parking space based on the status labels, and the spatial consistency confidence score is marked. Among them, "0: Idle status" corresponds to green, "1: Occupied status" corresponds to red, and "2: Uncertain status" corresponds to yellow. Based on the visualized status diagrams of multiple target parking spaces, a status distribution map of the target parking area is constructed, and the status distribution map of the target parking area is transmitted to the user terminal.

9. A smart parking space management system based on multi-source data fusion according to claim 2, characterized in that, Step seven specifically includes: The user interaction feedback data includes spatial consistency status label correction information, spatial consistency confidence score correction information, visualization layer annotation information, and abnormal scene description information. The user interaction feedback data is aligned and matched with the original sample set to form a feedback sample set, and incremental updates are performed on the improved DynMM structure and the improved GraphSAGE model based on the feedback sample set.

Images