Smart park multi-source data fusion method and system based on AI

Abstract

The invention discloses an AI-based smart park multi-source data fusion method and system, and the method comprises the steps: generating a time-space aligned standardized data flow according to environment parameters, energy consumption waveforms, security signals and personnel trajectory data collected by a heterogeneous sensor network; generating a multi-modal fusion feature matrix based on the standardized data stream; according to the multi-modal fusion feature matrix, generating a three-dimensional twinborn body including the equipment state, the people flow density and the energy consumption hot spot in real time; inputting the three-dimensional twin into a multi-target constrained reinforcement learning algorithm, and fusing real-time data and prediction data to generate a Pareto optimal solution set; and based on the Pareto optimal solution set, generating a final instruction set for driving park equipment regulation and control, and triggering collaborative response of a security and protection system and an energy consumption system at the same time. According to the embodiment of the invention, intelligent upgrading of park management can be realized through cross-modal feature extraction, dynamic digital twin modeling and reinforcement learning optimization.

Description

Technical Field

[0001] The present invention belongs to the field of data fusion technology, and in particular to an AI-based smart park multi-source data fusion method and system. Background Art

[0002] With the rapid development of smart city construction, smart parks, as a key component of urban intelligence, face the challenges of multi-source heterogeneous data fusion and real-time decision-making. Traditional park management systems typically use independent subsystems to process data for environmental monitoring, energy consumption management, security monitoring, and other fields, resulting in severe data silos and a lack of cross-modal correlation analysis capabilities. Existing rule-based data fusion methods are difficult to adapt to the dynamically changing park environment, while simple machine learning models cannot effectively handle the complex correlations of spatiotemporal and multidimensional data. In particular, existing systems often rely on manual experience or static threshold judgments for abnormal event detection and resource optimization, resulting in delayed responses and low efficiency. Summary of the Invention

[0003] The purpose of this invention is to provide an AI-based smart park multi-source data fusion method and system to address the shortcomings of the existing technology, and to achieve intelligent upgrades in park management through cross-modal feature extraction, dynamic digital twin modeling and reinforcement learning optimization.

[0004] One embodiment of the present application provides an AI-based smart park multi-source data fusion method, the method comprising: Based on the environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data collected by the heterogeneous sensor network, the raw data is dynamically cleaned using an adaptive noise suppression function to generate a standardized data stream aligned in time and space. The noise suppression function dynamically adjusts the filtering threshold based on the sensor type. Based on the standardized data stream, a cross-modal attention mechanism is used to extract related features of the environment, energy consumption, security, and personnel flow, and generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation; Based on the multimodal fusion feature matrix, a spatiotemporal graph neural network is used to construct a dynamic digital twin model of the park. By embedding a dynamic topology update mechanism, a three-dimensional twin containing equipment status, crowd density, and energy consumption hotspots is generated in real time. Inputting the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generating resource optimization strategies through a virtual simulation environment, and fusing real-time data with predicted data to generate a Pareto optimal solution set; Based on the Pareto optimal solution set, the executable strategy is screened through the virtual-to-real mapping verification engine to generate the final instruction set for driving the control of the campus equipment, while triggering the coordinated response of the security system and the energy consumption system.

[0005] Optionally, the environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data collected by the heterogeneous sensor network are dynamically cleaned using an adaptive noise suppression function to generate a time-space aligned standardized data stream, wherein the noise suppression function dynamically adjusts the filtering threshold according to the sensor type, including: Classify the raw data according to the sensor type and generate four types of data streams: environmental parameters, energy consumption waveforms, security signals, and personnel trajectories; According to the noise characteristics of each type of data stream, a Gaussian distribution model is dynamically fitted based on historical data to generate a dynamic filtering threshold that matches the sensor type; The dynamic filtering threshold is input into the adaptive median filter function to perform frame-by-frame noise suppression on the original data to obtain a denoised data stream; The denoised data stream is timestamp-calibrated and spatial coordinates are uniformly mapped to generate a standardized data stream that is time-space aligned. The time alignment accuracy is at the millisecond level, and the spatial error threshold is less than 0.5 meters.

[0006] Optionally, based on the standardized data stream, a cross-modal attention mechanism is used to extract related features of the environment, energy consumption, security, and personnel flow to generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation, including: Input the standardized data stream into the modal encoder to generate the environmental parameter modal vector, energy consumption waveform modal vector, security signal modal vector and personnel trajectory modal vector; Through the multi-head cross-modal attention mechanism, the dynamic correlation weights between modal vectors are calculated, where the attention weights corresponding to abnormal data are enhanced according to the exponential function; Perform weighted fusion of modal vectors based on dynamic association weights to generate a cross-modal joint feature vector; The cross-modal joint feature vector is processed hierarchically using gated recurrent units to extract long-term and short-term dependency features in the temporal dimension. The hierarchically processed feature vectors are spliced ​​according to the spatial dimension to generate a multimodal fusion feature matrix, where the matrix dimension corresponds one-to-one to the physical space grid of the park.

[0007] Optionally, the multimodal fusion feature matrix is ​​used to construct a dynamic digital twin model of the park using a spatiotemporal graph neural network, and a dynamic topology update mechanism is embedded to generate a three-dimensional twin containing equipment status, crowd density, and energy consumption hotspots in real time, including: According to the physical topology of the park, the multimodal fusion feature matrix is ​​mapped into the node features of the spatiotemporal graph to construct the initial spatiotemporal graph structure; Through the dynamic topology update mechanism, areas with sudden changes in pedestrian density and nodes with abnormal equipment status are detected in real time, generating spatiotemporal graph edge connections that are dynamically added and deleted; Embed a gated spatiotemporal convolution module in the spatiotemporal graph neural network to iteratively update node features to capture device state drift characteristics; The updated spatiotemporal graph node features are input into the 3D feature reconstruction layer to generate 3D feature voxels containing equipment health index, thermal distribution of human flow, and energy consumption gradient; Physically constrained back projection of three-dimensional feature voxels is performed to generate a three-dimensional digital twin that is updated synchronously with the real park, with a resolution of centimeters.

[0008] Optionally, the three-dimensional twin is input into a multi-objective constraint reinforcement learning algorithm, resource optimization strategies are iteratively generated through a virtual simulation environment, and real-time data and predicted data are integrated to generate a Pareto optimal solution set, including: Map the 3D digital twin into the state space of the virtual simulation environment and define three optimization objective functions: energy cost, security risk, and service efficiency; Based on the double-delayed deep deterministic policy gradient algorithm, device control strategies are iteratively generated in a virtual environment. Each strategy corresponds to a multi-objective weight combination. Fusion of real-time sensor data with LSTM-predicted data for the next 10 minutes to calculate the strategy's comprehensive return under spatiotemporal coupling constraints. The Pareto optimal solution set that satisfies multi-objective balance is screened by non-dominated sorting genetic algorithm, and the top 5% high-quality solutions are retained as candidate strategies.

[0009] Optionally, based on the Pareto optimal solution set, the executable policy is screened by a virtual-to-real mapping verification engine to generate a final instruction set for driving the control of campus equipment, while triggering a coordinated response of a security system and an energy consumption system, including: Input the Pareto optimal solution set into the mirror simulation environment of the digital twin, perform full life cycle stress testing, and generate a strategy stability assessment report; Eliminate candidate policies that violate device ratings or safety regulations through a physical constraint compliance verification engine; The remaining strategies are sorted by multi-objective utility, and the strategy with the highest comprehensive utility value is selected to generate the device control instruction coding sequence; The instruction code sequence is synchronously sent to the park's IoT hub, triggering the security system to start a dynamic inspection mode, while driving the energy consumption system to switch its operating strategy to complete a cross-system coordinated response.

[0010] Another embodiment of the present application provides an AI-based smart park multi-source data fusion system, the system comprising: A cleaning module is used to dynamically clean the raw data collected by the heterogeneous sensor network based on environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data using an adaptive noise suppression function to generate a standardized data stream that is aligned in time and space. The noise suppression function dynamically adjusts the filtering threshold based on the sensor type. an extraction module for extracting, based on the standardized data stream, related features of the environment, energy consumption, security, and personnel flow using a cross-modal attention mechanism to generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation; A construction module is used to construct a dynamic digital twin model of the park using a spatiotemporal graph neural network based on the multimodal fusion feature matrix, and to generate a three-dimensional twin that includes equipment status, crowd density, and energy consumption hotspots in real time by embedding a dynamic topology update mechanism; an iterative module for inputting the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generating a resource optimization strategy through a virtual simulation environment, and fusing real-time data with predicted data to generate a Pareto optimal solution set; A generation module is used to screen executable strategies based on the Pareto optimal solution set through a virtual-to-real mapping verification engine, generate a final instruction set to drive the control of campus equipment, and trigger a coordinated response of the security system and the energy consumption system.

[0011] Yet another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, wherein the computer program is configured to execute any of the above methods when run.

[0012] Yet another embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute any of the above methods.

[0013] Compared with the existing technology, the present invention provides an AI-based smart park multi-source data fusion method, which generates a time-space aligned standardized data stream based on environmental parameters, energy consumption waveforms, security signals and personnel trajectory data collected by heterogeneous sensor networks; generates a multimodal fusion feature matrix based on the standardized data stream; generates a three-dimensional twin containing equipment status, crowd density and energy consumption hotspots in real time based on the multimodal fusion feature matrix; inputs the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, and fuses real-time data with predicted data to generate a Pareto optimal solution set; based on the Pareto optimal solution set, generates a final instruction set to drive the park equipment regulation, and simultaneously triggers the coordinated response of the security system and the energy consumption system, thereby realizing the intelligent upgrade of park management through cross-modal feature extraction, dynamic digital twin modeling and reinforcement learning optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A hardware structure block diagram of a computer terminal for an AI-based smart park multi-source data fusion method provided by an embodiment of the present invention; Figure 2 A flowchart of a multi-source data fusion method for a smart park based on AI provided by an embodiment of the present invention; Figure 3 A schematic structural diagram of an AI-based smart park multi-source data fusion system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention.

[0016] The embodiment of the present invention first provides an AI-based smart park multi-source data fusion method, which can be applied to electronic devices such as computer terminals, specifically ordinary computers.

[0017] The following describes it in detail by taking running on a computer terminal as an example. Figure 1 The hardware structure block diagram of a computer terminal for an AI-based smart park multi-source data fusion method provided by an embodiment of the present invention. Figure 1 As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0018] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions that, when executed, enable the processor to execute any one of the AI-based smart park multi-source data fusion methods.

[0019] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0020] The internal memory provides an environment for the operation of computer programs in non-volatile storage media. When the computer program is executed by the processor, the processor can execute any AI-based smart park multi-source data fusion method.

[0021] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0022] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0023] See also Figure 2 , an embodiment of the present invention provides an AI-based smart park multi-source data fusion method, which may include the following steps: S201, dynamically cleaning the raw data, including environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data, collected by the heterogeneous sensor network using an adaptive noise suppression function to generate a spatiotemporally aligned standardized data stream, wherein the noise suppression function dynamically adjusts the filtering threshold based on the sensor type; Specifically, the raw data can be classified according to the sensor type to generate four types of data streams: environmental parameters, energy consumption waveforms, security signals, and personnel trajectories; In smart campuses, heterogeneous sensor networks include a variety of device types, such as temperature sensors, current sensors, cameras, and infrared human body sensors. The key to data classification lies in accurately distinguishing sensors based on their physical properties and data characteristics.

[0024] Environmental parameter data: Sensor type: temperature sensor, humidity sensor, PM2.5 sensor, light sensor.

[0025] Data characteristics: continuous numerical value, sampling frequency is 1 Hz (once per second), and the data range is fixed (such as temperature: -20℃~50℃).

[0026] Classification method: Use the sensor ID prefix (such as "ENV_001" for environment class) combined with the metadata field in the data packet structure (such as data type code 0x01).

[0027] Energy consumption waveform data: Sensor types: smart meters, current transformers, photovoltaic inverters.

[0028] Data characteristics: High-frequency time series signals (such as current waveform sampling rate 1kHz), containing harmonic components and transient fluctuations.

[0029] Classification method: Use data packet length identification (high-frequency data packet length > 500 bytes) and parse the device protocol header (such as the function code 03 of the Modbus protocol represents energy consumption reading).

[0030] Security signal data: Sensor type: access controller, infrared alarm, video analysis camera.

[0031] Data characteristics: event-triggered data (such as access control card swiping is recorded as a discrete event), and video streams are continuous image frames (30fps).

[0032] Classification method: Based on the data trigger mode, event-type data is marked as "security event" and the video stream is identified through the RTSP protocol header.

[0033] Personnel trajectory data: Sensor type: Bluetooth beacon, Wi-Fi probe, UWB positioning tag.

[0034] Data characteristics: spatial coordinate sequence (such as (x, y, z) three-dimensional coordinates), update frequency 5Hz, accuracy ±0.3 meters.

[0035] Classification method: Analyze the positioning data protocol (such as the UUID and Major / Minor values ​​in the iBeacon format) and filter static interference by the movement speed (>0.1m / s) in the data field.

[0036] For example, a temperature sensor (ID = ENV_002) uploads a data packet with the value "0x01 0x0023 25.6". After parsing, it is classified as an environmental parameter, where 0x01 represents the temperature, 0x0023 is the sensor address, and 25.6 is the current temperature value.

[0037] According to the noise characteristics of each type of data stream, a Gaussian distribution model is dynamically fitted based on historical data to generate a dynamic filtering threshold that matches the sensor type; Different sensor data are significantly affected by noise, so targeted modeling is required. Take environmental parameters and energy consumption waveforms as an example: Gaussian modeling of environmental parameters: Historical data collection: Continuously collect data for 7 days and divide it into hourly segments (e.g. 24 segments per day).

[0038] Distribution fitting: Calculate the mean μ and standard deviation σ for each data segment. For example, the temperature sensor's μ = 22.3°C and σ = 1.5°C between 9:00 and 10:00 AM.

[0039] Dynamic threshold calculation: Filter threshold = μ ± 3σ (covering 99.7% of normal data). If the current data exceeds the threshold, it is determined to be abnormal noise.

[0040] Adaptive filtering of energy consumption waveforms: Noise characteristics: power frequency interference (50Hz), impulse noise (such as equipment start-up and shutdown transients).

[0041] Wavelet transform denoising: The Daubechies4 wavelet basis is used to perform a 5-layer decomposition on the 1kHz sampling data, retaining the 2-4 layers of detail coefficients (corresponding to the 50-250Hz frequency band) to suppress power frequency noise.

[0042] Dynamic threshold adjustment: Dynamically adjust the threshold based on the load rate. For example, when the load is >80%, the current fluctuation range is expanded to ±15% (default ±10%).

[0043] Pulse filtering of security signals: Video stream denoising: Non-local mean filtering (NL-Means) is used, and the parameter h (filter strength) is dynamically adjusted according to the light intensity: When the light intensity is >300 lux, h=10 (strong noise reduction); When the light intensity is less than 50 lux, h=3 (preserve details).

[0044] Event-based data verification: Aggregation through time windows (e.g., repeated card swiping events at the same access control station within 10 seconds are considered false triggers).

[0045] Kalman filtering of personnel trajectories: State equation: Assuming that the person moves at a constant speed, predict the position at the next moment.

[0046] Observation noise covariance: Dynamically adjusted based on the positioning signal strength RSSI. When RSSI < -80dBm, the covariance matrix Q increases by 3 times to tolerate high noise.

[0047] For example, historical data from a current sensor shows a normal operating current of 10 A ± 1.5 A (μ = 10, σ = 0.5), and the dynamic threshold is set to 8.5 A to 11.5 A. When a transient current of 15 A is detected (e.g., when a motor starts), the transient protection mechanism is activated, temporarily relaxing the threshold to 14 A.

[0048] The dynamic filtering threshold is input into the adaptive median filter function to perform frame-by-frame noise suppression on the original data to obtain a denoised data stream; Adaptive median filtering achieves efficient denoising by dynamically adjusting the window size and threshold: Window size adaptation: Initial window: 3×3 (suitable for stationary data); Expansion condition: If the median value in the window exceeds the dynamic threshold, expand to 5×5; Termination condition: the window reaches 7×7 or the median meets the threshold.

[0049] Filtering process: Step 1: Read the current data frame (such as the temperature value 25.6℃); Step 2: Take the 3×3 neighborhood data and calculate the median (e.g. the median of 24.8°C, 25.6°C, and 26.1°C is 25.6°C); Step 3: If the difference between the median and the current value is ≤ 3σ (1.5°C), directly output the median; otherwise, expand the window; Step 4: Repeat until the condition is met or the window reaches the upper limit.

[0050] Special treatment: High-frequency energy consumption data: uses sliding window median filtering, with a window length of 50ms (50 sampling points) and real-time updates; Video data: Luminance (Y) and chrominance (UV) components are processed separately in the YUV color space to avoid color distortion.

[0051] Example: A security camera detects random noise (a sudden change in RGB value) in a frame. The adaptive median filter calculates the neighborhood median within a 5×5 window and corrects the abnormal pixel value from (255, 0, 0) to the mean of the surrounding pixels (120, 110, 115).

[0052] The denoised data stream is timestamp-calibrated and spatial coordinates are uniformly mapped to generate a standardized data stream that is time-space aligned. The time alignment accuracy is at the millisecond level, and the spatial error threshold is less than 0.5 meters.

[0053] Time and space alignment requires solving the problems of device clock drift and coordinate system differences: Timestamp calibration: Network Time Protocol (NTP): All sensors are synchronized to a central server with a clock deviation of <1ms; Event-based data alignment: If the access control event is recorded by sensor A as T1 = 12:00:00.123 and by the camera as T2 = 12:00:00.256, the former is used as the reference, and the latter is corrected to T1 + 0.133s through linear interpolation; Data buffering mechanism: Set up a 50ms buffer queue to wait for delayed data to arrive and sort it by timestamp.

[0054] Unification of spatial coordinates: Coordinate system conversion: Convert the local coordinates of each positioning device (such as UWB, Wi-Fi) to the campus global coordinate system (WGS84 or local grid); Error correction: Static calibration: Deploy reference tags at known coordinate points and calculate the coordinate transformation matrix of each device; Dynamic fusion: Multiple positioning data of the same target (such as Bluetooth and UWB) are weighted averaged, and the weights are dynamically assigned based on signal strength.

[0055] Error control: Kalman filtering is used to smooth the trajectory to ensure that the spatial error is less than 0.5 meters.

[0056] Standardized data stream format: Time field: UTC timestamp (accuracy 1ms), such as "2023-10-05T14:23:45.678Z"; Spatial field: three-dimensional coordinates (x, y, z), in meters, such as "(102.34, 58.71, 1.5)"; Data value: normalized value (e.g., temperature is mapped to a range of 0-1, where 0 corresponds to -20°C and 1 corresponds to 50°C).

[0057] Example: After coordinate conversion, a person's trajectory data is converted from UWB local coordinates (15.2, 8.7) to global coordinates (203.5, 415.8). Kalman filtering is used to eliminate jitter, resulting in a smooth trajectory with an error of less than 0.3 meters.

[0058] S202, based on the standardized data stream, using a cross-modal attention mechanism to extract related features of the environment, energy consumption, security, and personnel flow, and generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation; Specifically, the standardized data stream can be input into the modal encoder to generate the environmental parameter modal vector, energy consumption waveform modal vector, security signal modal vector and personnel trajectory modal vector; After preprocessing, the standardized data stream is divided into four independent data channels, each corresponding to a modality (environment, energy consumption, security, and personnel trajectory). The modality encoder uses a heterogeneous neural network architecture to extract features from different types of data: Environmental parameter modal encoding: Environmental data (temperature, humidity, PM2.5) is presented as a time series, and a one-dimensional convolutional network (CNN) is used to extract local features. The convolution kernel width is 5 (corresponding to a 5-second time window), the stride is 1, and the number of channels is 16. For example, given the input temperature sequence [25°C, 25.3°C, 25.1°C, …], after three layers of convolution (number of kernels: 16 → 32 → 64), the output is a 64-dimensional environmental modal vector.

[0059] Energy consumption waveform modal encoding: Energy consumption data (current and voltage waveforms) has high-frequency characteristics. This is achieved by combining wavelet transforms with long-short-term memory (LSTM) networks. First, a three-layer decomposition using the Daubechies-4 wavelet extracts approximate coefficients (low-frequency) and detail coefficients (high-frequency). These are then fed into an LSTM (128 hidden units) to capture temporal dependencies. For example, the current waveform of a device is encoded to generate a 128-dimensional energy consumption modal vector.

[0060] Security signal modality encoding: Security data (camera video streams and infrared sensor trigger signals) is multimodal. A 3D CNN (kernel 3×3×3, stride 2) is used to process video frames, while a gated recurrent unit (GRU) is used to process sensor event sequences. Video features (256 dimensions) are concatenated with sensor features (64 dimensions) to generate a 320-dimensional security modality vector.

[0061] Personnel trajectory modal encoding: Graph Convolutional Networks (GCNs) are used to model spatial relationships in person trajectories (sequences of coordinate points). Trajectory points are mapped to graph nodes, and edge weights are dynamically calculated based on the person's movement speed. The GCN has two layers, 64 hidden units, and outputs a 128-dimensional trajectory modal vector.

[0062] Finally, the dimensions of the four types of modal vectors are unified to 256 dimensions (adjusted by the fully connected layer), forming the environment vector , energy consumption vector , security vector , trajectory vector .

[0063] Through the multi-head cross-modal attention mechanism, the dynamic correlation weights between modal vectors are calculated, where the attention weights corresponding to abnormal data are enhanced according to the exponential function; The Multi-head Cross-modal Attention mechanism consists of 8 independent attention heads, each of which is responsible for capturing the relationship between specific modalities: Query, key, and value generation: Perform linear transformation on each modal vector. For example, the environment vector E is transformed by the weight matrix Transformed into , and similarly generate Key_P, Value_S, etc.

[0064] Attention score calculation: Cross-modal association: Calculate the dot product of the environment Query_E and the energy consumption Key_P, with a scaling factor of √32, to obtain the basic attention score.

[0065] Abnormal enhancement: If an abnormality is detected (e.g., a 30% sudden increase in energy consumption), the attention score for that modality pair is multiplied by an exponential enhancement factor, e^(kΔ), where k = 0.1 is the sensitivity coefficient and Δ is the degree of abnormality (normalized to 0-1). For example, if an energy consumption sudden increase of Δ = 0.8, the score is enhanced to e^(0.1 × 0.8) = 1.083 times.

[0066] Multi-head fusion: The outputs of each head are concatenated and fused through a linear layer to generate a comprehensive attention weight matrix (4 inter-modal weights). For example, the environment-energy attention weight A_E-P = 0.6, and the security-trajectory weight A_S-H = 0.3.

[0067] Dynamic weight allocation example: When a security camera detects an abnormal gathering of people (Δ=0.9), the security-trajectory attention weight is increased from 0.3 to 0.3×e^(0.1×0.9)=0.329, strengthening the correlation analysis between security and personnel flow.

[0068] Perform weighted fusion of modal vectors based on dynamic association weights to generate a cross-modal joint feature vector; Weighted fusion is performed in two steps: Intermodal feature interaction: Calculate a weighted combination of each modality based on the attention weight matrix A. For example, the new representation of the environment vector E is E' = A_E-E×E + A_E-P×P + A_E-S×S + A_E-H×H, where A_E-E is the weight of the environment itself (usually set to 0.2-0.5).

[0069] Feature dimensionality reduction and normalization: The interactive vectors (E', P', S', H') are concatenated into a 1024-dimensional vector, input into the fully connected layer (output 256 dimensions) and layer normalization (LayerNorm) is applied to generate a cross-modal joint feature vector. .

[0070] Abnormal feature retention: Before normalization, L1 regularization (λ=0.01) is applied to abnormality-related features to force the model to retain key abnormal information. For example, if the abnormal weight of the security mode is high, the amplitude of its corresponding feature in F will increase by 15%-20%.

[0071] The cross-modal joint feature vector is processed hierarchically using gated recurrent units to extract long-term and short-term dependency features in the temporal dimension. The Gated Recurrent Unit (GRU) processes time series dependencies in two layers: First layer GRU: Input is the cross-modal joint feature vector F, which is processed sequentially by time step (one step every 5 seconds). The hidden layer has 128 units to capture short-term fluctuations (such as changes in the flow of people within 5 minutes). Output hidden state .

[0072] The second layer GRU: input is , hidden layer 256 units, capturing long-term trends (such as hourly energy consumption cycles). Output hidden state .

[0073] Layered fusion: and Concatenation (384 dimensions), compressed to 256 dimensions by the fully connected layer, generating temporal enhancement features .

[0074] Example: A short-term surge in energy consumption is detected during a certain period ( Capture) and long-term upward trend ( Capture), the activation value of the corresponding dimension in the T vector after fusion increases by 30%.

[0075] The hierarchically processed feature vectors are spliced ​​according to the spatial dimension to generate a multimodal fusion feature matrix, where the matrix dimension corresponds one-to-one to the physical space grid of the park.

[0076] Spatial grid division: Divide the park into grids of 1 meter × 1 meter. The total number of grids N = length × width (e.g. 200 × 300 = 60,000).

[0077] Feature Mapping: The time-enhanced features T are bound to the spatial coordinates (x, y). The initial features of each grid are obtained by interpolating the most recent sensor data. For example, the features of the grid (50, 70) are Taken from the weighted average of 3 nearby environmental sensors.

[0078] Graph convolution aggregation: Construct a spatial adjacency graph (grids are considered adjacent if the distance between them is less than 5 meters) and apply a 2-layer graph convolutional network (GCN) to aggregate neighborhood features: First layer: input 256 dimensions, output 128 dimensions, activation function ReLU; Second layer: input 128 dimensions, output 64 dimensions, activation function Sigmoid.

[0079] Matrix generation: Arrange the 64-dimensional features according to grid coordinates to generate a multimodal fusion feature matrix .

[0080] For example, in a crowded area (such as a cafeteria), the "people density" channel value of the corresponding grid in matrix M reaches 0.9 (normalized value), while the same channel value in the edge area is 0.1.

[0081] S203: Based on the multimodal fusion feature matrix, a spatiotemporal graph neural network is used to construct a dynamic digital twin model of the park. By embedding a dynamic topology update mechanism, a three-dimensional twin including equipment status, crowd density, and energy consumption hotspots is generated in real time. Specifically, the multimodal fusion feature matrix can be mapped into spatiotemporal graph node features according to the physical topology of the park to construct the initial spatiotemporal graph structure; The campus' physical topology is divided into a three-dimensional grid, with each grid cell (1m×1m×1m) corresponding to a node in the space-time graph. Each eigenvector (256 dimensions) of the multimodal fusion feature matrix contains information on environmental parameters (temperature, humidity, PM2.5), energy consumption waveforms (RMS current, harmonic distortion), security signals (number of camera alarms, access control card swipe frequency), and occupant trajectories (movement speed, dwell time). The mapping process utilizes a spatial interpolation algorithm: Grid binding: Bind the sensor deployment coordinates (e.g., the camera is located in grid A1-B2) to the corresponding grid nodes. If a grid has no sensor, the features of adjacent nodes are aggregated using the K-nearest neighbor algorithm (K=5). Feature dimensionality reduction: Principal component analysis (PCA) is used to compress 256-dimensional features to 64 dimensions, retaining 95% of the variance information and reducing computational complexity; Spatiotemporal alignment: Using a 5-minute time window, align the sampling times of different sensors (for example, energy consumption data is sampled every minute, and personnel trajectories are updated every second), and fill in missing time data through linear interpolation.

[0082] For example, the feature vector for a conference room node (grid C3) includes: temperature 25°C, current 12A, three access card swipes in the past five minutes, and an average stay of 15 minutes. The initial spatiotemporal graph contains approximately 100,000 nodes (corresponding to 100,000 cubic meters of campus space). Edge connections are based on physical adjacency (for example, automatically creating edges between adjacent grids) and functional relevance (for example, establishing strong connections between power distribution cabinets and lighting equipment).

[0083] Through the dynamic topology update mechanism, areas with sudden changes in pedestrian density and nodes with abnormal equipment status are detected in real time, generating spatiotemporal graph edge connections that are dynamically added and deleted; The dynamic topology update mechanism is event-driven, and key detection algorithms include: Detection of sudden changes in crowd density: Sliding window statistics: Calculate the change rate of the number of people in each grid using a 1-minute window, with a threshold set at ±30% / minute (e.g., 5 people → 7 people = +40%, triggering a sudden change); Thermal map gradient analysis: Use the Sobel operator to detect areas where the pedestrian density gradient suddenly changes (e.g., a gradient value > 0.8 at a corridor intersection); Device status abnormality detection: Threshold alarm: The current exceeds 15% of the rated value (for example, the current of a 20A device exceeds 23A) or the temperature exceeds the safety threshold (for example, the power distribution cabinet exceeds 60°C). Waveform anomaly identification: The dynamic time warping (DTW) algorithm is used to compare the real-time energy consumption waveform with the historical normal waveform. An anomaly is determined when the difference is greater than 0.7.

[0084] Edge connection dynamic adjustment strategy: New edge: In areas where the flow of people changes suddenly (such as the exhibition hall entrance), a strong connection (weight 0.9) is established with the adjacent security equipment node; Deleting edges: An abnormal device node (such as a faulty air conditioner) is disconnected from the temperature control system; Weight update: Dynamically decay edge weights based on the duration of the anomaly (e.g., 10% per hour).

[0085] For example, when it is detected that the number of people in the E5 area of ​​the exhibition hall increases from 10 to 50 within 5 minutes (a change rate of 400%), the system automatically establishes a new edge between the E5 node and the surrounding emergency exit nodes and security camera nodes, with a weight set to 0.95.

[0086] Embed a gated spatiotemporal convolution module in the spatiotemporal graph neural network to iteratively update node features to capture device state drift characteristics; The Gated Spatio-Temporal Convolution Module (GSTCM) combines spatio-temporal convolution with a gating mechanism: Spatiotemporal convolutional layer: Spatial convolution: 3×3×3 three-dimensional convolution kernel, extracting features of 8 adjacent grids; Temporal convolution: Sliding a window (length 3 time steps) along the time axis to capture device state change trends (such as a slow increase in current); Gating mechanism: Forget gate: The Sigmoid function determines how much historical state is retained (for example, 80% of the current change caused by device aging is retained); Update gate: Tanh function generates new state (such as the latest detected temperature mutation); Feature fusion: Add the gated output with the residual connection to prevent the gradient from vanishing.

[0087] Iterative update process: Input: node feature matrix (64 dimensions), spatiotemporal edge connection weights; Round 1: Spatial convolution captures regional associations (such as the temperature and humidity transmission between a conference room and an adjacent corridor); Round 2: Temporal convolution identifies equipment state drift (e.g., air conditioner compressor efficiency drops by 2% per month); Output: Updated node features (64 dimensions), including the device health index (0-1, 1 being the best).

[0088] For example, after three rounds of iterations, the health index of a distribution cabinet node in the feature vector dropped from 0.85 to 0.78, reflecting the aging of the internal capacitor.

[0089] The updated spatiotemporal graph node features are input into the 3D feature reconstruction layer to generate 3D feature voxels containing equipment health index, thermal distribution of human flow, and energy consumption gradient; The 3D feature reconstruction layer maps discrete node features into a continuous voxel space (resolution 1 cm³): Voxelization: Subdivide the 1m³ grid into 100×100×100 voxels (1cm³), each voxel inherits the 64-dimensional features of the grid to which it belongs; Feature interpolation: trilinear interpolation is used to smooth the feature differences between adjacent voxels (e.g., the energy gradient difference between two adjacent grids is 10% → the inter-voxel gradient is gradually changed); Multi-channel fusion: Channel 1: Device health index (normalized to grayscale values ​​of 0 to 255); Channel 2: Crowd heat (RGB encoding: blue low density → red high density); Channel 3: Energy gradient (HSV color space: hue represents energy type, saturation represents intensity).

[0090] For example, the channel 1 value of a conference room voxel (coordinates X=153, Y=67, Z=2) is 210 (health index 0.82), channel 2 is RGB(180,0,0) (medium density), and channel 3 is HSV(120°, 80%, 90%) (high lighting energy consumption).

[0091] Physically constrained back projection of three-dimensional feature voxels is performed to generate a three-dimensional digital twin that is updated synchronously with the real park, with a resolution of centimeters.

[0092] Physical constraint backprojection is achieved through multiphysics coupling verification: Structural constraints: Use finite element analysis (FEA) to verify the rationality of equipment layout (for example, retaining 50cm of heat dissipation space around the power distribution cabinet); Crowd flow constraints: Based on the Social Force Model, verify whether the channel width meets emergency evacuation requirements (per capita occupied area ≥ 0.3 m2); Energy Consumption Constraints: Use circuit simulation to verify that the cable load does not exceed the rated value (for example, the current of a 6mm² copper cable is ≤36A).

[0093] Synchronous update mechanism: Event triggering: When sensor data is updated or topology changes, local voxels are recalculated (for example, if a device fails, only the surrounding 1m³ voxels are updated); Real-time rendering: Using WebGL technology to refresh 60 frames per second, ensuring that the delay between the twin and the physical world is less than 200ms.

[0094] Centimeter-level resolution enables: Point cloud fusion: LiDAR scan data (accuracy ±1cm) is integrated with voxel features to correct spatial deviations; Dynamic downsampling: Non-key areas (such as green belts) are downsampled to 10cm resolution to save computing power.

[0095] The resulting digital twin can display in real time: a conference room's air conditioning health index of 0.78 (yellow warning), a red warning for pedestrian density in the east corridor, and areas where the energy consumption gradient of the third-floor data center exceeds the standard (HSV red highlight). Operations and maintenance personnel can view and intervene immediately through VR equipment.

[0096] S204, inputting the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generating a resource optimization strategy through a virtual simulation environment, and fusing real-time data with predicted data to generate a Pareto optimal solution set; Specifically, the three-dimensional digital twin can be mapped into the state space of the virtual simulation environment, and three optimization objective functions can be defined: energy consumption cost, security risk, and service efficiency; The 3D digital twin is mapped to the state space of the virtual simulation environment using a centimeter-level spatial grid (e.g., 1m×1m×1m voxels). Each voxel contains the following state parameters: Energy consumption cost: equipment power (kW), lighting energy consumption (kWh / m²), air conditioning system energy efficiency ratio (COP); Security risks: camera blind spot coverage (%), abnormal crowd density (people / m²), firefighting facility response delay (seconds); Service efficiency: elevator waiting time (seconds), conference room reservation conflict rate (%), parking lot vacancy rate (%).

[0097] State space encoding rules: Voxel encoding: Divide the campus into 1000×1000×10 (length×width×height) grids, and assign each grid a unique ID (e.g., GID_123456); Parameter normalization: All parameters are scaled to the range [0, 1]. For example, the device power of 50kW (rated 100kW) is encoded as 0.5. Dynamic update: Synchronize digital twin data every 5 seconds and update state space parameters.

[0098] Optimization objective function definition: Energy cost objective: Minimize the total energy consumption of the park (kWh), with a weight of 0.4; Security risk objective: Minimize the risk index (blind spot coverage × 0.3 + cluster density × 0.5 + response delay × 0.2), weight 0.3; Service efficiency goal: Maximize efficiency score (1 - elevator waiting time / 60 seconds) × 0.6 + parking lot vacancy rate × 0.4), with a weight of 0.3.

[0099] For example, if the parameters of a voxel on a conference floor in the state space at a certain moment are: energy consumption 0.7 (high load), security risk 0.4 (insufficient camera coverage), and service efficiency 0.8 (fast elevator response), then this area becomes the optimization focus.

[0100] Based on the double-delayed deep deterministic policy gradient algorithm, device control strategies are iteratively generated in a virtual environment. Each strategy corresponds to a multi-objective weight combination. The Double-Delayed Deep Deterministic Policy Gradient algorithm (TD3) implements multi-objective optimization through the Actor-Critic framework: Actor network (policy network): Input state space voxel features (dimension 1000) and output device control actions (such as air conditioning temperature setting ±2°C, lighting brightness adjustment ±10%); Critic network (value network): Dual-Q network structure (Q1, Q2), input state and action, and output three-objective weighted value (energy cost × 0.4 + security risk × 0.3 + service efficiency × 0.3).

[0101] Algorithm flow: Exploration phase: Add Gaussian noise (mean 0, standard deviation 0.1) to the action space, for example, adjust the air conditioner target temperature from 24°C to 23.8°C or 24.2°C; Experience replay: stores 100,000 historical state-action-reward samples, with a batch size of 256; Strategy Update: Critic update: Update the Q network parameters by minimizing the temporal difference error (TD Error) with a learning rate of 0.0001; Actor update: After every two critic updates, the expected value is improved through policy gradient, with a learning rate of 0.00001; Target network soft update: parameter update coefficient τ = 0.005.

[0102] Dynamic adjustment of multi-objective weights: Adaptive weighting mechanism: adjusts target weights based on real-time data. For example, if the security system detects that the number of people gathering exceeds a threshold (such as 5 people / m²), it will automatically increase the security risk weight from 0.3 to 0.5; Strategy branch: Generate three typical strategies: Energy conservation priority: energy consumption weight 0.6, service efficiency weight 0.2; Security priority: security weight is 0.6, and other weights are 0.2 each; Balanced mode: The weights of the three objectives are all 0.33.

[0103] Real-time and predictive data fusion: LSTM prediction module: inputs historical 10 minutes of data and outputs energy consumption trends (with an error of ±3%) and passenger flow distribution (with an error of ±5%) for the next 10 minutes; Calculation of comprehensive return: real-time data accounts for 70%, and predicted data accounts for 30%. For example, if a strategy has a return of 0.8 in the real-time scenario and 0.7 in the predicted scenario, the comprehensive return = 0.8×0.7+0.7×0.3=0.77.

[0104] Fusion of real-time sensor data with LSTM-predicted data for the next 10 minutes to calculate the strategy's comprehensive return under spatiotemporal coupling constraints. Spatiotemporal alignment and fusion of real-time data and predicted data Real-time sensor data: Data types: including ambient temperature and humidity (accuracy ±0.5°C), equipment energy consumption (sampling rate 1Hz), personnel positioning coordinates (UWB technology, error <10cm), security camera status (online / offline), etc.

[0105] Preprocessing: Kalman filtering is used to eliminate instantaneous noise, for example, to smooth the ±0.3°C random fluctuation of the temperature sensor.

[0106] LSTM prediction data: Model structure: A two-layer LSTM network (128 units per layer) is used, the input is historical data of the past 30 minutes (time step 1800 seconds), and the output is a prediction sequence of the next 10 minutes (600 seconds).

[0107] Input features: equipment power, crowd density, air conditioning set temperature, light intensity, etc. Prediction output: Energy consumption trend every 30 seconds for the next 10 minutes, crowd distribution heat map, and equipment failure probability.

[0108] Training process: Using the Adam optimizer (learning rate 0.001) and the mean absolute error (MAE) loss function, the model was trained on 1 million historical data points until convergence (validation set error < 3%).

[0109] Space-time alignment: Time axis interpolation: real-time data (1-second granularity) and forecast data (30-second granularity) are unified into 10-second intervals, and cubic spline interpolation is used to fill the time gaps.

[0110] Spatial grid mapping: Maps the predicted occupant trajectory data to the campus's three-dimensional grid (1m×1m resolution). For example, it predicts that the occupant density in a certain area will increase from 2 people / m² to 5 people / m² in 5 minutes.

[0111] Fusion rules: Real-time data weight: Real-time data at the current moment (t=0) accounts for 70%, and predicted data accounts for 30%; Prediction decay coefficient: As time goes by, the prediction weight decays exponentially (attenuation factor λ=0.1). For example, at t=5 minutes, real-time data accounts for 60% and prediction accounts for 40%.

[0112] Spatiotemporal coupling constraint modeling Space constraints: Equipment linkage restrictions: For example, air conditioning and fresh air systems need to start and stop synchronously. If the policy requires shutting down the air conditioning in a certain area but not shutting down the fresh air, it is considered a space conflict. Security coverage constraints: Reducing lighting or shutting down surveillance equipment is prohibited in camera blind spots (such as fire escape routes).

[0113] Time constraints: Equipment response delay: Elevator dispatch instructions must be responded to within 2 seconds, and a timeout will trigger a penalty; Energy consumption gradient limit: The air conditioning power adjustment rate shall not exceed ±5% / minute to avoid equipment overload.

[0114] Dynamic Constraint Modeling: Tensor representation: Encode the constraints as a three-dimensional tensor (time × space × constraint type), for example: Constraint type 1: Nighttime (20:00-6:00) lighting brightness in public areas ≥ 70%; Constraint type 2: The air conditioning temperature setting must not be lower than 24°C when the conference room is in use.

[0115] Conflict detection: The matching degree between the policy action and the constraint tensor is calculated through the matrix inner product. If the matching degree is less than 0.8, it is considered a violation.

[0116] Calculation of comprehensive income value Revenue composition: Energy consumption benefit (weight 0.4): Calculated based on the difference between real-time and predicted energy consumption. For example, if strategy A adjusts the air conditioning setting in a certain area from 24°C to 26°C, the real-time energy consumption is reduced by 10kW, and the cumulative energy saving in the next 10 minutes is predicted to be 50kWh, then the energy consumption benefit = 50 × 0.4 = 20; Security benefit (weight 0.3): calculated based on the risk reduction percentage. For example, if Strategy B adds two temporary cameras, the blind spot coverage rate drops from 20% to 5%. The security benefit = (20-5) / 20 × 100 × 0.3 = 22.5. Efficiency gain (weight 0.3): Calculated by the service efficiency improvement rate. For example, if Strategy C optimizes elevator scheduling, the average waiting time is reduced from 30 seconds to 18 seconds. The efficiency gain = (30-18) / 30×100×0.3=12.

[0117] Penalty items: Spatial conflict penalty: 5% of the total revenue will be deducted for each device linkage conflict detected; Time delay penalty: If the policy execution delay exceeds the threshold (e.g. elevator response > 2 seconds), cumulative points will be deducted based on the number of seconds of delay × 0.1%; Forecast deviation penalty: When the deviation between the predicted data and the actual data exceeds 10% (for example, the crowd density is predicted to be 5 people / m², but the actual density is 7 people / m²), 20% of the profit of the deviation will be deducted.

[0118] Dynamic adjustment example: The total return of a strategy at t=0 = 20 + 22.5 + 12 = 54.5; If the predicted deviation of the passenger flow is detected to be 15% at t = 5 minutes, the corrected profit = 54.5 × (1-0.15 × 0.2) = 54.5 × 0.97 = 52.87; The final payoff values ​​are used for Pareto ranking of NSGA-III.

[0119] The Pareto optimal solution set that satisfies multi-objective balance is screened by non-dominated sorting genetic algorithm, and the top 5% high-quality solutions are retained as candidate strategies.

[0120] The NSGA-III algorithm selects the Pareto optimal solution through multi-objective optimization. The specific steps are as follows: Population initialization: The population size is 500, and the initial strategies are randomly generated (each strategy contains a set of device control parameters); For example, policy A includes "air conditioning setting 26°C, lighting brightness 80%, and elevator response interval 30 seconds."

[0121] Non-dominated sorting: Calculate the three target values ​​(energy consumption, security, and efficiency) for each strategy; Hierarchical sorting: the first layer contains solutions that are not dominated by any other strategy, the second layer contains solutions that are only dominated by the first layer, and so on; For example, strategy A (energy consumption 0.6, security 0.4, efficiency 0.7) dominates strategy B (0.7, 0.5, 0.6).

[0122] Reference point generation: 100 reference points are evenly distributed in the target space to ensure the diversity of the solution set; For example, reference point 1 corresponds to (energy consumption 0.3, security 0.3, efficiency 0.4), and reference point 2 corresponds to (0.4, 0.3, 0.3).

[0123] Crowding calculation and selection: Calculate the normalized distance from each solution to the nearest reference point; Solutions with high congestion are retained to avoid local aggregation. For example, if solution C is 0.1 away from the reference point and solution D is 0.2 away, solution D is retained first. Genetic Operations: Crossover: simulated binary crossover (SBX), crossover probability 0.9, distribution index 20; Mutation: polynomial mutation, mutation probability 0.1, distribution index 30; For example, parent strategy A (air conditioning 26℃) and strategy B (25℃) cross to generate child strategy C (25.5℃).

[0124] Elite Retention: The top 25 non-dominated solutions (top 5%) are retained in each generation, and a total of 100 generations are iterated; The final Pareto front contains 50 solutions, covering the trade-off ranges of energy consumption 0.4~0.8, security 0.2~0.6, and efficiency 0.5~0.9.

[0125] Dynamic screening mechanism: Scenario adaptation: Automatically select low-energy solutions based on real-time weather (such as high temperature warnings); Manual intervention: Operations personnel can manually adjust target weights and reorder solution sets.

[0126] For example, during the peak electricity consumption period in summer, the system automatically selects a solution with energy consumption below 0.5 and efficiency above 0.7, and forcibly turns off non-essential lighting equipment.

[0127] S205 , based on the Pareto optimal solution set, screen the executable strategies through the virtual-to-real mapping verification engine, generate the final instruction set for driving the control of the park equipment, and trigger the coordinated response of the security system and the energy consumption system.

[0128] Specifically, the Pareto optimal solution set can be input into the mirror simulation environment of the digital twin, a full life cycle stress test can be performed, and a strategy stability assessment report can be generated; The full lifecycle stress test simulates the long-term operation of campus equipment in extreme scenarios using a digital twin's mirrored simulation environment. This mirrored simulation environment is based on a real-world 3D model of the campus and includes parameterized digital images of all physical equipment (e.g., air conditioners rated at 5kW power and lighting systems with a dimming range of 10% to 100%). The stress test is divided into three phases: Load shock test: Monte Carlo simulation is used to simulate fluctuations in passenger density over the next 24 hours (peak passenger density reaches 3 people / m2, off-peak passenger density is 0.2 people / m2), to test the strategy's ability to respond to sudden increases in air conditioning load (e.g., from 30% to 90%). Equipment aging test: A Weibull distribution model (shape parameter β = 2.5, scale parameter η = 10,000 hours) was introduced to simulate equipment performance degradation and evaluate the robustness of the strategy when the equipment efficiency dropped by 10% to 30%; Abnormal event injection: Randomly inject security intrusion events (such as an illegal break-in simulation once per second) and energy consumption anomalies (such as instantaneous power consumption in a certain area exceeding the standard by 200%), and record the fault recovery time of the strategy (target: <30 seconds).

[0129] The stability assessment report includes the following core indicators: Load balancing rate: standard deviation of device utilization (target <15%); Fault recovery rate: Success rate of abnormal event recovery (target > 98%); Energy consumption fluctuation coefficient: standard deviation of daily average energy consumption (target <5%).

[0130] For example, if a strategy has a load balancing rate of 12%, a failure recovery rate of 99.2%, and an energy consumption fluctuation coefficient of 4.3% in the test, it is marked as a "high stability strategy."

[0131] Eliminate candidate policies that violate device ratings or safety regulations through a physical constraint compliance verification engine; The physical constraint compliance verification engine uses a joint working mode of the rule engine (Drools) and the knowledge graph (Neo4j): Rule base construction: Equipment rating parameter rules: such as "the operating power of the air conditioner shall not exceed 120% of the rated value"; Safety regulations and rules: such as "Emergency exit lighting brightness must be maintained at ≥300 lumens"; Industry standard rules: such as "Data center PUE (power usage effectiveness) ≤ 1.5".

[0132] Knowledge graph association: This system constructs a graph structure based on device parameters (such as the rated current of 15A for an air conditioner with ID AC-203), spatial topology (such as the maximum carrying current of 200A for the B1 floor distribution box), and regulatory provisions (such as the GB 50034-2013 lighting standard). It then implements multi-hop association reasoning using the Cypher query language.

[0133] Verification process: Static verification: Directly matches policy parameters with rule thresholds. For example, if a policy requires an air conditioner's power to be increased to 6.3kW (126% of its rated power of 5kW), a violation alarm is triggered. Dynamic verification: This simulates the cascading effects of policy execution. For example, dimming the lighting in a certain area to 5% may affect the recognition rate of security cameras (a minimum brightness of 15% must be maintained). Conflicts are detected through graph reasoning.

[0134] The candidate policy set after verification must meet 100% compliance. For example, a policy may be eliminated because it requires turning off the lighting of the fire escape.

[0135] The remaining strategies are sorted by multi-objective utility, and the strategy with the highest comprehensive utility value is selected to generate the device control instruction coding sequence; The improved TOPSIS (top-to-ideal solution ranking method) algorithm is used for multi-objective utility ranking: Target weight allocation: Based on the Analytic Hierarchy Process (AHP), the priorities of energy consumption cost (weight 0.4), security risk (0.3), and service efficiency (0.3) are determined; Normalization: Normalize the energy consumption (kWh), security response time (seconds), and service waiting time (seconds) of each strategy to the range [0, 1]. Ideal solution calculation: The positive ideal solution is [lowest energy consumption = 0, fastest security response = 0, shortest service wait = 0], and the negative ideal solution is [highest energy consumption = 1, slowest response = 1, longest wait = 1]; Utility value calculation:

[0136] For example, if strategy A is 0.2 away from the positive ideal solution and 0.8 away from the negative ideal solution, then the utility value = 0.8 / (0.2+0.8) = 0.8; Sorting and filtering: Sort by utility value in descending order and select the TOP1 strategy.

[0137] Instruction encoding sequence generation: Command structure: uses TLV (Type-Length-Value) encoding format, for example, the type field is 1 byte (0x01 indicates air conditioning control), the length field is 2 bytes (data length), and the value field contains the target parameter (such as the temperature setting value 24°C); Fault-tolerant design: Each instruction comes with a CRC-16 checksum (polynomial 0x8005), with an error detection rate of >99.99%.

[0138] For example, the instruction sequence generated by the optimal strategy is: [0x01][0x0004][24°C][CRC] → [0x02][0x0003][80%][CRC] (air conditioning at 24°C + lighting at 80% brightness).

[0139] The instruction code sequence is synchronously sent to the park's IoT hub, triggering the security system to start a dynamic inspection mode, while driving the energy consumption system to switch its operating strategy to complete a cross-system coordinated response.

[0140] The IoT hub implements command synchronization based on 5G TSN (Time-Sensitive Network): Instruction distribution: Unicast channel: Key commands (such as fire equipment control) are directly connected to the device through the URLLC (Ultra Reliable Low Latency) channel with a latency of <10ms; Multicast channel: Bulk commands (such as lighting adjustment) are broadcast through the eMBB (enhanced mobile broadband) channel with a bandwidth of ≥100Mbps.

[0141] Security system linkage: Dynamic inspection mode: Adjust the camera patrol path according to policy requirements (for example, increase the inspection frequency from 30 minutes to 10 minutes), and tighten the comparison threshold of the face recognition algorithm (ArcFace) from 0.7 to 0.8; Emergency response: If an area of ​​abnormal energy consumption is detected, a patrol robot (AGV) will be automatically dispatched to investigate, with a maximum speed of 3m / s.

[0142] Energy consumption system switching: Strategy loading: injecting new operating strategies into the BMS (building management system), such as switching the air conditioning group control algorithm from PID to model predictive control (MPC); Real-time monitoring: collects device status (such as real-time power and temperature) through the Modbus TCP protocol, with a refresh cycle of 500ms.

[0143] Exception handling mechanism: Command retry: If the device does not respond (timeout 2 seconds), trigger exponential backoff retry (maximum 3 retries); Transaction rollback: When a cross-system command partially fails (for example, the air conditioner has adjusted the temperature but the lighting does not respond), the transaction log is rolled back to the last stable state.

[0144] Ultimately, the system achieved a command issuance success rate of ≥99.9% and a cross-system collaborative response delay of ≤50ms, meeting the real-time control needs of the smart park.

[0145] It can be seen that according to the environmental parameters, energy consumption waveforms, security signals and personnel trajectory data collected by the heterogeneous sensor network, a standardized data stream aligned in time and space is generated; based on the standardized data stream, a multimodal fusion feature matrix is ​​generated; according to the multimodal fusion feature matrix, a three-dimensional twin containing equipment status, crowd density and energy consumption hotspots is generated in real time; the three-dimensional twin is input into the reinforcement learning algorithm with multi-objective constraints, and the real-time data and predicted data are integrated to generate a Pareto optimal solution set; based on the Pareto optimal solution set, the final instruction set for driving the equipment control in the park is generated, and the coordinated response of the security system and the energy consumption system is triggered at the same time, so that the intelligent upgrade of park management can be realized through cross-modal feature extraction, dynamic digital twin modeling and reinforcement learning optimization.

[0146] Another embodiment of the present invention provides an AI-based smart park multi-source data fusion system, see Figure 3 , the system may include: Cleaning module 301 is used to dynamically clean the raw data collected by the heterogeneous sensor network based on environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data using an adaptive noise suppression function to generate a standardized data stream aligned in time and space. The noise suppression function dynamically adjusts the filtering threshold based on the sensor type. An extraction module 302 is configured to extract, based on the standardized data stream, related features of the environment, energy consumption, security, and personnel flow using a cross-modal attention mechanism to generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism enhances the feature expression of abnormal data through dynamic weight allocation; Construction module 303 is used to construct a dynamic digital twin model of the park using a spatiotemporal graph neural network based on the multimodal fusion feature matrix, and to generate a three-dimensional twin that includes equipment status, crowd density, and energy consumption hotspots in real time by embedding a dynamic topology update mechanism; Iterative module 304 is used to input the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generate resource optimization strategies through a virtual simulation environment, and fuse real-time data with predicted data to generate a Pareto optimal solution set; The generation module 305 is used to screen executable strategies based on the Pareto optimal solution set through the virtual-real mapping verification engine, generate a final instruction set to drive the control of campus equipment, and trigger the coordinated response of the security system and the energy consumption system.

[0147] It can be seen that according to the environmental parameters, energy consumption waveforms, security signals and personnel trajectory data collected by the heterogeneous sensor network, a standardized data stream aligned in time and space is generated; based on the standardized data stream, a multimodal fusion feature matrix is ​​generated; according to the multimodal fusion feature matrix, a three-dimensional twin containing equipment status, crowd density and energy consumption hotspots is generated in real time; the three-dimensional twin is input into the reinforcement learning algorithm with multi-objective constraints, and the real-time data and predicted data are integrated to generate a Pareto optimal solution set; based on the Pareto optimal solution set, the final instruction set for driving the equipment control in the park is generated, and the coordinated response of the security system and the energy consumption system is triggered at the same time, so that the intelligent upgrade of park management can be realized through cross-modal feature extraction, dynamic digital twin modeling and reinforcement learning optimization.

[0148] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program is configured to execute the steps of any one of the above method embodiments when running.

[0149] Specifically, in this embodiment, the above-mentioned storage medium may be configured to store a computer program for performing the following steps: S201, dynamically cleaning the raw data, including environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data, collected by the heterogeneous sensor network using an adaptive noise suppression function to generate a spatiotemporally aligned standardized data stream, wherein the noise suppression function dynamically adjusts the filtering threshold based on the sensor type; S202, based on the standardized data stream, using a cross-modal attention mechanism to extract related features of the environment, energy consumption, security, and personnel flow, and generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation; S203: Based on the multimodal fusion feature matrix, a spatiotemporal graph neural network is used to construct a dynamic digital twin model of the park. By embedding a dynamic topology update mechanism, a three-dimensional twin including equipment status, crowd density, and energy consumption hotspots is generated in real time. S204, inputting the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generating a resource optimization strategy through a virtual simulation environment, and fusing real-time data with predicted data to generate a Pareto optimal solution set; S205 , based on the Pareto optimal solution set, screen the executable strategies through the virtual-to-real mapping verification engine, generate the final instruction set for driving the control of the park equipment, and trigger the coordinated response of the security system and the energy consumption system.

[0150] It can be seen that according to the environmental parameters, energy consumption waveforms, security signals and personnel trajectory data collected by the heterogeneous sensor network, a standardized data stream aligned in time and space is generated; based on the standardized data stream, a multimodal fusion feature matrix is ​​generated; according to the multimodal fusion feature matrix, a three-dimensional twin containing equipment status, crowd density and energy consumption hotspots is generated in real time; the three-dimensional twin is input into the reinforcement learning algorithm with multi-objective constraints, and the real-time data and predicted data are integrated to generate a Pareto optimal solution set; based on the Pareto optimal solution set, the final instruction set for driving the equipment control in the park is generated, and the coordinated response of the security system and the energy consumption system is triggered at the same time, so that the intelligent upgrade of park management can be realized through cross-modal feature extraction, dynamic digital twin modeling and reinforcement learning optimization.

[0151] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0152] Specifically, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0153] Specifically, in this embodiment, the processor may be configured to execute the following steps through a computer program: S201, dynamically cleaning the raw data, including environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data, collected by the heterogeneous sensor network using an adaptive noise suppression function to generate a spatiotemporally aligned standardized data stream, wherein the noise suppression function dynamically adjusts the filtering threshold based on the sensor type; S202, based on the standardized data stream, using a cross-modal attention mechanism to extract related features of the environment, energy consumption, security, and personnel flow, and generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation; S203: Based on the multimodal fusion feature matrix, a spatiotemporal graph neural network is used to construct a dynamic digital twin model of the park. By embedding a dynamic topology update mechanism, a three-dimensional twin including equipment status, crowd density, and energy consumption hotspots is generated in real time. S204, inputting the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generating a resource optimization strategy through a virtual simulation environment, and fusing real-time data with predicted data to generate a Pareto optimal solution set; S205 , based on the Pareto optimal solution set, screen the executable strategies through the virtual-to-real mapping verification engine, generate the final instruction set for driving the control of the park equipment, and trigger the coordinated response of the security system and the energy consumption system.

[0154] It can be seen that according to the environmental parameters, energy consumption waveforms, security signals and personnel trajectory data collected by the heterogeneous sensor network, a standardized data stream aligned in time and space is generated; based on the standardized data stream, a multimodal fusion feature matrix is ​​generated; according to the multimodal fusion feature matrix, a three-dimensional twin containing equipment status, crowd density and energy consumption hotspots is generated in real time; the three-dimensional twin is input into the reinforcement learning algorithm with multi-objective constraints, and the real-time data and predicted data are integrated to generate a Pareto optimal solution set; based on the Pareto optimal solution set, the final instruction set for driving the equipment control in the park is generated, and the coordinated response of the security system and the energy consumption system is triggered at the same time, so that the intelligent upgrade of park management can be realized through cross-modal feature extraction, dynamic digital twin modeling and reinforcement learning optimization.

[0155] The above describes in detail the structure, features and effects of the present invention based on the embodiments shown in the drawings. The above is only a preferred embodiment of the present invention, but the scope of implementation of the present invention is not limited to what is shown in the drawings. Any changes made in accordance with the concept of the present invention, or modifications to equivalent embodiments with equivalent changes, which do not exceed the spirit covered by the description and drawings, should be within the scope of protection of the present invention.

Claims

1. A multi-source data fusion method for smart parks based on AI, characterized by: The method comprises: Based on the environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data collected by the heterogeneous sensor network, the raw data is dynamically cleaned using an adaptive noise suppression function to generate a standardized data stream aligned in time and space. The noise suppression function dynamically adjusts the filtering threshold based on the sensor type. Based on the standardized data stream, a cross-modal attention mechanism is used to extract related features of the environment, energy consumption, security, and personnel flow, and generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation; Based on the multimodal fusion feature matrix, a spatiotemporal graph neural network is used to construct a dynamic digital twin model of the park. By embedding a dynamic topology update mechanism, a three-dimensional twin containing equipment status, crowd density, and energy consumption hotspots is generated in real time. Inputting the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generating resource optimization strategies through a virtual simulation environment, and fusing real-time data with predicted data to generate a Pareto optimal solution set; Based on the Pareto optimal solution set, the executable strategy is screened through the virtual-to-real mapping verification engine to generate the final instruction set for driving the control of the park equipment, while triggering the coordinated response of the security system and the energy consumption system.

2. The method according to claim 1, characterized in that The environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data collected by the heterogeneous sensor network are dynamically cleaned using an adaptive noise suppression function to generate a standardized data stream aligned in time and space, wherein the noise suppression function dynamically adjusts the filtering threshold according to the sensor type, including: Classify the raw data according to the sensor type and generate four types of data streams: environmental parameters, energy consumption waveforms, security signals, and personnel trajectories; According to the noise characteristics of each type of data stream, a Gaussian distribution model is dynamically fitted based on historical data to generate a dynamic filtering threshold that matches the sensor type; The dynamic filtering threshold is input into the adaptive median filter function to perform frame-by-frame noise suppression on the original data to obtain a denoised data stream; The denoised data stream is timestamp-calibrated and spatial coordinates are uniformly mapped to generate a standardized data stream that is time-space aligned. The time alignment accuracy is at the millisecond level, and the spatial error threshold is less than 0.5 meters.

3. The method according to claim 2, characterized in that Based on the standardized data stream, a cross-modal attention mechanism is used to extract the correlation features of the environment, energy consumption, security and personnel flow, and generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation, including: Input the standardized data stream into the modal encoder to generate the environmental parameter modal vector, energy consumption waveform modal vector, security signal modal vector and personnel trajectory modal vector; Through the multi-head cross-modal attention mechanism, the dynamic correlation weights between modal vectors are calculated, where the attention weights corresponding to abnormal data are enhanced according to the exponential function; Perform weighted fusion of modal vectors based on dynamic association weights to generate a cross-modal joint feature vector; The cross-modal joint feature vector is processed hierarchically using gated recurrent units to extract long-term and short-term dependency features in the temporal dimension. The hierarchically processed feature vectors are spliced ​​according to the spatial dimension to generate a multimodal fusion feature matrix, where the matrix dimension corresponds one-to-one to the physical space grid of the park.

4. The method according to claim 3, characterized in that The multimodal fusion feature matrix is ​​used to construct a dynamic digital twin model of the park using a spatiotemporal graph neural network. By embedding a dynamic topology update mechanism, a three-dimensional twin containing equipment status, crowd density, and energy consumption hotspots is generated in real time, including: According to the physical topology of the park, the multimodal fusion feature matrix is ​​mapped into the node features of the spatiotemporal graph to construct the initial spatiotemporal graph structure; Through the dynamic topology update mechanism, areas with sudden changes in pedestrian density and nodes with abnormal equipment status are detected in real time, generating spatiotemporal graph edge connections that are dynamically added and deleted; Embed a gated spatiotemporal convolution module in the spatiotemporal graph neural network to iteratively update node features to capture device state drift characteristics; The updated spatiotemporal graph node features are input into the 3D feature reconstruction layer to generate 3D feature voxels containing equipment health index, thermal distribution of human flow, and energy consumption gradient; Physically constrained back projection of three-dimensional feature voxels is performed to generate a three-dimensional digital twin that is updated synchronously with the real park, with a resolution of centimeters.

5. The method according to claim 4, characterized in that The three-dimensional twin is input into a reinforcement learning algorithm with multi-objective constraints, resource optimization strategies are iteratively generated through a virtual simulation environment, and real-time data and predicted data are integrated to generate a Pareto optimal solution set, including: Map the 3D digital twin into the state space of the virtual simulation environment and define three optimization objective functions: energy cost, security risk, and service efficiency; Based on the double-delayed deep deterministic policy gradient algorithm, device control strategies are iteratively generated in a virtual environment. Each strategy corresponds to a multi-objective weight combination. Fusion of real-time sensor data with LSTM-predicted data for the next 10 minutes to calculate the strategy's comprehensive return under spatiotemporal coupling constraints. The Pareto optimal solution set that satisfies multi-objective balance is screened by non-dominated sorting genetic algorithm, and the top 5% high-quality solutions are retained as candidate strategies.

6. The method according to claim 5, characterized in that Based on the Pareto optimal solution set, the executable strategy is screened through the virtual-real mapping verification engine to generate the final instruction set that drives the control of the park equipment, and at the same time triggers the coordinated response of the security system and the energy consumption system, including: Input the Pareto optimal solution set into the mirror simulation environment of the digital twin, perform full life cycle stress testing, and generate a strategy stability assessment report; Eliminate candidate policies that violate device ratings or safety regulations through a physical constraint compliance verification engine; The remaining strategies are sorted by multi-objective utility, and the strategy with the highest comprehensive utility value is selected to generate the device control instruction coding sequence; The instruction code sequence is synchronously sent to the park's IoT hub, triggering the security system to start a dynamic inspection mode, while driving the energy consumption system to switch its operating strategy to complete a cross-system coordinated response.

7. An AI-based smart park multi-source data fusion system, characterized by: The system comprises: A cleaning module is used to dynamically clean the raw data collected by the heterogeneous sensor network based on environmental parameters, energy consumption waveforms, security signals, and personnel trajectory data using an adaptive noise suppression function to generate a standardized data stream that is aligned in time and space. The noise suppression function dynamically adjusts the filtering threshold based on the sensor type. an extraction module for extracting, based on the standardized data stream, related features of the environment, energy consumption, security, and personnel flow using a cross-modal attention mechanism to generate a multimodal fusion feature matrix, wherein the cross-modal attention mechanism strengthens the feature expression of abnormal data through dynamic weight allocation; A construction module is used to construct a dynamic digital twin model of the park using a spatiotemporal graph neural network based on the multimodal fusion feature matrix, and to generate a three-dimensional twin that includes equipment status, crowd density, and energy consumption hotspots in real time by embedding a dynamic topology update mechanism; an iterative module for inputting the three-dimensional twin into a multi-objective constraint reinforcement learning algorithm, iteratively generating a resource optimization strategy through a virtual simulation environment, and fusing real-time data with predicted data to generate a Pareto optimal solution set; A generation module is used to screen executable strategies based on the Pareto optimal solution set through a virtual-to-real mapping verification engine, generate a final instruction set to drive the control of campus equipment, and trigger a coordinated response of the security system and the energy consumption system.

8. The system according to claim 7, characterized in that The cleaning module is specifically used for: Classify the raw data according to the sensor type and generate four types of data streams: environmental parameters, energy consumption waveforms, security signals, and personnel trajectories; According to the noise characteristics of each type of data stream, a Gaussian distribution model is dynamically fitted based on historical data to generate a dynamic filtering threshold that matches the sensor type; The dynamic filtering threshold is input into the adaptive median filter function to perform frame-by-frame noise suppression on the original data to obtain a denoised data stream; The denoised data stream is timestamp-calibrated and spatial coordinates are uniformly mapped to generate a standardized data stream that is time-space aligned. The time alignment accuracy is at the millisecond level, and the spatial error threshold is less than 0.5 meters.

9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 6 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 6.

Images