Smart building management method and device based on multi-source data fusion

Abstract

The invention provides an intelligent building management method and device based on multi-source data fusion, and relates to the technical field of the Internet of Things, and the method comprises the steps: obtaining a multi-source data set of a target building through a sensor network, the multi-source data set comprises a parking area image set outside the target building, a state data set inside the target building and a canteen area image set inside the target building; inputting the parking area image set into a vehicle illegal parking detection model for vehicle illegal parking detection to obtain a vehicle illegal parking detection result; inputting the state data set into a building energy consumption prediction model for building energy consumption prediction to obtain a building energy consumption prediction result; inputting the canteen area image set into a face recognition model for face recognition and dining authorization judgment to obtain a dining authorization result; and according to the vehicle illegal parking detection result, carrying out illegal parking alarm on the user of the illegal parking vehicle, carrying out power adjustment on the equipment of the target building according to the building energy consumption prediction result, and carrying out unauthorized alarm according to the dining authorization result.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a smart building management method and device based on multi-source data fusion. Background Technology

[0002] With the acceleration of urbanization and the rapid development of IoT technology, smart buildings have become an important trend in modern building management. By integrating various sensors, communication networks, and intelligent control systems, smart buildings can achieve real-time monitoring, data analysis, and automated management of the building environment, thereby improving operational efficiency, energy conservation, and user experience. However, existing smart building management methods suffer from insufficient multi-source data collection and fusion capabilities, low accuracy in detecting illegal parking, and ineffective regulation of building energy consumption, making it difficult to meet the actual needs of efficient building operation and safety management. These issues are mainly reflected in the following aspects.

[0003] Existing illegal parking detection methods do not optimize the network structure for illegal parking scenarios, resulting in excessive computational load during illegal parking detection. They also lack efficient multi-scale feature fusion mechanisms, leading to low model recognition accuracy and an inability to accurately identify relevant data of illegally parked vehicles.

[0004] Existing energy consumption prediction models rely on single energy consumption data for modeling, failing to fully integrate multiple influencing factors such as indoor temperature and humidity, light intensity, and human activity status, and failing to effectively capture the temporal correlation between data, resulting in large errors in energy consumption prediction results and ultimately leading to inaccurate adjustment of equipment power. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a smart building management method based on multi-source data fusion, comprising:

[0006] The target building is acquired through a sensor network. The multi-source data set includes images of the parking area outside the target building, status data inside the target building, and images of the canteen area inside the target building. The status data set includes indoor temperature data, indoor humidity data, indoor light intensity data, smoke concentration data, personnel activity status data, personnel entry and exit record data, and equipment energy consumption data.

[0007] A vehicle illegal parking detection model is constructed based on the YOLOv8 network. The set of parking area images is input into the vehicle illegal parking detection model to detect vehicle illegal parking and obtain the vehicle illegal parking detection results.

[0008] Construct a building energy consumption prediction model, input the state data set into the building energy consumption prediction model to predict building energy consumption, and obtain the building energy consumption prediction results;

[0009] A face recognition model is constructed, and the set of images of the canteen area is input into the face recognition model to perform face recognition and dining authorization judgment, and obtain the dining authorization result;

[0010] Based on the results of vehicle illegal parking detection, illegal parking alerts are issued to users of illegally parked vehicles; based on the building energy consumption prediction results, the power of equipment in the target building is adjusted; and based on the dining authorization results, unauthorized alerts are issued.

[0011] Optionally, the sensor network includes temperature and humidity sensors, light sensors, smoke sensors, surveillance cameras, human infrared sensors, access control sensors, and energy consumption sensors.

[0012] Optionally, the vehicle illegal parking detection model built based on the YOLOv8 network includes:

[0013] The YOLOv8 network consists of an initial backbone network, an initial Neck network, and a detection head network.

[0014] An improved backbone network is constructed by adding Ghost Shuffle convolutional modules and CBAM modules after each C2F module in the initial backbone network.

[0015] Replace the initial Neck network with a BiFPN bidirectional feature pyramid network;

[0016] The vehicle illegal parking detection model consists of an improved backbone network, a BiFPN bidirectional feature pyramid network, and a detection head network.

[0017] Optionally, the step of inputting the parking area image set into the vehicle illegal parking detection model to detect vehicle illegal parking and obtain the vehicle illegal parking detection result includes:

[0018] The parking area images in the parking area image set are input into the vehicle illegal parking detection model in chronological order. The parking area images are then processed by three sets of C2F modules, Ghost Shuffle convolution modules and CBAM modules to extract feature maps, perform channel attention weighting and spatial attention weighting, and obtain high-level semantic feature maps at three different scales.

[0019] The high-level semantic feature map is bidirectionally sampled and fused using a BiFPN bidirectional feature pyramid network to obtain three fused and enhanced feature maps at different scales.

[0020] The fused and enhanced feature map is input into the detection head network to identify and locate illegally parked vehicles, and the vehicle illegal parking detection results are obtained, including the ID and location of the illegally parked vehicle.

[0021] Optionally, the vehicle illegal parking detection model is trained using a multi-task joint loss function, which is a weighted sum of classification loss, regression loss, and illegal parking judgment loss.

[0022] Optionally, the step of inputting the state data set into the building energy consumption prediction model to predict building energy consumption and obtain the building energy consumption prediction result includes:

[0023] The building energy consumption prediction model includes a feature extraction layer, a bidirectional LSTM network, an attention mechanism layer, and a fully connected layer;

[0024] The state data set is input into the feature extraction layer for feature extraction to obtain a set of energy consumption feature vectors.

[0025] The energy consumption feature vectors in the energy consumption feature vector set are input into the bidirectional LSTM network in chronological order to perform forward hidden state extraction, reverse hidden state extraction and hidden state concatenation to obtain the final hidden state.

[0026] The key feature vector is obtained by applying attention weights to the final hidden state through an attention mechanism layer.

[0027] The key feature vectors are mapped to building energy consumption prediction results through a fully connected layer. The building energy consumption prediction results include building energy consumption change curves for future time periods.

[0028] Optionally, the step of inputting the set of images of the canteen area into a face recognition model for face recognition and dining authorization determination to obtain the dining authorization result includes:

[0029] The image set of the canteen area includes images of the canteen entrance, the food tray collection area, the food serving area, and the food tray return area;

[0030] Images from the cafeteria entrance, the food tray collection area, the food serving area, and the food tray return area are input into a facial recognition model for facial recognition. If an unauthorized face is detected, the dining authorization result is "unauthorized presence exists"; otherwise, the dining authorization result is "all authorized".

[0031] The present invention also provides a smart building management device based on multi-source data fusion, used to implement the aforementioned smart building management method based on multi-source data fusion, the device comprising:

[0032] The data acquisition module is used to acquire a multi-source data set of the target building through a sensor network. The multi-source data set includes a set of images of the parking area outside the target building, a set of status data inside the target building, and a set of images of the canteen area inside the target building.

[0033] The vehicle illegal parking detection result acquisition module is used to build a vehicle illegal parking detection model based on the YOLOv8 network, input the set of parking area images into the vehicle illegal parking detection model to detect vehicle illegal parking, and obtain the vehicle illegal parking detection result.

[0034] The building energy consumption prediction result acquisition module is used to construct a building energy consumption prediction model, input the state data set into the building energy consumption prediction model to predict building energy consumption, and obtain the building energy consumption prediction result.

[0035] The dining authorization result acquisition module is used to build a face recognition model. The set of images of the canteen area is input into the face recognition model to perform face recognition and dining authorization judgment, and obtain the dining authorization result.

[0036] The management module is used to issue parking violation alerts to users of illegally parked vehicles based on vehicle parking violation detection results, adjust the power of equipment in target buildings based on building energy consumption prediction results, and issue unauthorized alerts based on dining authorization results.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned smart building management method based on multi-source data fusion.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned smart building management method based on multi-source data fusion.

[0039] The present invention has the following beneficial effects:

[0040] 1. Sensor networks can comprehensively and in real-time collect images of the target building's external parking areas, internal status data, and canteen areas, laying a comprehensive and dynamic data foundation for smart management. A vehicle illegal parking detection model built on an improved YOLOv8 platform can accurately output the ID and location of illegally parked vehicles. Combined with automatic alarms, this significantly improves the efficiency of illegal parking handling, reduces errors and omissions during manual patrols, and effectively maintains parking order. A building energy consumption prediction model can output future energy consumption change curves, allowing for priority power adjustments to equipment to achieve energy saving, consumption reduction, and extended equipment lifespan. A facial recognition model can perform authorization checks on images from multiple areas of the canteen, accurately intercepting unauthorized personnel, improving dining efficiency, and ensuring canteen safety and rational resource utilization. Finally, automated alarms and adjustments based on these three types of results achieve efficient and automated building management.

[0041] 2. A vehicle illegal parking detection model is constructed based on the YOLOv8 network. By adding a GhostShuffle convolutional module to the initial backbone network, the computational cost of the network is reduced. By adding a CBAM module to the initial backbone network, the important features of the vehicle are focused. By replacing the initial Neck network with a BiFPN bidirectional feature pyramid network, the fusion capability of multi-scale features is enhanced. This enables the vehicle illegal parking detection model to accurately identify the vehicle location and determine whether it is illegally parked, thereby improving the accuracy of vehicle illegal parking detection.

[0042] 3. A building energy consumption prediction model is constructed based on a feature extraction layer, a bidirectional LSTM network, an attention mechanism layer, and a fully connected layer. The feature extraction layer extracts features from multi-source data, the bidirectional LSTM network captures the temporal correlation between features from multi-source data, and the attention mechanism layer focuses on key influencing factors in the features from multi-source data. This enables the building energy consumption prediction model to accurately predict energy consumption changes in future time periods, thereby improving the efficiency and accuracy of building power regulation. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0045] Figure 2 This is a structural diagram of a vehicle illegal parking detection model;

[0046] Figure 3 This is a structural diagram of the device according to an embodiment of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0048] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0049] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0050] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0051] Reference Figure 1 This invention provides a smart building management method based on multi-source data fusion, comprising:

[0052] The target building is acquired through a sensor network. The multi-source data set includes images of the parking area outside the target building, status data inside the target building, and images of the canteen area inside the target building. The status data set includes indoor temperature data, indoor humidity data, indoor light intensity data, smoke concentration data, personnel activity status data, personnel entry and exit record data, and equipment energy consumption data.

[0053] In some embodiments, the sensor network includes temperature and humidity sensors, light sensors, smoke sensors, surveillance cameras, human infrared sensors, access control sensors, and energy consumption sensors;

[0054] In some embodiments, during sensor deployment, differentiated planning based on data acquisition scenarios is required. For parking areas outside the target building, the focus is on deploying surveillance cameras, employing a deployment strategy of entrance / exit coverage plus precise parking space coverage. One camera is installed at each parking area entrance and exit to record vehicle entry / exit and license plate information. One camera is deployed directly above or to the side of each parking space to ensure complete capture of the vehicle's position within the space, with a camera coverage overlap of at least 10% to avoid data loss due to obstruction. In open-air scenarios, sunshades and rainproof covers must be installed on the cameras. For status data acquisition inside the building, a set of temperature and humidity sensors and a light sensor are deployed every 50 meters in public areas such as corridors, and one human sensor is installed on each floor of stairwells. Human infrared sensors are deployed in all equipment rooms, including one temperature and humidity sensor, one smoke sensor, and one energy consumption sensor. In functional areas such as offices, one set of temperature and humidity sensors and one light sensor are installed every 20 square meters. In large spaces such as conference rooms, two sets of temperature and humidity sensors and one human infrared sensor are installed every 100 square meters, following a diagonal deployment principle. Access control sensors are deployed at all entrances and exits requiring identity verification and are linked to the access control controller. For the canteen area inside the building, surveillance cameras need to cover the canteen entrance, the tray collection area, the food serving area, and the tray return area. The camera height is 1.8-2.2 meters and level with people's faces. In backlit scenes, supplementary lighting is used. At the same time, one smoke sensor is deployed in the canteen kitchen, and one set of temperature and humidity sensors is installed every 100 square meters in the dining area.

[0055] During data collection, a combination of timed collection and event-triggered methods is used to set the collection frequency and triggering mechanism. The parking area image set is collected at 1fps by default, and the frame rate is increased to 5fps when vehicle movement is detected. Indoor temperature and humidity data are collected every 5 minutes. Indoor light intensity data is collected every 10 minutes. Smoke concentration data is collected every 5 minutes, and real-time reporting is triggered when the concentration exceeds 0.1mg / m³. Personnel activity status data is reported in real-time when human movement is detected by human infrared sensors, and a heartbeat packet is sent every 30 minutes when there is no activity. Personnel entry and exit record data is collected in real-time when personnel swipe their cards or pass facial verification. Equipment energy consumption data is collected every minute for power data and every hour for cumulative energy consumption data. The canteen area image set is collected at 1fps by default, and the frame rate is increased to 3fps when a face is detected.

[0056] The collected raw data needs to be preprocessed by the edge computing gateway, including converting binary or text data from different sensors into standard JSON format, removing obvious abnormal data and marking suspicious data, compressing image data with H.265 encoding and status data with Gzip compression. Then, the edge nodes transmit data to the data processing unit according to the principle of classification aggregation + timed upload. Status data is uploaded in batches every 1 minute, image data is uploaded in batches of key frames every 5 minutes, and abnormal data is uploaded in real time. In the transmission link, short-range low-power sensors use the LoRa protocol, high-bandwidth devices use WiFi 6 or RS485 bus, and long-distance uses fiber optic Ethernet with 4G / 5G backup links. All transmitted data is encrypted with TLS 1.3.

[0057] A vehicle illegal parking detection model is constructed based on the YOLOv8 network. The set of parking area images is input into the vehicle illegal parking detection model to detect vehicle illegal parking and obtain the vehicle illegal parking detection results.

[0058] In some embodiments, the construction of the vehicle illegal parking detection model based on the YOLOv8 network includes:

[0059] The YOLOv8 network consists of an initial backbone network, an initial Neck network, and a detection head network.

[0060] An improved backbone network is constructed by adding Ghost Shuffle convolutional modules and CBAM modules after each C2F module in the initial backbone network.

[0061] In some embodiments, a concatenated sequence of C2F module - Ghost Shuffle convolution module - CBAM module is adopted. The computational load is reduced by introducing the Ghost Shuffle convolution module. The number of input channels of the Ghost Shuffle convolution module is the same as the number of output channels of the corresponding C2F module, and the number of output channels is the same as the number of input channels. The calculation process is as follows:

[0062] (X)

[0063] (Y)

[0064] in, This is the input feature map for the Ghost Shuffle convolutional module. This is an intermediate feature map. This is the output feature map of the GhostShuffle convolutional module. This is the main convolution operation; This is a linear transformation, implemented using Depthwise convolution. This is a channel splicing operation; Shuffle is a channel shuffling operation, enhancing cross-channel information interaction.

[0065] The convolutional kernel is 3×3, the stride is 1, the padding is 1, and the number of groups is set to 1 / 4 of the number of input channels. The shuffle operation requires the output feature map to be divided into 4 groups according to the channels. The channels in each group are randomly shuffled and then reassembled to realize the interaction of channel information.

[0066] The CBAM module employs a channel attention-spatial attention concatenated structure. The input is the output feature map from the GhostShuffle convolution module. The channel attention submodule performs global average pooling and global max pooling on the input feature map, yielding two vectors respectively. These two vectors are then input into a multilayer perceptron with one hidden layer. The sum of the two vectors is followed by sigmoid activation to obtain the channel attention weights. The first feature map is obtained by multiplying it channel by channel with the input feature map. The calculation process is as follows:

[0067]

[0068]

[0069] in, For global average pooling, GMP() is global max pooling, and MLP() is a multilayer perceptron. It is the Sigmoid activation function. For successive multiplication;

[0070] The first feature map after channel attention processing by the spatial attention submodule Average pooling and max pooling are performed along the channel dimension to obtain two H×W×1 feature maps, which are then concatenated into an H×W×2 feature map. Spatial attention weights are obtained by performing 3×3 convolution and sigmoid activation. , with the first feature map Element-wise multiplication yields a high-level semantic feature map. The calculation process is as follows:

[0071]

[0072]

[0073] Where f() is the convolution operation, For splicing operations;

[0074] Replace the initial Neck network with a BiFPN bidirectional feature pyramid network;

[0075] In some embodiments, the fusion efficiency of multi-scale features is improved by using a BiFPN bidirectional feature pyramid network, and the calculation process is as follows:

[0076]

[0077] Where i is the number of the input feature map; These are learnable weights; To prevent division by zero, =1×10 -4 ;P i Let i be the i-th input feature map; Resize is an upsampling or downsampling operation to make the feature maps of each layer have the same resolution; To output a fused feature map;

[0078] A vehicle illegal parking detection model is composed of an improved backbone network, a BiFPN bidirectional feature pyramid network, and a detection head network.

[0079] In some embodiments, the structure of the vehicle illegal parking detection model is as follows: Figure 2 As shown, the vehicle illegal parking detection model adopts a lightweight deployment and prunes the lanes:

[0080]

[0081] in, For the first The BN scaling factor for the c-th channel of layer , C represents the initial total number of network layers, and C represents the number of channels in each layer. This represents the total number of layers in the pruned network.

[0082] In some embodiments, inputting a set of parking area images into a vehicle illegal parking detection model to detect illegal parking and obtain vehicle illegal parking detection results includes:

[0083] The parking area images in the parking area image set are input into the vehicle illegal parking detection model in chronological order. The parking area images are then processed by three sets of C2F modules, Ghost Shuffle convolution modules and CBAM modules to extract feature maps, perform channel attention weighting and spatial attention weighting, and obtain high-level semantic feature maps at three different scales.

[0084] In some embodiments, the parking area image set acquired from the sensor network carries a timestamp accurate to milliseconds for each image. The images are first sorted in ascending order of timestamps to form a continuous temporal sequence, avoiding frame gaps in detection due to disordered order. Next, the image size is standardized: the standard input for the vehicle illegal parking detection model is 640×640. The sorted images are then scaled proportionally with black padding, i.e., scaled according to a scaling factor. Areas not filling the 640×640 canvas after scaling are filled with RGB(0,0,0) to ensure the image size is 640×640×3 and the vehicle proportions remain unchanged. Subsequently, image quality is screened, removing blurry images with a gradient mean below 50, images with vehicle occlusion exceeding 30%, and images with abnormal lighting conditions (brightness mean below 80 or above 200). Valid images are then input into the vehicle illegal parking detection model frame by frame in temporal order.

[0085] High-level semantic feature maps are extracted using three sets of C2F modules, a Ghost Shuffle convolutional module, and a CBAM module. These three sets of modules correspond to the improved backbone network's C2F2, C2F3, and C2F4 modules, respectively. The first set, P3, is a 40×40×128 high-level semantic feature map output by C2F2, focusing on local vehicle details. The second set, P4, is a 20×20×256 high-level semantic feature map output by C2F3, focusing on the overall vehicle outline. The third set, P5, is a 10×10×512 high-level semantic feature map output by C2F4, focusing on the global association between the vehicle and the background. Furthermore, the module parameters are kept consistent during frame-by-frame extraction to ensure feature continuity.

[0086] The high-level semantic feature map is bidirectionally sampled and fused using a BiFPN bidirectional feature pyramid network to obtain three fused and enhanced feature maps at different scales.

[0087] In some embodiments, the top-to-bottom path starts with P5, and the number of channels is unified to 128 through 1×1 convolution. Bilinear interpolation is used to upsample to a scale of 20×20. The feature map P4' is obtained by weighted summation with the feature map after 1×1 convolution with P4. Then, P4' is upsampled to a scale of 40×40 and the feature map after 1×1 convolution with P3 is obtained by weighted summation with P3 to obtain the fused and enhanced feature map P3'.

[0088] Starting from P3', the bottom-up path is downsampled to a scale of 20×20 by a 3×3 convolution. The feature map is then weighted and summed with the feature map of P4' after a 1×1 convolution to obtain the fused enhanced feature map P4''. Then, P4'' is downsampled to a scale of 10×10 and weighted and summed with the feature map of P5 after a 1×1 convolution to obtain the fused enhanced feature map P5''.

[0089] The fused and enhanced feature map is input into the detection head network to identify and locate illegally parked vehicles, and the vehicle illegal parking detection results are obtained, including the ID and location of the illegally parked vehicle.

[0090] In some embodiments, the fused enhanced feature maps P3', P4'', and P5'' are input into the three scale detection branches of the detection head network, including a classification branch, a regression branch, and a parking violation judgment branch. The classification branch outputs 2 channels of 1×1 Conv2d for each fused feature map, and obtains the class probability by Sigmoid activation. The regression branch outputs 4 channels, and obtains the coordinate values ​​by ReLU activation. Then, non-maximum suppression is performed. Candidate boxes with a class probability lower than 0.5 are first removed, and then sorted in descending order of parking violation probability. Duplicate boxes with IOU > 0.6 with the baseline box are removed, and valid detection boxes are retained. Next, the vehicle ID is associated. The license plate is obtained through the license plate recognition submodule integrated with the camera. The license plate-vehicle ID mapping is pre-stored, and the ID is matched according to the center distance between the detection box and the license plate box being < 10 pixels. The position coordinates are restored. The normalized coordinates are converted to the vehicle position according to the original image size / 640 ratio. Finally, the parking violation judgment branch outputs the vehicle parking violation detection result, including the vehicle ID, vehicle position, image timestamp, camera ID, parking violation category probability, and whether an alarm was triggered. The result is presented in JSON format.

[0091] In some embodiments, the vehicle illegal parking detection model is trained using a multi-task joint loss function, which is a weighted sum of classification loss, regression loss, and illegal parking judgment loss.

[0092] In some embodiments, the expression for the multi-task joint loss function is:

[0093]

[0094] in, For weight parameters, =0.5, =0.7, =0.3, For classifying losses, To regress the loss, Assess the damage caused by illegal parking;

[0095] Focal loss is used to address class imbalance in classification.

[0096]

[0097] in, and To adjust the parameters, =0.8, =2, Predict the probability that a sample belongs to the true class for the model;

[0098] The regression loss is improved by using SIOU to enhance positioning accuracy.

[0099]

[0100] in, is the distance between the center points of the predicted bounding box and the ground truth bounding box, and c is the diagonal length of the minimum bounding box. As a penalty term for aspect ratio, Here, IOU is the intersection-union ratio;

[0101] The penalty for illegal parking is determined using the BCE (Block Accumulation and Consequence) penalty with time decay:

[0102]

[0103] Where N is the total number of samples, The time the vehicle is stationary. To control time weighting, For the sigmoid function, For real labels, This is the logical value for the illegal parking status output by the model.

[0104] Construct a building energy consumption prediction model, input the state data set into the building energy consumption prediction model to predict building energy consumption, and obtain the building energy consumption prediction results;

[0105] In some embodiments, the step of inputting the state data set into the building energy consumption prediction model to predict building energy consumption and obtain building energy consumption prediction results includes:

[0106] The building energy consumption prediction model includes a feature extraction layer, a bidirectional LSTM network, an attention mechanism layer, and a fully connected layer;

[0107] The state data set is input into the feature extraction layer for feature extraction to obtain a set of energy consumption feature vectors.

[0108] In some embodiments, the status data is divided into numerical data and discrete data. The numerical data includes indoor temperature data, indoor humidity data, indoor light intensity data, smoke concentration data, and equipment energy consumption data. The discrete data includes personnel activity status data and personnel entry and exit record data.

[0109] The feature extraction layer includes a numerical feature processing branch, a discrete feature processing branch, and a feature fusion module. The numerical feature processing branch contains two fully connected network layers. The input is [B,T,5], where B is the batch size (default 32) and T is the time step length (default 12), corresponding to the data collected every 1 hour and 5 minutes. The first fully connected network extracts basic numerical correlations, and the second fully connected network enhances feature representation, ultimately outputting a numerical feature vector of [B,T,16].

[0110] The discrete feature processing branch includes an embedding layer and a 1-layer fully connected network. The embedding layer maps the discrete data of [B,T,2] to a low-dimensional vector of [B,T,4]. The subsequent fully connected network captures discrete semantic relationships and finally outputs a discrete feature vector of [B,T,8].

[0111] The feature fusion module concatenates the outputs of the two branches according to the channel dimension into an energy consumption feature vector of [B,T,24]. All vectors are arranged in the order of input time to form a set of energy consumption feature vectors.

[0112] The energy consumption feature vectors in the energy consumption feature vector set are input into the bidirectional LSTM network in chronological order to perform forward hidden state extraction, reverse hidden state extraction and hidden state concatenation to obtain the final hidden state.

[0113] In some embodiments, the expression for the final hidden state is:

[0114]

[0115]

[0116]

[0117] in, This represents the energy consumption eigenvector at time t. This represents the hidden state of the forward LSTM at time t; This represents the hidden state of the inverse LSTM at time t; This represents the final hidden state after bidirectional concatenation, with a dimension of 2d, where d is the number of hidden units in a single LSTM layer; [;] represents the tensor concatenation operation.

[0118] The key feature vector is obtained by applying attention weights to the final hidden state through an attention mechanism layer.

[0119] In some embodiments, the expression for the key feature vector is:

[0120]

[0121]

[0122]

[0123] Where T is the total number of time steps; This represents a learnable weight matrix used for feature space transformation; This represents the attention score mapping vector at time step t, which maps features to scalar scores. The unnormalized attention score represents the time step t. represents the normalized attention weights at time step t; c represents the key feature vector, which aggregates information from important time steps.

[0124] The key feature vectors are mapped to building energy consumption prediction results through a fully connected layer. The building energy consumption prediction results include building energy consumption change curves for future time periods.

[0125] In some embodiments, the key feature vector is input into the input layer of the fully connected layer, and after linear transformation by the weight matrix W1 and bias b1, a first intermediate vector H1 is obtained. This is then processed by a Dropout layer to obtain an optimized first intermediate vector H1′. The optimized first intermediate vector H1′ is input into the hidden layer, and after operation with the ReLU activation function, weight matrix W2, and bias b2, a second intermediate vector H2 is obtained from the output of the hidden layer. The second intermediate vector H2 is input into the output layer, and after operation with the linear activation function, weight matrix W3, and bias b3, a sequence of predicted energy consumption values ​​Y for the next 24 time steps is obtained. Each element y... i The predicted energy consumption value corresponds to the i-th hour in the future; Matplotlib is used, with the horizontal axis representing the time label T. i Every 6 time steps, a specific time is marked; the vertical axis represents the energy consumption value in kWh, with the value range set to 1.2 times the historical maximum energy consumption; the predicted energy consumption curve is drawn with a solid blue line, and the 95% confidence interval is marked with a light blue shading. The specific energy consumption value and time are marked at the peak and trough positions of the curve. At the same time, CSV format structured data is output, which includes 4 columns: time label, predicted energy consumption value, upper confidence interval, and lower confidence interval, for direct reading and use by the equipment power regulation system.

[0126] A face recognition model is constructed, and the set of images of the canteen area is input into the face recognition model to perform face recognition and dining authorization judgment, and obtain the dining authorization result;

[0127] In some embodiments, inputting the set of images of the canteen area into a face recognition model for face recognition and dining authorization determination to obtain a dining authorization result includes:

[0128] The image set of the canteen area includes images of the canteen entrance, the food tray collection area, the food serving area, and the food tray return area;

[0129] Images from the cafeteria entrance, the food tray collection area, the food serving area, and the food tray return area are input into a facial recognition model for facial recognition. If an unauthorized face is detected, the dining authorization result is "unauthorized presence exists"; otherwise, the dining authorization result is "all authorized".

[0130] In some embodiments, the face recognition model uses a lightweight MobileNet backbone network for face recognition. Four smart cameras are deployed in the cafeteria, placed at the cafeteria entrance, the food tray collection area, the food serving area, and the food tray return area. There are three types of people dining in the cafeteria: park employees, visitors, and park branch employees. The information entry system for park employees and park branch employees is as follows: when park employees go to the cafeteria for meals, their faces are first recognized and recorded by the smart camera at the cafeteria entrance, then their faces are recognized by the camera at the food tray collection area, and finally their faces are recognized again at the food tray return area after they finish eating. If the face is entered into the system, it is considered authorized; otherwise, it is considered unauthorized. The recognition results from the four cameras are combined using a logical AND operation to determine the dining authorization result.

[0131] Based on the results of vehicle illegal parking detection, illegal parking alerts are issued to users of illegally parked vehicles; based on the building energy consumption prediction results, the power of equipment in the target building is adjusted; and based on the dining authorization results, unauthorized alerts are issued.

[0132] In some embodiments, the illegal parking alarm is executed in two steps: multi-channel push and tracking processing. Multi-channel push prioritizes direct access to users, retrieving user contact numbers and building APP accounts from the associated database, and sending SMS messages containing illegal parking information and a 15-minute countdown to departure via Alibaba Cloud SMS service. Simultaneously, a pop-up window containing a real-time map of the illegal parking location, a countdown, and the location marked on Gaode Map is pushed through the building management APP. On-site audible and visual alarms are triggered simultaneously, with alarm zones divided according to parking areas. Each zone has an audible and visual alarm deployed at a 2.5m column, emitting a 60-80 decibel beeping sound and flashing a red light once per second until the vehicle leaves. In the tracking processing phase, the vehicle illegal parking detection model rechecks every minute. If the vehicle moves into a legal parking space or leaves the parking area, all alarms are automatically terminated and the user is notified. If the vehicle does not leave within 15 minutes, the system synchronizes the illegal parking record to the property management backend, generating a work order for on-site verification by property management personnel. To avoid false alarms, after the user submits proof of the fault, management personnel can mark it as "temporarily exempt from liability," and all alarm records must be retained for one year.

[0133] When implementing equipment power regulation, first analyze the energy consumption forecast results, and divide the energy consumption change curve for the next 24 hours into peak, off-peak, and valley periods according to the energy consumption value. The goal is to control the actual energy consumption during peak periods within 90% of the forecast value. Then, formulate regulation strategies according to equipment priority: non-essential equipment is classified as Level 1 priority regulation, shut down during peak periods and turned on as needed during valley periods; semi-essential equipment is classified as Level 2, with power reduced by 20%-30% during peak periods; essential equipment is classified as Level 3, with power reduced by ≤10% during peak periods or staggered operation, and operation must meet constraints.

[0134] The adjustment is implemented in stages. One hour before peak hours, power-off commands are sent to non-essential equipment, power setting commands to semi-essential equipment, and operating mode commands to essential equipment via the building automation system's API interface. Energy consumption sensors collect actual energy consumption every 5 minutes. If actual energy consumption still exceeds the predicted value by 90%, the power of semi-essential equipment is further reduced. During off-peak hours, the power of semi-essential equipment is maintained, while non-essential equipment is used at high frequency. When actual energy consumption is lower than 80% of the predicted value, the efficiency of essential equipment is restored. During off-peak hours, non-essential equipment is shut down, while semi-essential equipment retains its basic power. Equipment that can be off-peak is concentrated between 2-4 AM. A daily adjustment report is generated, comparing the actual and predicted energy consumption curves. If the peak energy consumption reduction rate is less than 5%, the strategy is optimized. If equipment abnormalities are detected during adjustment, an alarm is immediately triggered and the original power is restored. Before implementation, BAS interface compatibility must be ensured. When adjusting equipment that affects comfort, a notification is sent in advance via the app.

[0135] When an unauthorized personnel alarm is triggered, on-site management personnel will be notified first. The system will push a real-time alarm containing a captured image of the unauthorized person, their location and time of appearance, and handling instructions to the gatekeeper and food service staff via the cafeteria management APP. When multiple scenarios trigger the alarm, a small audio-visual alert will be activated at the corresponding location, emitting a 50-decibel buzzing sound and flashing a green light. In subsequent handling, if the person is an external visitor, the staff will guide them to register at the front desk and temporarily add their face to the authorization database. If the person is an irrelevant individual, they will be asked to leave and marked in the APP. If leaving is ineffective, security will be contacted, and the face will be added to the abnormal list. Privacy must be protected during implementation. The captured images of unauthorized personnel are only used for cafeteria management and are subject to administrator access. If an authorized person is misidentified due to obstruction, authorization can be supplemented through card verification.

[0136] Reference Figure 3 This invention provides a smart building management device 30 based on multi-source data fusion, used to implement a smart building management method based on multi-source data fusion. The device includes:

[0137] The data acquisition module 31 is used to acquire a multi-source data set of the target building through a sensor network. The multi-source data set includes a set of images of the parking area outside the target building, a set of status data inside the target building, and a set of images of the canteen area inside the target building.

[0138] The vehicle illegal parking detection result acquisition module 32 is used to build a vehicle illegal parking detection model based on the YOLOv8 network, input the parking area image set into the vehicle illegal parking detection model to detect vehicle illegal parking, and obtain the vehicle illegal parking detection result.

[0139] The building energy consumption prediction result acquisition module 33 is used to construct a building energy consumption prediction model, input the state data set into the building energy consumption prediction model to predict building energy consumption, and obtain building energy consumption prediction results.

[0140] The dining authorization result acquisition module 34 is used to construct a face recognition model. The set of images of the canteen area is input into the face recognition model to perform face recognition and dining authorization judgment, and obtain the dining authorization result.

[0141] The management module 35 is used to issue illegal parking alerts to users of illegally parked vehicles based on the results of vehicle illegal parking detection, adjust the power of equipment in the target building based on the building energy consumption prediction results, and issue unauthorized alerts based on the dining authorization results.

[0142] This application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements any of the above-described schemes of a smart building management method based on multi-source data fusion.

[0143] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0144] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0145] This application also provides a computer-readable medium storing a computer program thereon, which, when executed by a processor, implements the smart building management method based on multi-source data fusion as described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0146] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0147] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A smart building management method based on multi-source data fusion, characterized in that, include: The target building is acquired through a sensor network. The multi-source data set includes images of the parking area outside the target building, status data inside the target building, and images of the canteen area inside the target building. The status data set includes indoor temperature data, indoor humidity data, indoor light intensity data, smoke concentration data, personnel activity status data, personnel entry and exit record data, and equipment energy consumption data. A vehicle illegal parking detection model is constructed based on the YOLOv8 network. The set of parking area images is input into the vehicle illegal parking detection model to detect vehicle illegal parking and obtain the vehicle illegal parking detection results. Construct a building energy consumption prediction model, input the state data set into the building energy consumption prediction model to predict building energy consumption, and obtain the building energy consumption prediction results; A face recognition model is constructed, and the set of images of the canteen area is input into the face recognition model to perform face recognition and dining authorization judgment, and obtain the dining authorization result; Based on the results of vehicle illegal parking detection, illegal parking alerts are issued to users of illegally parked vehicles; based on the building energy consumption prediction results, the power of equipment in the target building is adjusted; and based on the dining authorization results, unauthorized alerts are issued.

2. The intelligent building management method based on multi-source data fusion according to claim 1, characterized in that, The sensor network includes temperature and humidity sensors, light sensors, smoke sensors, surveillance cameras, human infrared sensors, access control sensors, and energy consumption sensors.

3. The intelligent building management method based on multi-source data fusion according to claim 1, characterized in that, The vehicle illegal parking detection model built on the YOLOv8 network includes: The YOLOv8 network consists of an initial backbone network, an initial Neck network, and a detection head network. An improved backbone network is constructed by adding Ghost Shuffle convolutional modules and CBAM modules after each C2F module in the initial backbone network. Replace the initial Neck network with a BiFPN bidirectional feature pyramid network; The vehicle illegal parking detection model consists of an improved backbone network, a BiFPN bidirectional feature pyramid network, and a detection head network.

4. The intelligent building management method based on multi-source data fusion according to claim 3, characterized in that, The step of inputting a set of parking area images into a vehicle illegal parking detection model to detect illegal parking and obtain vehicle illegal parking detection results includes: The parking area images in the parking area image set are input into the vehicle illegal parking detection model in chronological order. The parking area images are then processed by three sets of C2F modules, Ghost Shuffle convolution modules and CBAM modules to extract feature maps, perform channel attention weighting and spatial attention weighting, and obtain high-level semantic feature maps at three different scales. The high-level semantic feature map is bidirectionally sampled and fused using a BiFPN bidirectional feature pyramid network to obtain three fused and enhanced feature maps at different scales. The fused and enhanced feature map is input into the detection head network to identify and locate illegally parked vehicles, and the vehicle illegal parking detection results are obtained, including the ID and location of the illegally parked vehicle.

5. The intelligent building management method based on multi-source data fusion according to claim 3, characterized in that, The vehicle illegal parking detection model is trained using a multi-task joint loss function, which is a weighted sum of classification loss, regression loss, and illegal parking judgment loss.

6. The intelligent building management method based on multi-source data fusion according to claim 1, characterized in that, The step of inputting the state data set into the building energy consumption prediction model to predict building energy consumption and obtain the building energy consumption prediction result includes: The building energy consumption prediction model includes a feature extraction layer, a bidirectional LSTM network, an attention mechanism layer, and a fully connected layer; The state data set is input into the feature extraction layer for feature extraction to obtain a set of energy consumption feature vectors. The energy consumption feature vectors in the energy consumption feature vector set are input into the bidirectional LSTM network in chronological order to perform forward hidden state extraction, reverse hidden state extraction and hidden state concatenation to obtain the final hidden state. The key feature vector is obtained by applying attention weights to the final hidden state through an attention mechanism layer. The key feature vectors are mapped to building energy consumption prediction results through a fully connected layer. The building energy consumption prediction results include building energy consumption change curves for future time periods.

7. The intelligent building management method based on multi-source data fusion according to claim 1, characterized in that, The process of inputting the set of images from the canteen area into a face recognition model for face recognition and dining authorization determination to obtain dining authorization results includes: The image set of the canteen area includes images of the canteen entrance, the food tray collection area, the food serving area, and the food tray return area; Images from the cafeteria entrance, the food tray collection area, the food serving area, and the food tray return area are input into a facial recognition model for facial recognition. If an unauthorized face is detected, the dining authorization result is "unauthorized presence exists"; otherwise, the dining authorization result is "all authorized".

8. A smart building management device based on multi-source data fusion, used to implement the smart building management method based on multi-source data fusion as described in any one of claims 1 to 7, characterized in that, The device includes: The data acquisition module is used to acquire a multi-source data set of the target building through a sensor network. The multi-source data set includes a set of images of the parking area outside the target building, a set of status data inside the target building, and a set of images of the canteen area inside the target building. The vehicle illegal parking detection result acquisition module is used to build a vehicle illegal parking detection model based on the YOLOv8 network, input the set of parking area images into the vehicle illegal parking detection model to detect vehicle illegal parking, and obtain the vehicle illegal parking detection result. The building energy consumption prediction result acquisition module is used to construct a building energy consumption prediction model, input the state data set into the building energy consumption prediction model to predict building energy consumption, and obtain the building energy consumption prediction result. The dining authorization result acquisition module is used to build a face recognition model. The set of images of the canteen area is input into the face recognition model to perform face recognition and dining authorization judgment, and obtain the dining authorization result. The management module is used to issue parking violation alerts to users of illegally parked vehicles based on vehicle parking violation detection results, adjust the power of equipment in target buildings based on building energy consumption prediction results, and issue unauthorized alerts based on dining authorization results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the smart building management method based on multi-source data fusion as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the smart building management method based on multi-source data fusion as described in any one of claims 1 to 7.

Images