Intelligent community comprehensive management method and system

Abstract

The invention provides a comprehensive management method and system for a smart community, and the method comprises the steps: collecting multi-modal basic data which comprises weather data, building data, elevator data, vehicle data, plant growth data and network power water supply pipeline data; based on a pre-trained first model and a pre-trained second model, processing the multi-modal basic data to obtain multi-modal information data; acquiring community basic data, constructing a community digital twinborn model based on the multi-modal information data and the community basic data, and displaying the community digital twinborn model in real time; and performing early warning and processing on community abnormity based on the multi-modal information data. The multi-modal data is collected and is processed based on the first model and the second model, so that the community management comprehensiveness and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of community management, and particularly to a method, system, storage medium and electronic device for digital comprehensive management of smart communities. Background Art

[0002] With the acceleration of the urbanization process and the increase in the urban population scale, urban communities have greatly improved in terms of scale, quantity, and complexity. As a result, the traditional community management method is increasingly unsuitable for the management of complex communities. In the prior art represented by CN119229008A, CN117541054A, CN117726162A, etc., for the multi-modal data collected during the community management process, there are problems such as difficult data processing and utilization, single analysis method, uneven quality, and unintuitive information display, which affect the scientific nature of community management decisions and are difficult to meet the complex management needs of modern communities. Summary of the Invention

[0003] The present invention aims to provide a comprehensive management method and system for smart communities to improve the intelligent and digital management capabilities of complex communities.

[0004] The present invention provides a comprehensive management method for smart communities, including the following steps: Step 1, collect multi-modal basic data, where the multi-modal data includes collected weather data, building data, elevator data, vehicle data, plant growth data, network power and water supply pipeline data, etc.

[0005] Specifically, different types of data collection devices are arranged at different locations such as buildings, roads, greening areas, and public pipelines in the community, such as ordinary cameras, infrared cameras, high-definition cameras, temperature sensors, humidity sensors, pressure sensors, displacement sensors, speed sensors, etc., to collect weather data, the structure and deformation data of buildings, the operation data of elevators, the entry and driving data of vehicles, the growth data of plants, and the operation data of network power and water supply pipelines. These sensors and cameras transmit the collected data in real time or periodically.

[0006] Furthermore, before transmitting the collected multi-modal basic data, preprocessing operations such as data cleaning, filtering, and normalization can also be performed on the data to improve the data quality and usability.

[0007] Step 2, based on the pre-trained first model and second model, process the multi-modal basic data to obtain multi-modal information data.

[0008] Specifically, a pre-trained first deep model with a CNN-LSTM hybrid architecture processes the sensor data collected by temperature sensors, humidity sensors, pressure sensors, displacement sensors, speed sensors, etc., and a spatio-temporal attention module is introduced into the first deep model; a pre-trained second deep model with a 3D convolution-Transformer hybrid processes the image processing collected by ordinary cameras, infrared cameras, high-definition cameras, etc. The data output by the first deep model and the second deep model is the multi-modal information data.

[0009] Further, during the process of pre-training the model, the historical sensor data is automatically annotated, and the first deep model is trained using the automatically annotated historical sensor data; the historical image data is manually or semi-automatically annotated, and the second deep model is trained using the annotated historical image data.

[0010] Step 3: Obtain the community basic data, and construct a community digital twin model based on the multi-modal information data and the community basic data; display the community digital twin model in real time.

[0011] Specifically, obtain the basic data such as building drawings, resident information, and facility maintenance records of the community, integrate the basic data to form a complete basic data set. Integrate the multi-modal information data obtained after processing by the first model and the second model with the basic data set, and use 3D modeling technology to construct a digital twin model of the community, and display the multi-modal data of the community in real time.

[0012] Further, the digital twin model dynamically updates the displayed information through the data transmitted in real time or periodically by sensors and cameras.

[0013] Further, the digital twin model can also dynamically adjust the display mode based on the multi-modal information data.

[0014] Step 4: Based on the multi-modal information data, give early warnings and handle community anomalies.

[0015] Specifically, when the multi-modal information data reflects that there are anomalies in the community, based on the type of the anomalies, give early warnings through digital twin displays, anomaly information pushes, etc., and at the same time match corresponding handling measures.

[0016] Further, weights can also be assigned to the multi-modal data, and the weights can be dynamically adjusted according to the actual situation. Based on the multi-modal data and weights, calculate the anomaly level of the community, and adopt different early warning methods and handling methods for anomalies of different levels.

[0017] Furthermore, the multi-modal information data can be further fused, and community anomalies can be warned and processed based on the fused multi-modal information data.

[0018] Corresponding to the intelligent community comprehensive management method, the present invention also provides an intelligent community comprehensive management system, including: A collection module that collects multi-modal basic data, and the multi-modal data includes collected weather data, building data, elevator data, vehicle data, plant growth data, network power and water supply pipeline data, etc.

[0019] A processing module that processes the multi-modal basic data based on pre-trained first and second models to obtain multi-modal information data.

[0020] A display module that obtains community basic data and constructs a community digital twin model based on the multi-modal information data and the community basic data; and displays the community digital twin model in real time.

[0021] An early warning module that warns and processes community anomalies based on the multi-modal information data.

[0022] The present invention also provides an electronic device, which includes: a memory and a processor, and the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the intelligent community comprehensive management method of the present invention.

[0023] The present invention also provides a computer-readable storage medium, including a computer program, and when the computer program runs on an electronic device, the electronic device executes the intelligent community comprehensive management method of the present invention. An intelligent community comprehensive management method and system provided by the present invention collect multi-modal basic data, and the multi-modal data includes collected weather data, building data, elevator data, vehicle data, plant growth data, network power and water supply pipeline data; based on pre-trained first and second models, the multi-modal basic data is processed to obtain multi-modal information data; community basic data is obtained, and a community digital twin model is constructed based on the multi-modal information data and the community basic data, and the community digital twin model is displayed in real time; based on the multi-modal information data, community anomalies are warned and processed. Collecting multi-modal data and processing the multi-modal data respectively based on the first and second models, the first model can effectively mine the hidden rules and trends in the sensor data, and the second model can accurately capture the changes and motion information in the three-dimensional space and highlight the objects and areas of concern, which can more comprehensively and accurately perceive various abnormal situations in the community, provide more reliable data support for community early warning, and improve the comprehensiveness and accuracy of community management at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the comprehensive management method for the intelligent community of the present invention.

[0025] Figure 2 It is a schematic diagram of the comprehensive management system for the intelligent community of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited by the exemplary embodiments described herein.

[0027] In the prior art, community management usually relies on property management or community staff for manual management, or relies on a single data source for digital management, and there is no way to adapt to the increasingly complex community management situation.

[0028] Based on this, the purpose of the present invention is to provide a comprehensive management method and system for an intelligent community to improve the intelligent and digital management capabilities of complex communities.

[0029] The present invention provides a comprehensive management method for an intelligent community, including the following steps: Step 1, collect multi-modal basic data, and the multi-modal data includes collected weather data, building data, elevator data, vehicle data, plant growth data, network power supply and water supply pipeline data, etc.

[0030] Specifically, different types of data collection devices are arranged at different positions such as buildings, roads, greening sites, and public pipelines in the community, such as ordinary cameras, infrared cameras, high-definition cameras, temperature sensors, humidity sensors, pressure sensors, displacement sensors, speed sensors, etc., to collect weather data, the structure and deformation data of buildings, the operation data of elevators, the entry and driving data of vehicles, the growth data of plants, and the operation data of network power supply and water supply pipelines. These sensors and cameras transmit the collected data in real time or periodically.

[0031] Furthermore, before transmitting the collected multi-modal basic data, preprocessing operations such as data cleaning, filtering, and normalization can also be performed on the data to improve the data quality and usability.

[0032] Step 2, based on the pre-trained first model and second model, process the multi-modal basic data to obtain multi-modal information data.

[0033] Specifically, the types of the multimodal basic data are numerous and the features are complex. Therefore, if the multimodal data is processed based on a single data processing model, problems such as distorted processing results are likely to occur due to model mismatch, interference between data, etc. In this embodiment, a pre-trained first deep model with a CNN-LSTM hybrid architecture is used to process the sensor data collected by temperature sensors, humidity sensors, pressure sensors, displacement sensors, speed sensors, etc., and a spatio-temporal attention module is introduced into the first deep model. The CNN part uses 3 layers of convolution, the convolution kernel size is 3×3, the stride is 1, ReLU is used as the activation function, and the pooling layer uses max pooling (2×2). The LSTM part sets the number of hidden layer units to 128, the time step is 10, and the output fully connected layer uses the Sigmoid activation function. The first deep model can accurately extract the local features of the time series of sensor data and capture the long-term time dependence relationship, so as to accurately discover the hidden laws and trends in the sensor data, such as the tiny deformation of the building, the abnormal operation of the elevator, etc.; A pre-trained second deep model with a 3D convolution-Transformer hybrid is used to process the image processing collected by ordinary cameras, infrared cameras, high-definition cameras, etc. The 3D convolution layer uses a 5×5×5 convolution kernel, the number of channels is 64, the stride is 2, followed by a BatchNorm layer, and the Transformer part sets 4 attention heads. The second deep model can focus on the changes and motion information of the object in the three-dimensional space and build a global dependence relationship, and at the same time highlight the features of the objects and regions that need to be concerned in the image. The data output by the first deep model and the second deep model is the multimodal information data. Through the accurate processing of the multimodal data by the two models, the actual situation in the community can be perceived more comprehensively and accurately, providing accurate data support for subsequent processing.

[0034] Further, during the pre-training of the model, the historical sensor data is automatically labeled. For example, the building deformation data collected by the displacement sensor is automatically labeled as normal or abnormal, and the vehicle speed collected by the speed sensor is automatically labeled as safe or speeding, etc. Then, the first deep model is trained using the automatically labeled historical sensor data. Further, a clustering algorithm (such as K-means clustering) can be used to automatically label the historical sensor data. Using a clustering algorithm can quickly and objectively label a large amount of data, avoiding the subjectivity and inconsistency of manual labeling, while improving the efficiency and quality of data labeling, providing reliable training data for the training of the first deep model, and thus improving the accuracy and performance of the model. For historical image data, manual labeling or semi-automatic labeling is performed. For example, the targets in the image (such as vehicles, people, infrastructure, green vegetation, etc.) are labeled, and information such as their categories, positions, behaviors, and states is labeled. Then, the second deep model is trained using the labeled historical image data. Further, when performing manual labeling, a combination of bounding box labeling and semantic segmentation labeling is used; when performing semi-automatic labeling, a template matching-based or deep learning-based object detection algorithm (such as YOLO or Faster R-CNN) is used for preliminary labeling, and then corrected manually. The preliminary automatic labeling in semi-automatic labeling can quickly locate the target area in the image, reducing the workload of manual labeling, and then corrected and supplemented manually to ensure the accuracy of labeling. This labeling method can improve the labeling efficiency while ensuring the data quality, providing high-quality data support for the training of the second deep model.

[0035] Further, the first deep model and the second deep model are trained using the Stochastic Gradient Descent (SGD) or Adam optimization algorithm. The loss function of the first deep model uses the Mean Squared Error (MSE) loss function: [formula], and the loss function of the second deep model uses the cross-entropy loss: [formula]. The total loss is the weighted sum: [formula], where [parameter].

[0036] Further, data augmentation techniques (such as rotating, flipping, scaling, and cropping image data, adding noise and data interpolation to sensor data) are used to expand the training dataset, improving the generalization ability and robustness of the model. At the same time, the cross-validation method is used to evaluate and optimize the first deep model and the second deep model to ensure the stability and accuracy of the model in different scenarios.

[0037] Step 3: Obtain the community basic data, and construct a community digital twin model based on the multi-modal information data and the community basic data; and display the community digital twin model in real time.

[0038] Specifically, obtain basic data such as architectural drawings, resident information, and facility maintenance records of the community, integrate the basic data to form a complete basic data set. Integrate the multimodal information data obtained after processing the first model and the second model with the basic data set, and use 3D modeling technology to construct a digital twin model of the community to display the multimodal data of the community in real time.

[0039] Furthermore, the digital twin model dynamically updates the displayed information through data transmitted in real time or periodically by sensors and cameras, including the operating status of infrastructure, environmental data, personnel activity data, etc.

[0040] Furthermore, the digital twin model can also dynamically adjust the display mode based on the multimodal information data. For example, it can switch between the daytime mode, nighttime mode, seasonal mode, etc. of the digital twin model based on time information. During the night, it highlights the display of lighting facilities and security monitoring. During summer, it highlights the display of drainage systems and flood prevention facilities. During winter, it highlights the display of ice fall and ground anti-slip warnings. It shows the growth changes of community greening plants in different seasons and the traffic flow changes of people and vehicles in the community at different time periods, etc., providing intuitive and clear community information display and decision-making support for community management personnel and residents. It can also switch between the normal mode, warning mode, maintenance mode, etc. of the digital twin model based on multimodal information data. When the community is in a normal state, it displays the normal information of the community. When the community is at risk of strong winds, it switches to the wind resistance mode to display the wind resistance risk distribution of the community. When the community is at risk of rain and snow, it switches to the rain and snow mode to display the risk distribution of water accumulation and slippery areas in the community. When the pipelines in the community are under maintenance, it switches to the pipeline maintenance mode to display the pipeline network topology structure, etc.

[0041] Step 4: Based on the multimodal information data, give early warnings and handle community anomalies.

[0042] Specifically, when the multimodal information data reflects that there are anomalies in the community, based on the type of the anomaly, give early warnings through means such as digital twin display and anomaly information push, and at the same time match corresponding handling measures. For example, when abnormal deformation data of a certain building is detected, display warning information in the digital twin model and accurately push the warning information to the affected residents and relevant management departments, and at the same time match the corresponding building maintenance plan, such as strengthening the building structure and repairing the damaged parts. When abnormal driving of community vehicles is detected, accurately push the warning information to relevant parties such as community management personnel, vehicle owners, and affected residents. When there are anomalies such as lodging and diseases in community green plants, accurately push the warning information to community management personnel and match the corresponding maintenance plan and update plan according to plant growth data and environmental data.

[0043] Further, weights can also be assigned to the multimodal data, and the weights can be dynamically adjusted according to the actual situation to reflect the influence degree of different multimodal information data on community management. Based on the multimodal data and weights, the anomaly level of the community is calculated, and different warning methods and processing methods are adopted for anomalies at different levels.

[0044] Further, the multimodal information data can also be fused, and based on the fused multimodal information data, early warnings and processing of community anomalies are carried out.

[0045] Corresponding to the intelligent community comprehensive management method, the present invention also provides an intelligent community comprehensive management system, including: A collection module that collects multimodal basic data, where the multimodal data includes collected weather data, building data, elevator data, vehicle data, plant growth data, network power supply and water supply pipeline data, etc.

[0046] A processing module that processes the multimodal basic data based on a pre-trained first model and a second model to obtain multimodal information data.

[0047] A display module that obtains community basic data, constructs a community digital twin model based on the multimodal information data and the community basic data; and displays the community digital twin model in real time.

[0048] An early warning module that conducts early warnings and processing of community anomalies based on the multimodal information data.

[0049] The present invention also provides an electronic device, where the electronic device includes: a memory and a processor, and the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the intelligent community comprehensive management method of the present invention.

[0050] The present invention also provides a computer-readable storage medium, including a computer program, and when the computer program runs on an electronic device, the electronic device executes the intelligent community comprehensive management method of the present invention.

[0051] In summary, the present invention discloses a comprehensive management method and system for an intelligent community, which collects multi-modal basic data. The multi-modal data includes weather data, building data, elevator data, vehicle data, plant growth data, and network power and water supply pipeline data. Based on the pre-trained first model and second model, the multi-modal basic data is processed to obtain multi-modal information data. Community basic data is acquired, and a community digital twin model is constructed based on the multi-modal information data and the community basic data, and the community digital twin model is displayed in real time. Based on the multi-modal information data, early warnings and processing of community anomalies are carried out. By collecting multi-modal data and processing the multi-modal data respectively based on the first model and the second model, the first model can effectively mine the hidden laws and trends in the sensor data, and the second model can accurately capture the changes and motion information in the three-dimensional space and highlight the objects and regions of concern, enabling a more comprehensive and accurate perception of various abnormal situations in the community, providing more reliable data support for community early warnings, and at the same time improving the comprehensiveness and accuracy of community management.

[0052] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. In addition, the above-mentioned specific details are only for illustrative and easy-to-understand purposes, rather than limitations. The above details do not limit the present disclosure to necessarily adopt the above specific details for implementation.

Claims

1. A comprehensive management method for intelligent communities, characterized in that, Including: Step 1: Collect multi-modal basic data, where the multi-modal basic data includes weather data, building data, elevator data, vehicle data, plant growth data, and network power and water supply pipeline data; Step 2: Process the multi-modal basic data based on pre-trained first and second models to obtain multi-modal information data; Step 3: Obtain community basic information data, and construct a community digital twin model based on the multi-modal information data and the community basic information data; display the community digital twin model in real time; Step 4: Based on the multi-modal information data, give early warnings and handle community anomalies.

2. The method according to claim 1, characterized in that Collect multi-modal basic data, including: Arrange data collection devices in the community, including ordinary cameras, infrared cameras, high-definition cameras, temperature sensors, humidity sensors, pressure sensors, displacement sensors, and speed sensors; Collect weather data, building structure and deformation data, elevator operation data, vehicle entry and driving data, plant growth data, and network power and water supply pipeline operation data; Clean, filter, and normalize the data.

3. The method according to claim 1, wherein Based on pre-trained first and second models, process the multi-modal basic data, including: Use a pre-trained first deep model with a CNN-LSTM hybrid architecture to process the sensor data collected by temperature sensors, humidity sensors, pressure sensors, displacement sensors, and speed sensors, and introduce a spatio-temporal attention module into the first deep model; Use a pre-trained second deep model with a 3D convolution-Transformer hybrid to process the image processing collected by ordinary cameras, infrared cameras, and high-definition cameras.

4. The method according to claim 3, wherein: During the pre-training of the model, automatically annotate historical sensor data, and use the automatically annotated historical sensor data to train the first deep model; Manually annotate or semi-automatically annotate historical image data, and use the annotated historical image data to train the second deep model.

5. The method according to claim 1, wherein The said Step 3 includes: Obtain the basic information data of the community, and the basic information data includes: building drawings, resident information, and facility maintenance records; Integrate the basic information data to form a complete basic information data set; Integrate the multi-modal information data with the basic information data set, and use 3D modeling technology to construct a digital twin model of the community, and display the multi-modal data of the community in real time.

6. The method according to claim 5, wherein: The digital twin model dynamically adjusts the display mode based on the multi-modal information data.

7. The method according to claim 1, characterized in that, The said Step 4 includes: When the multi-modal information data reflects that there are anomalies in the community, based on the type of the anomalies, give early warnings through digital twin display and anomaly information push, and match corresponding handling measures, and / or, assign weights to the multi-modal data; calculate the anomaly level of the community based on the multi-modal data and weights, and give early warnings based on the anomaly level and match corresponding handling measures; And / or, fuse the multi-modal information data, and based on the fused multi-modal information data, give early warnings of and handle community anomalies.

8. An integrated intelligent community management system, characterized in that, It includes: A collection module that collects multi-modal basic data, where the multi-modal data includes collected weather data, building data, elevator data, vehicle data, plant growth data, and network power and water supply pipeline data; A processing module that processes the multi-modal basic data based on a pre-trained first model and a second model to obtain multi-modal information data; A display module that obtains community basic information data, constructs a community digital twin model based on the multi-modal information data and the community basic information data; and displays the community digital twin model in real time; An early warning module that gives early warnings of and handles community anomalies based on the multi-modal information data.

9. An electronic device, characterized in that, The electronic device includes: a memory and a processor, and the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the intelligent community comprehensive management method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It includes a computer program, and when the computer program runs on an electronic device, the electronic device executes the intelligent community comprehensive management method described in any one of claims 1 to 7.

Images