Multi-modal dynamic synchronous database system for smart city

Abstract

The invention relates to the technical field of data processing, and particularly discloses a multi-modal dynamic synchronous database system for a smart city, which is used for solving the problem that on the premise that an existing real-time database cannot keep low delay and consistency at the same time, structured sensor time sequence data is associated and fused with social media emotion and standing information, and the real-time database is not consistent. The system comprises a data interface module, an agent module, a subscription publishing module, a control command module, a real-time data processing module, a kernel model management module, a cluster and data synchronization module, a plug-in module and a data storage module. Structured sensor time sequence data and social media emotion and standing are fused in a real-time database, low-delay high-consistency linkage perception and differentiated accurate warning of multi-modal data are achieved, and the sensitivity and decision accuracy of smart city environment monitoring, traffic management and public safety early warning are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a multimodal dynamic synchronization database system for smart cities. Background Art

[0002] In smart cities, IoT sensors such as traffic cameras, environmental monitoring stations, and smart meters continuously generate structured time-series data for real-time perception of physical phenomena such as road congestion, water quality changes, and energy consumption. At the same time, citizens will express their opinions, emotions, and positions on the same event on social platforms such as Weibo, WeChat public accounts, and forums. These unstructured public opinion information and sensor data are coupled in time and space. For example, during periods of road congestion, the popularity of related topics on social media increases sharply, and during periods of deteriorating domestic water quality, the number of public complaints and suggestions rises rapidly. By integrating sensor data with social media public opinion information, we can not only understand the operating status of the city at a macro level, but also capture public needs and feedback at a micro level, providing more complete decision-making support for emergency warning, traffic management, and environmental regulation. Mainstream multimodal dynamic synchronization databases (RTDBs) focus on high-throughput, low-latency storage and computing of structured time series signals. They enable real-time processing of physical data through subscriptions, publishing, scripts / expressions, and alarms. However, they are unable to directly collect, parse, and associate unstructured text on social media. When associating, storing, and synchronously processing sensor data and public opinion information on the same platform, how to establish spatiotemporal associations between structured physical signals and unstructured text comments at the database level, and how to balance low-latency synchronization and high consistency between social public opinion and sensor data in scenarios with high concurrency writes and queries, and how to support the combination of physical indicators and public sentiment / positions to achieve precise real-time alarms and data push, have become urgent problems that need to be solved by multimodal synchronization databases in smart cities. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a multimodal dynamic synchronization database system for smart cities, which integrates structured sensor time series data with social media emotions and positions in a real-time database, realizes low-latency and high-consistency linkage perception and differentiated and accurate alarms of multimodal data, and improves the sensitivity and decision-making accuracy of smart city environmental monitoring, traffic management and public safety warning.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A multimodal dynamic synchronous database system for smart cities includes a data interface module, an agent module, a subscription and release module, a control command module, a real-time data processing module, a kernel model management module, a cluster and data synchronization module, a plug-in module and a data storage module. The data interface module receives and distributes subscription and release, command, data access and data update requests. The agent module proxies the above requests to the target real-time database and returns the results. The subscription and release module manages channel and subscription mapping, encodes and pushes model data, and performs differentiated subscription of sensor time series data and public_attitude and stance_label. The control command module finely controls analog output and discrete output through lock detection, initial condition verification, value range verification and feedback process. The data processing module aligns sensor signals with public_attitude, fuses public_attitude, stance_label, and the deviation ratio of real-time sensor data according to preset weights to obtain composite indicators and trigger alarms. The core model management module dynamically loads and grayscale releases BERT+CompGCN, stance detection models, and sentiment analysis models, and supports online hot updates and rollbacks. The cluster and data synchronization module manages multiple nodes, synchronizes real-time data and models across nodes, and monitors heartbeats. The plug-in module supports the access of third-party business plug-ins through plug-in management and communication interfaces. The data storage module persistently stores sensor signals, control data, intermediate data, and inference results of public_attitude and stance_label.

[0006] As a further solution of the present invention, the data interface module includes a subscription and publishing interface, a command interface, a data access interface and a data update interface. The subscription and publishing interface creates and maintains a subscription channel for the client, and cancels the corresponding subscription and cleans up unsent messages when the channel fails. It receives the client's single-point subscription, multi-point subscription, group subscription and corresponding cancellation requests, and maintains the subscription mapping relationship, scans the publishing buffer, encodes the model data to be published and pushes it to the corresponding subscription channel, and performs channel cleanup in case of failure. The command interface receives the client's control command and passes it to the kernel data model object for processing, and returns the command execution result to the command sender. The data access interface receives the client's query, update and modification requests, parses the request content and entrusts it to the kernel data model object for execution, and returns the execution result to the client. The data update interface receives the model update data submitted by the client and sends it to the kernel data model object and stores it persistently.

[0007] As a further solution of the present invention, the real-time data processing module includes a script sub-module, an expression sub-module and an alarm sub-module. The script sub-module synchronizes the sensor structured signal with the public_attitude real-time signal data point in terms of timestamp, and sends the synchronization result to the alarm sub-module. The expression sub-module calculates the composite index based on the combination conditions of the sensor threshold, public_attitude and stance_label. The alarm sub-module triggers the alarm based on the composite index and pushes it to the corresponding subscriber through the data interface module.

[0008] As a further solution of the present invention, the process of calculating the composite index by the expression submodule includes:

[0009] Step 1, calculating the deviation ratio: normalizing the difference between the received sensor real-time data and the preset threshold of the corresponding sensor, and calculating the deviation ratio of the sensor real-time data;

[0010] Step 2: Obtain sentiment score: linearly map the data of the public_attitude signal to [0, 1] to obtain the sentiment score of the public_attitude signal;

[0011] Step 3: Obtain the quantitative score of stance_label: Call the sample stance detection model integrated with the social network knowledge graph, output the probability vectors of the three types of stances (favorable, neutral, and opposed) based on the document representation, topic representation, and relationship vector, and fuse the probability vectors of these three types of stances according to the preset weights to obtain the original stance score. The original stance score is linearly mapped to the interval [0,1] to obtain the quantitative score of stance_label;

[0012] Step 4, calculate the composite index: according to the preset weighted fusion deviation ratio, sentiment score and quantitative score, obtain the composite index;

[0013] Step 5: Compare the composite indicator with the alarm threshold: Compare the composite indicator with the alarm threshold. When the composite indicator is greater than the alarm threshold, an alarm is triggered and pushed to the corresponding subscriber through the data interface module.

[0014] It should be noted that in the process of calculating the composite index by the expression submodule, the script submodule first aligns the real-time sensor data and the public_attitude signal loaded in the kernel model management module in advance according to the timestamp to ensure that the three types of data can be calculated in parallel within the same time window; then in step 1, the deviation ratio is calculated by using the pre-configured threshold after alignment to obtain the degree of deviation of the physical signal from the normal range; in step 2, the original sentiment score of public_attitude is linearly mapped to the [0,1] interval to form a sentiment score that can directly participate in the fusion; in step 3, the stance detection service is triggered and the constructed The social network knowledge graph and small sample stance detection model built by the researchers jointly reason on the input text and topic, outputting three types of probabilities: "agree", "neutral", and "oppose", which are then fused and normalized according to the designed weight coefficients to obtain a continuous stance_label quantitative score. In step 4, the deviation ratio, sentiment score, and stance score are weighted and summed according to the preset weights to form a composite indicator. In step 5, the composite indicator is compared with the system-level or subscriber-defined alarm threshold. When the composite indicator exceeds the threshold, an alarm is immediately triggered and pushed to the corresponding subscriber through the data interface module, realizing real-time and linked alarms for the physical state of the sensor and the social public opinion situation.

[0015] As a further solution of the present invention, the core model management module includes a model positioning submodule and a model management submodule. The model positioning submodule is used to dynamically retrieve the BERT+CompGCN sample stance detection model and sentiment analysis model according to the model identifier. The model management submodule is used to load the BERT+CompGCN sample stance detection model and sentiment analysis model, manage the model version, and perform grayscale release, online hot update and rollback.

[0016] As a further solution of the present invention, the control command module executes the following analog output control process:

[0017] Step 11: receiving the analog output control command sent by the client;

[0018] Step 12, retrieving the analog output data point AO pointed to by the command;

[0019] Step 13: If the AO does not exist, the command execution status is updated to not support this type of control and control failure is returned;

[0020] Step 14: Check the control prohibition status of the device model corresponding to AO. If prohibited, update the execution status to prohibit control and return;

[0021] Step 15: Verify the initial execution conditions of the command. If the verification fails / times out, update the command status to "initial conditions not met / timed out" and return;

[0022] Step 16: Verify whether the command value exceeds the valid range. If so, update the execution status to "value exceeds limit" and return;

[0023] Step 17: Send the control command to the field execution system FES. If the sending fails or times out, update the execution status to sending failure / timeout and return;

[0024] Step 18: Receive the execution result returned by FES. If the execution result fails / times out, update the execution status to result failure / timeout and return;

[0025] Step 19: After the checks and issuances from steps 11 to 18 are successful, the execution status is updated to success and returned.

[0026] It should be noted that the model identifier of the model positioning submodule is automatically matched and obtained by the unique identifier maintained in the model registry (including model name, version number, social platform tag, etc.), and can be dynamically expanded through the configuration center; when loading the BERT+CompGCN stance detection model and sentiment analysis model, the model management submodule adopts distributed scheduling and containerized deployment to implement the grayscale release strategy - switching to the new version in batches according to the preset traffic ratio; online hot updates ensure zero downtime through dual-container switching; when the model performance monitoring indicators (such as latency, accuracy) or security audit alarms trigger the preset thresholds, they automatically roll back to the previous stable version, and write the rollback log to the operation and maintenance audit table. In the analog output control process, the AO location mapping table supports unified identification of multi-vendor, multi-protocol devices. Lockout detection and initial execution condition verification are performed according to the configured lock flag and condition timeout period, and the value range is verified according to the preset minimum and maximum values. When the control command is sent to the FES, asynchronous RPC communication is used to receive ACK / NACK receipts, and the communication timeout period is the current corresponding time. When the reception timeout or NACK is returned, the execution status is updated according to steps 17 and 18 and the subsequent process is terminated to ensure the real-time, reliability, and monitorability of the command.

[0027] As a further solution of the present invention, the control command module executes the following digital control process:

[0028] Step 21: receiving a discrete output control command sent by the client;

[0029] Step 22, retrieve the discrete output data point DO pointed to by the command;

[0030] Step 23: If DO does not exist, the command execution status is updated to not support this type of control and control failure is returned;

[0031] Step 24: Check the control prohibition status of the device model corresponding to the DO. If prohibited, update the execution status to prohibit control and return;

[0032] Step 25: Verify the initial execution conditions of the command. If the verification fails / times out, update the command status to "initial conditions not met / timed out" and return;

[0033] Step 26: Send the control command to the field execution unit FEP. If the sending fails / times out, update the execution status to sending failure / timeout and return;

[0034] Step 27: After the checks and issuances described in steps 21 to 16 are successful, the execution status is updated to success and returned.

[0035] It should be noted that the identification of discrete output data points (DOs) is managed by a unified device mapping table, which supports multiple communication protocols such as OPC UA and Modbus. Lockout detection and initial execution condition verification are completed based on the configured lock flag and timeout period. In step 26, the control command is sent to the field execution unit (FEP) via asynchronous RPC or message queue, and waits for an ACK / NACK receipt within the timeout period. If a NACK is received or a timeout occurs, the execution status is updated to "send failure / timeout." The entire process is strictly executed within a time limit by the state machine to ensure the real-time, reliability, and traceability of the digital control commands.

[0036] As a further solution of the present invention, the process of data storage by the data storage module is as follows: after writing the data to be stored into the data storage cache queue, the data storage module starts a batch storage thread and performs persistence operations on the data in the data storage cache queue. If the storage is successful, it is deleted from the data storage cache queue. If the storage fails, it is retried three times and removed from the data storage cache queue after still failing. When deleting records, the data storage operation is directly called to delete the corresponding records from the working database.

[0037] As a further solution of the present invention, the plug-in module includes a plug-in manager and a plug-in communication interface. When the plug-in module loads and starts the plug-in, the plug-in information table is loaded from the configuration file when the system starts or runs to check the activation status of the plug-in. If activated, the plug-in is loaded and started. The plug-in is dynamically stopped, unloaded or reloaded through control commands. The plug-in communication interface supports the plug-in to call the model operation interface / data interface in the system, execute control commands / send and receive messages, and return the processing results to the system through the model operation interface / data interface in the system.

[0038] As a further solution of the present invention, the cluster and data synchronization module includes a cluster management submodule, a synchronization submodule and a heartbeat submodule. The cluster management submodule is responsible for the registration, discovery and fault elimination of cluster nodes. The synchronization submodule is responsible for the cross-node replication and consistency maintenance of system model data and status data. The heartbeat submodule periodically sends heartbeat signals to the cluster management submodule. If the node heartbeat times out, the node elimination process is triggered.

[0039] The technical effects of the multimodal dynamic synchronization database system for smart cities of the present invention are as follows:

[0040] The present invention realizes the unified storage and retrieval of sensor structured time series data and social media unstructured public opinion data by hierarchically managing signal data, control data and intermediate data in the data storage module; accesses public_attitude and stance_label in the data interface and proxy module to map the text sentiment and topic stance reasoning results to the database; the real-time data processing module aligns the three types of signals on the timestamp with the help of the script submodule, and the expression submodule calculates the composite index by fusing sensor bias, sentiment score and stance score according to adjustable weights, and realizes push and alarm through differentiated conditions supported by the subscription and publishing module, thereby realizing accurate linkage perception and early warning of urban physical events and public attitudes; the kernel model management module provides loading, grayscale publishing and hot update of the BERT+CompGCN model, and the cluster and data synchronization module ensures multi-node fault tolerance; comprehensively improves the situational awareness decision support accuracy and emergency response efficiency of smart cities in the fields of environmental monitoring, traffic management and public safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a system block diagram of the present invention;

[0042] Figure 2 This is a data access processing flow chart of the present invention;

[0043] Figure 3 This is a timing diagram of data request processing of the present invention;

[0044] Figure 4 This is a data update processing flow chart of the present invention;

[0045] Figure 5 A data receiving flow chart for data updating of the present invention;

[0046] Figure 6 This is a timing diagram of receiving a FES report message according to the present invention;

[0047] Figure 7 A timing diagram of subscribing to a pre-release service of the present invention;

[0048] Figure 8 A flow chart showing the control command module of the present invention executing simulation output;

[0049] Figure 9 This is a timing diagram of the control command module of the present invention executing analog output;

[0050] Figure 10 This is a flow chart of digital quantity control executed by the control command module of the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the content described is only part of the present invention, not the entire invention. Based on the content of the present invention, all other technical solutions obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0052] like Figure 1 As shown, the present invention proposes a multimodal dynamic synchronous database system for smart cities, including a data interface module, an agent module, a subscription and publication module, a control command module, a real-time data processing module, a kernel model management module, a cluster and data synchronization module, a plug-in module and a data storage module. The data interface module receives and distributes subscription and publication, commands, data access and data update requests. The agent module proxies the above requests to the target real-time database and returns the results. The subscription and publication module manages the channel and subscription mapping, encodes and pushes the model data, and performs differentiated subscription of sensor time series data and public_attitude and stance_label. The control command module finely controls the analog output and discrete output through lock detection, initial condition verification, numerical range verification and feedback process. The real-time data processing module aligns sensor signals with public_attitude, fuses public_attitude, stance_label, and the deviation ratio of real-time sensor data according to preset weights to obtain composite indicators and trigger alarms. The kernel model management module dynamically loads and grayscale releases BERT+CompGCN, stance detection model, and sentiment analysis model, and supports online hot updates and rollbacks. The cluster and data synchronization module performs multi-node management, cross-node synchronization of real-time data and models, and heartbeat monitoring. The plug-in module supports the access of third-party business plug-ins through plug-in management and communication interfaces. The data storage module persistently stores sensor signals, control data, intermediate data, and inference results of public_attitude and stance_label.

[0053] By aligning, weightedly fusing, and differentially subscribing sensor structured time series data with social media sentiment signals and topic stance signals on the same platform in real time, the system can achieve collaborative perception and precise linkage alerts of urban physical events and public attitudes without sacrificing high throughput, low latency, and strong consistency, significantly improving the accuracy of situational warnings and emergency response efficiency in smart cities in scenarios such as environmental monitoring, traffic management, and public safety.

[0054] It should be noted that the data interface module includes a subscription and publishing interface, a command interface, a data access interface and a data update interface. The subscription and publishing interface creates and maintains a subscription channel for the client, and cancels the corresponding subscription and cleans up unsent messages when the channel fails. It receives the client's single-point subscription, multi-point subscription, group subscription and corresponding cancellation requests, maintains the subscription mapping relationship, scans the publishing buffer, encodes the model data to be published and pushes it to the corresponding subscription channel, and performs channel cleanup in case of failure. The command interface receives the client's control command and passes it to the kernel data model object for processing, and returns the command execution result to the command sender. The data access interface receives the client's query, update and modification requests, parses the request content and entrusts it to the kernel data model object for execution, and returns the execution result to the client. The data update interface receives the model update data submitted by the client and sends it to the kernel data model object and stores it persistently.

[0055] The data interface module implements the data interaction between the system proposed by the present invention and the database client. It sends the client's data request to the corresponding module and returns the request result to the client. Figure 2 As shown in the figure, when the client needs to query, update, or modify the properties of the device model managed by the real-time database in the system, or send a device control command to the real-time database, the data access interface of the real-time database receives these requests from the client, then parses the request content and searches for the requested data points (device points, input and output signal points, etc.) in the kernel data model of the real-time database. If the requested target data model does not exist, the request fails and a failure result is returned to the data accessor. Otherwise, the requested operation is performed on the target data model and the operation result is returned to the data accessor. The client's data request processing sequence diagram is shown in the figure. Figure 3 shown.

[0056] like Figures 4 to 6 As shown, in the data interface module, the data update process is divided into three steps: message reception, message parsing, and model object update. This is how the proposed system receives and processes device status data reported by the FES. To improve the proposed system's ability to handle large amounts of device status data reported by the FES in extreme situations such as avalanches and failures, the proposed system adopts a three-step strategy for processing FES data: asynchronous reception (level 1 cache), asynchronous parsing (level 2 cache), and asynchronous update.

[0057] In the data interface module, the subscription and publishing results realize the push of data between the system proposed by the present invention and the client. The client registers the model data of interest to the real-time database. After the model data changes, the real-time database will push these changes and the model data of interest to the interested client. The subscription and publishing interface receives the data subscription of the client object model (single-point subscription, multi-point subscription, group subscription and other subscription methods), receives the client's cancellation of subscription (single-point cancellation, multi-point cancellation, group cancellation and other cancellation methods), and maintains the client's subscription information (subscription content maintenance, subscription channel maintenance, etc.). Figure 7 As shown in the figure, the subscription and publishing services are divided into three parts: subscription channel management, data subscription and subscription cancellation, and data publishing. Subscription channel management is responsible for creating and maintaining subscription channels. When a subscription channel fails, it cancels all subscriptions corresponding to that channel, clears all unpublished information, and then removes the data from the relevant units of subscription and publishing management. The data subscription and subscription cancellation services process data subscription and cancellation requests sent by the subscribing client. The data publishing service is responsible for publishing data (the subscription channel corresponding to the channel) to subscribers.

[0058] It should be noted that the real-time data processing module includes a script sub-module, an expression sub-module and an alarm sub-module. The script sub-module synchronizes the sensor structured signal with the public_attitude real-time signal data point in terms of timestamp and sends the synchronization result to the alarm sub-module. The expression sub-module calculates the composite indicator based on the combination of sensor threshold, public_attitude and stance_label. The alarm sub-module triggers the alarm based on the composite indicator and pushes it to the corresponding subscriber through the data interface module.

[0059] The script sub-module precisely aligns the sensor timing signal with the public_attitude signal, and the expression sub-module fuses physical deviation, sentiment score, and stance score to calculate a composite risk indicator. Finally, the alarm sub-module triggers and pushes an alarm based on this indicator. This enables real-time collaborative monitoring of urban operating status and public opinion attitudes, significantly improving the timeliness and accuracy of alarms, reducing false alarms and missed alarms, and providing refined and perceptible data support for smart city environmental monitoring, traffic scheduling, and public safety.

[0060] It should be noted that the process of calculating the composite index by the expression submodule includes:

[0061] Step 1, calculating the deviation ratio: normalizing the difference between the received sensor real-time data and the preset threshold of the corresponding sensor, and calculating the deviation ratio of the sensor real-time data;

[0062] Step 2: Obtain sentiment score: linearly map the data of the public_attitude signal to [0, 1] to obtain the sentiment score of the public_attitude signal;

[0063] Step 3: Obtain the quantitative score of stance_label: Call the sample stance detection model integrated with the social network knowledge graph, output the probability vectors of the three types of stances (favorable, neutral, and opposed) based on the document representation, topic representation, and relationship vector, and fuse the probability vectors of these three types of stances according to the preset weights to obtain the original stance score. The original stance score is linearly mapped to the interval [0,1] to obtain the quantitative score of stance_label;

[0064] Step 4, calculate the composite index: according to the preset weighted fusion deviation ratio, sentiment score and quantitative score, obtain the composite index;

[0065] Step 5: Compare the composite indicator with the alarm threshold: Compare the composite indicator with the alarm threshold. When the composite indicator is greater than the alarm threshold, an alarm is triggered and pushed to the corresponding subscriber through the data interface module.

[0066] In step 1, the original values ​​of physical quantities (such as temperature, flow, concentration, etc.) have different magnitudes, and direct comparison is meaningless. They need to be normalized with normal thresholds to obtain standardized values ​​reflecting the degree of deviation of the equipment or environment. The numerical deviation ratio is used to intuitively present the degree of abnormality of the physical state and provide a comparable basis for subsequent fusion; in step 2, the distribution of public sentiment on social media is often extremely volatile. Linearly mapping the original sentiment value to the interval [0,1] is compatible with the value range of the physical deviation ratio, simplifying the fusion calculation, so that the sentiment signal can participate in the calculation of composite indicators at the same level as the physical data, and reflect the public's positive and negative emotions towards the current event in real time; in step 3, the simple three-category "agree / neutral / oppose" is difficult to reflect the model's confidence in each position. Through probability-weighted fusion and normalization, a continuous position score is obtained, which can capture subtle differences in positions and provide a more refined quantitative output of public opinion tendencies. , so that the alarm can not only respond to the intensity of public sentiment, but also distinguish the atmosphere of public opinion that clearly supports or opposes it; in step 4, a single dimension is often not enough to comprehensively judge the risk of urban operation. The physical deviation, emotional score and stance score are integrated with adjustable weights. Different factors can be focused on according to different scenarios to generate a unified indicator reflecting the combined impact of "physical anomaly × public sentiment × stance tendency", so that the system has the dual advantages of increased sensitivity to multi-dimensional risk signals and false alarm suppression; in step 5, combined with business or subscriber-defined thresholds, the composite indicators are monitored in real time to ensure that the alarm is triggered only when all multimodal signals meet the warning conditions. Through the "multi-factor" trigger mechanism, false alarms caused by fluctuations in a single data are significantly reduced, while the ability to timely warn of potential emergencies is enhanced, providing smart cities with more accurate and efficient decision-making support in scenarios such as environmental monitoring, traffic control and public safety.

[0067] It should be noted that the core model management module includes a model positioning submodule and a model management submodule. The model positioning submodule is used to dynamically retrieve the BERT+CompGCN sample stance detection model and sentiment analysis model based on the model identifier. The model management submodule is used to load the BERT+CompGCN sample stance detection model and sentiment analysis model, manage model versions, and perform grayscale releases, online hot updates, and rollbacks.

[0068] Doing so can achieve seamless online switching and controllable rollback of sentiment analysis and stance detection models, and complete model upgrades without affecting the normal read, write, and subscription and publishing services of the database; gradually increase the volume through grayscale releases to verify the accuracy of the new version, greatly reducing the risk of model updates; use hot updates and dual-container switching to ensure zero downtime and improve the overall availability of the system; and use version management and performance monitoring to achieve model performance backtracking and rapid problem location, enhancing the stability, flexibility and security of the smart city platform in multimodal public opinion analysis scenarios.

[0069] It should be noted that if Figures 8 and 9 As shown in the figure, the control command module executes the following analog output control process:

[0070] Step 11: receiving the analog output control command sent by the client;

[0071] Step 12, retrieving the analog output data point AO pointed to by the command;

[0072] Step 13: If the AO does not exist, the command execution status is updated to not support this type of control and control failure is returned;

[0073] Step 14: Check the control prohibition status of the device model corresponding to AO. If prohibited, update the execution status to prohibit control and return;

[0074] Step 15: Verify the initial execution conditions of the command. If the verification fails / times out, update the command status to "initial conditions not met / timed out" and return;

[0075] Step 16: Verify whether the command value exceeds the valid range. If so, update the execution status to "value exceeds limit" and return;

[0076] Step 17: Send the control command to the field execution system FES. If the sending fails or times out, update the execution status to sending failure / timeout and return;

[0077] Step 18: Receive the execution result returned by FES. If the execution result fails / times out, update the execution status to result failure / timeout and return;

[0078] Step 19: After the checks and issuances from steps 11 to 18 are successful, the execution status is updated to success and returned.

[0079] Doing so can incorporate the entire life cycle of analog output control commands into multi-stage refined management and control, including lockout detection, initial condition verification, numerical range verification, asynchronous issuance and receipt processing. Through status updates and abnormal branch processing at each step, the remote control signal can be fully traceable, monitored and responded to quickly, which significantly improves the safety, reliability and real-time performance of equipment control, reduces the risk of misoperation and communication failure, and thus ensures the stable execution of key control instructions in smart city operations.

[0080] It should be noted that if Figure 10 As shown in the figure, the control command module executes the following digital control process:

[0081] Step 21: receiving a discrete output control command sent by the client;

[0082] Step 22, retrieve the discrete output data point DO pointed to by the command;

[0083] Step 23: If DO does not exist, the command execution status is updated to not support this type of control and control failure is returned;

[0084] Step 24: Check the control prohibition status of the device model corresponding to the DO. If prohibited, update the execution status to prohibit control and return;

[0085] Step 25: Verify the initial execution conditions of the command. If the verification fails / times out, update the command status to "initial conditions not met / timed out" and return;

[0086] Step 26: Send the control command to the field execution unit FEP. If the sending fails / times out, update the execution status to sending failure / timeout and return;

[0087] Step 27: After the checks and issuances described in steps 21 to 16 are successful, the execution status is updated to success and returned.

[0088] This approach integrates the security, reliability, and real-time nature of discrete output control commands into the full lifecycle management of these commands by sequentially verifying DO existence, detecting control inhibit states, verifying initial conditions and value ranges, and asynchronously issuing and processing receipts. This ensures traceability and precise execution of discrete signals from field devices, significantly reducing the risk of control failures caused by communication anomalies and misoperation. Traditional alarms that rely solely on physical thresholds are susceptible to sensor noise or transient fluctuations. By jointly evaluating public sentiment (public_attitude) and stance (stance_label) with physical deviations, it is possible to filter out noisy alarms that indicate "physical anomalies with no social concern" or "public sentiment fluctuations with normal physical conditions," significantly reducing false alarm rates. Public events often ferment on social media before physical indicators show significant deviations. By integrating sentiment and stance signals, the system can detect signs of public concern at the earliest stages of physical anomalies, triggering earlier composite indicator threshold violations and issuing alerts in advance. Through a configurable weighting mechanism, different application scenarios (such as traffic management, environmental monitoring, or public safety) can focus on physical deviations, public sentiment, or stance tendencies, and flexibly adjust composite indicator calculation strategies to meet diverse business needs. Alarms not only reflect the "equipment or environmental status," but also carry information on "public attention" and "social attitudes," enabling the operator to simultaneously analyze public opinion trends while responding to physical events, providing a more socially informed decision-making basis. After multimodal fusion, the impact of failure or anomalies in a single data source on alarms is dispersed, improving the system's fault tolerance for heterogeneous data. When new sentiment or stance models are launched, they can be seamlessly integrated into the composite indicator calculation framework to ensure the long-term maintainability of the platform.

[0089] It should be noted that the data storage process of the data storage module is as follows: after writing the data to be stored into the data storage cache queue, the data storage module starts the batch storage thread and performs persistence operations on the data in the data storage cache queue. If the storage is successful, it will be deleted from the data storage cache queue. If the storage fails, it will be retried three times. If it still fails, it will be removed from the data storage cache queue. When deleting records, the data storage operation is directly called to delete the corresponding records from the working database.

[0090] Doing so can ensure that the key composite indicators composed of the three conditions of deviation ratio, sentiment score and stance score are not lost in extreme network jitter or storage failure, ensuring the integrity and traceability of multimodal alarm signals; through batch asynchronous writing and retry mechanism, it not only takes into account the performance and real-time nature of persistent operations, but also automatically cleans up after three retries to avoid system blockage due to single point storage failure; at the same time, the direct call of the deletion operation ensures that historical composite indicators can be deleted in time, meeting the strict requirements of smart city public opinion and physical status linkage analysis on data consistency, reliability and auditability.

[0091] It should be noted that the plug-in module includes a plug-in manager and a plug-in communication interface. When the plug-in module loads and starts the plug-in, the plug-in information table is loaded from the configuration file when the system starts or runs to check the activation status of the plug-in. If activated, the plug-in is loaded and started. The plug-in is dynamically stopped, unloaded or reloaded through control commands. The plug-in communication interface supports the plug-in to call the model operation interface / data interface in the system, execute control commands / send and receive messages, and return the processing results to the system through the model operation interface / data interface in the system.

[0092] This enables the system to dynamically load, start, stop or uninstall third-party plug-ins during operation, and multimodal analysis, control or data access functions can be expanded or replaced on demand without downtime for upgrades; plug-ins call system models and data services through a unified communication interface, ensuring loosely coupled integration of the core database with external algorithms and business modules, thereby greatly improving the system's access speed and flexibility to new sentiment analysis, stance detection models or new data sources, reducing maintenance costs, and ensuring that the smart city multimodal situational awareness platform can continue to evolve and quickly respond to changing business needs.

[0093] It should be noted that the cluster and data synchronization module includes a cluster management submodule, a synchronization submodule and a heartbeat submodule. The cluster management submodule is responsible for the registration, discovery and fault elimination of cluster nodes. The synchronization submodule is responsible for the cross-node replication and consistency maintenance of system model data and status data. The heartbeat submodule regularly sends heartbeat signals to the cluster management submodule. If the node heartbeat times out, the node elimination process is triggered.

[0094] Through the node registration and fault elimination of the cluster management submodule, any single point of failure will not lead to the interruption of public opinion or physical data processing, ensuring the continuous online service of sentiment analysis, stance detection and composite alarm services; the synchronization submodule replicates model parameters, operating status and composite indicators between nodes, ensuring that no matter which node the client connects to, it can obtain the same public_attitude, stance_label and composite alarm results; the heartbeat submodule monitors node health in real time, and combined with the automatic elimination and addition of node mechanisms, it can dynamically expand and shrink capacity according to data volume and computing pressure to meet the high concurrency processing requirements of multi-source and multi-modal data in smart cities; when the node heartbeat times out, the elimination process is immediately triggered and tasks are reallocated to avoid performance degradation of the model management or real-time data processing modules due to partial node failures, thereby comprehensively improving the system's rapid response capability and reliability to emergencies.

[0095] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0096] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimodal dynamic synchronization database system for smart cities, characterized in that: It includes data interface module, proxy module, subscription and release module, control command module, real-time data processing module, kernel model management module, cluster and data synchronization module, plug-in module and data storage module. The data interface module receives and distributes subscription and release, command, data access and data update requests. The proxy module proxies the above requests to the target real-time database and returns the results. The subscription and release module manages channel and subscription mapping, encodes and pushes model data, and performs differentiated subscription of sensor time series data and public_attitude and stance_label. The control command module finely controls analog output and discrete output through lock detection, initial condition verification, numerical range verification and feedback process. The real-time data processing module aligns sensor signals. The kernel model management module dynamically loads and releases BERT+CompGCN, stance detection model, and sentiment analysis model in grayscale, and supports online hot updates and rollbacks. The cluster and data synchronization module manages multiple nodes, synchronizes real-time data and models across nodes, and monitors heartbeats. The plug-in module supports the access of third-party business plug-ins through plug-in management and communication interfaces. The data storage module persistently stores sensor signals, control data, intermediate data, and inference results of public_attitude and stance_label.

2. A multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The data interface module includes a subscription and publishing interface, a command interface, a data access interface, and a data update interface. The subscription and publishing interface creates and maintains a subscription channel for the client, cancels the corresponding subscription, and cleans up unsent messages when the channel fails. It receives the client's single-point subscription, multi-point subscription, group subscription, and corresponding cancellation requests, maintains the subscription mapping relationship, scans the publishing buffer, encodes the model data to be published, and pushes it to the corresponding subscription channel. It performs channel cleanup when a failure occurs. The command interface receives the client's control command, passes it to the kernel data model object for processing, and returns the command execution result to the command sender. The data access interface receives the client's query, update, and modification requests, parses the request content, and entrusts it to the kernel data model object for execution, and returns the execution result to the client. The data update interface receives the model update data submitted by the client and sends it to the kernel data model object for persistent storage.

3. A multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The real-time data processing module includes a script submodule, an expression submodule, and an alarm submodule. The script submodule synchronizes the sensor structured signal with the public_attitude real-time signal data point in terms of timestamps and sends the synchronization results to the alarm submodule. The expression submodule calculates composite indicators based on the combination of sensor thresholds, public_attitude, and stance_label. The alarm submodule triggers alarms based on the composite indicators and pushes them to the corresponding subscribers through the data interface module.

4. A multimodal dynamic synchronization database system for smart cities according to claim 3, characterized in that: The process of calculating composite indicators by the expression submodule includes: Step 1, calculating the deviation ratio: normalizing the difference between the received sensor real-time data and the preset threshold of the corresponding sensor, and calculating the deviation ratio of the sensor real-time data; Step 2: Obtain sentiment score: linearly map the data of the public_attitude signal to [0, 1] to obtain the sentiment score of the public_attitude signal; Step 3: Obtain the quantitative score of stance_label: Call the sample stance detection model integrated with the social network knowledge graph, output the probability vectors of the three types of stances (favorable, neutral, and opposed) based on the document representation, topic representation, and relationship vector, and fuse the probability vectors of these three types of stances according to the preset weights to obtain the original stance score. The original stance score is linearly mapped to the interval [0,1] to obtain the quantitative score of stance_label; Step 4, calculate the composite index: according to the preset weighted fusion deviation ratio, sentiment score and quantitative score, obtain the composite index; Step 5: Compare the composite indicator with the alarm threshold: Compare the composite indicator with the alarm threshold. When the composite indicator is greater than the alarm threshold, an alarm is triggered and pushed to the corresponding subscriber through the data interface module.

5. The multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The core model management module includes a model positioning submodule and a model management submodule. The model positioning submodule is used to dynamically retrieve the BERT+CompGCN sample stance detection model and sentiment analysis model based on the model identifier. The model management submodule is used to load the BERT+CompGCN sample stance detection model and sentiment analysis model, manage model versions, and perform grayscale releases, online hot updates, and rollbacks.

6. A multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The control command module executes the following analog output control process: Step 11: receiving the analog output control command sent by the client; Step 12, retrieving the analog output data point AO pointed to by the command; Step 13: If the AO does not exist, the command execution status is updated to not support this type of control and control failure is returned; Step 14: Check the control prohibition status of the device model corresponding to AO. If prohibited, update the execution status to prohibit control and return; Step 15: Verify the initial execution conditions of the command. If the verification fails / times out, update the command status to "initial conditions not met / timed out" and return; Step 16: Verify whether the command value exceeds the valid range. If so, update the execution status to "value exceeds limit" and return; Step 17: Send the control command to the field execution system FES. If the sending fails or times out, update the execution status to sending failure / timeout and return; Step 18: Receive the execution result returned by FES. If the execution result fails / times out, update the execution status to result failure / timeout and return; Step 19: After the checks and issuances from steps 11 to 18 are successful, the execution status is updated to success and returned.

7. A multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The control command module executes the following digital control process: Step 21: receiving a discrete output control command sent by the client; Step 22, retrieve the discrete output data point DO pointed to by the command; Step 23: If DO does not exist, the command execution status is updated to not support this type of control and control failure is returned; Step 24: Check the control prohibition status of the device model corresponding to the DO. If prohibited, update the execution status to prohibit control and return; Step 25: Verify the initial execution conditions of the command. If the verification fails / times out, update the command status to "initial conditions not met / timed out" and return; Step 26: Send the control command to the field execution unit FEP. If the sending fails / times out, update the execution status to sending failure / timeout and return; Step 27: After the checks and issuances described in steps 21 to 16 are successful, the execution status is updated to success and returned.

8. The multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The data storage process of the data storage module is as follows: after writing the data to be stored into the data storage cache queue, the data storage module starts the batch storage thread and performs persistence operations on the data in the data storage cache queue. If the storage is successful, it will be deleted from the data storage cache queue. If the storage fails, it will be retried three times and removed from the data storage cache queue if it still fails. When deleting records, the data storage operation is directly called to delete the corresponding records from the working database.

9. The multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The plug-in module includes a plug-in manager and a plug-in communication interface. When the plug-in module loads and starts the plug-in, the plug-in information table is loaded from the configuration file when the system starts or runs to check the activation status of the plug-in. If it is activated, the plug-in is loaded and started. The plug-in is dynamically stopped, unloaded or reloaded through control commands. The plug-in communication interface supports the plug-in to call the model operation interface / data interface in the system, execute control commands / send and receive messages, and return the processing results to the system through the model operation interface / data interface in the system.

10. The multimodal dynamic synchronization database system for smart cities according to claim 1, characterized in that: The cluster and data synchronization module includes a cluster management submodule, a synchronization submodule and a heartbeat submodule. The cluster management submodule is responsible for the registration, discovery and fault elimination of cluster nodes. The synchronization submodule is responsible for the cross-node replication and consistency maintenance of system model data and status data. The heartbeat submodule regularly sends heartbeat signals to the cluster management submodule. If the node heartbeat times out, the node elimination process is triggered.

Images