Wharf safety intelligent monitoring method, device and equipment based on digital twinning and medium

Abstract

The invention relates to a wharf safety intelligent monitoring method, device and equipment based on digital twinning and a medium. The method comprises the following steps: firstly, carrying out space-time alignment fusion on multi-source heterogeneous original data of a wharf site to generate a dynamic digital twinborn scene; extracting and matching entity relationships and events based on the dynamic digital twinborn scene and a pre-constructed wharf operation knowledge graph, and generating a dynamic knowledge situation sub-graph; performing graph neural network coding processing on the dynamic knowledge situation sub-graph to obtain coding features of nodes in the graph; risk and conflict analysis is carried out based on the coding features, and a current risk list and a potential conflict prediction list are generated; and generating security alarm information based on the list. By adopting the method, safety monitoring can be improved from a perception level based on simple rule matching to a cognition level fusing semantic understanding and context association reasoning, so that false alarm and missing alarm are remarkably reduced, and accurate early warning of potential risks in a complex working environment is realized.

Description

Technical Field

[0001] This invention belongs to the field of intelligent safety monitoring technology, and in particular relates to a method, device, equipment and medium for intelligent safety monitoring of docks based on digital twins. Background Technology

[0002] With the continuous improvement of port automation and intelligence, the terminal operation environment is becoming increasingly complex, with dynamic interactions between various elements such as people, automated guided vehicles (IGVs), and gantry cranes, which places higher demands on production safety monitoring. Currently, terminal safety monitoring mainly relies on widely deployed video surveillance systems and various sensors, with core technologies largely based on deep learning models of computer vision, enabling real-time detection and tracking of independent targets such as "people," "vehicles," and "containers."

[0003] In existing technological practices, the commonly used approach is to acquire live video streams using sensors such as cameras, identify targets and their locations within a specific area using pre-trained target detection algorithms, and pre-set a series of static rules, such as "personnel are prohibited from entering the driving lane." When the system detects that a target's behavior triggers a rule (e.g., the person's bounding box overlaps with the lane area), an alarm is generated. This approach is essentially a matching judgment based on visual perception and simple spatial rules.

[0004] However, the above approach has significant limitations. Because the system lacks deep semantic understanding and contextual reasoning capabilities regarding the operational scenario, it cannot effectively distinguish between behaviors that appear similar but have vastly different meanings. For example, the system cannot determine whether a person approaching the IGV is performing compliant vehicle maintenance or engaging in dangerous path intrusion; these two situations may be extremely similar at the pixel level, but their risk levels are completely different. This results in a persistently high false alarm rate for existing monitoring systems, while also posing a risk of missed detections for potential conflicts that truly rely on complex contexts (such as equipment status, task instructions, and personnel identity), making it difficult to achieve intelligent safety early warning from "perception" to "cognition." Summary of the Invention

[0005] Therefore, it is necessary to provide a digital twin-based intelligent monitoring method, device, equipment, and medium for port safety that can elevate safety monitoring from a perception level based on simple rule matching to a cognitive level that integrates semantic understanding and contextual reasoning, in order to address the aforementioned technical problems.

[0006] Firstly, this application provides a digital twin-based intelligent monitoring method for port safety, comprising:

[0007] S1. Based on the multi-source heterogeneous raw data from the dock site, perform spatiotemporal alignment and fusion to generate a dynamic digital twin scene;

[0008] S2. Extract and match entity relationships and events based on dynamic digital twin scenarios and pre-built dock operation knowledge graphs to generate dynamic knowledge situation subgraphs.

[0009] S3. Perform graph neural network encoding on the dynamic knowledge situation subgraph to obtain the encoding features of the nodes in the graph;

[0010] S4. Based on the coding characteristics of the nodes in the graph, perform risk and conflict analysis to generate a current risk list and a potential conflict prediction list.

[0011] S5. Generate and process security alarm information based on the current risk list and the potential conflict prediction list.

[0012] In one embodiment, S2 includes:

[0013] S21. Based on the real-time state attributes of the twins in the dynamic digital twin scenario, perform spatiotemporal interaction measurement calculations to obtain the distance and proximity duration between the twins;

[0014] S22. Based on distance and proximity duration, calculate the probability of relationship existence using a preset probability calculation model to obtain the probability value corresponding to the dynamic relationship triplet; the probability value is calculated using the following formula:

[0015]

[0016] in, This is a probability value. For distance, For the nearest duration, The preset distance threshold, The preset minimum duration threshold, This is the adjustment coefficient;

[0017] S23. Based on real-time task information and equipment status data from the terminal operation management system, perform semantic consistency verification on the dynamic relationship triples to obtain the verified dynamic relationship triples and conflict event nodes.

[0018] S24. Perform graph structure combination processing on the verified dynamic relation triples, conflict event nodes, and static entities and relations matched from the dock operation knowledge graph to generate a dynamic knowledge situation subgraph.

[0019] In one embodiment, S3 includes:

[0020] S31. Initialize the feature vectors based on the type and attribute information of the nodes in the dynamic knowledge situation subgraph to obtain the initial feature vectors of each node.

[0021] S32. Based on the initial feature vector and the topological structure and edge relationship type of the dynamic knowledge situation subgraph, perform relation-aware message aggregation iterative processing to obtain the iterated node feature vector;

[0022] S33. Based on the iterated node feature vectors, feature output is performed to obtain the encoded features of the nodes in the graph.

[0023] In one embodiment, S4 includes:

[0024] S41. Based on the encoding features of the entity nodes in the graph, the risk probability distribution is calculated using a classifier to obtain the probability distribution of each entity node corresponding to different risk categories.

[0025] S42. Based on the probability distribution and the preset judgment threshold, identify high-risk entities and generate the current risk list;

[0026] S43. Based on the encoding features of any two entity nodes, a conflict relationship prediction is performed using a relationship classifier to obtain the conflict relationship prediction results between entity pairs.

[0027] S44. Based on the conflict relationship prediction results, assemble the conflict records to generate a potential conflict prediction list.

[0028] In one embodiment, prior to S44, the following is also included:

[0029] S45. Based on multiple dynamic knowledge situation subgraphs in historical time series, the future state is predicted by a spatiotemporal graph neural network model to obtain the predicted state of each entity node at a specified future time.

[0030] S46. Based on the predicted state of each entity node at a future specified time, identify potential collisions and generate a set of potential future conflicts.

[0031] S47. Calculate the time-to-collision value based on the set of future potential conflicts to obtain the time-to-collision value corresponding to each future potential conflict; the time-to-collision value is the time interval from the current time to the predicted time of the collision.

[0032] S48. Based on the time to collision value, perform risk level quantification to generate quantified future conflict information.

[0033] In one embodiment, it further includes:

[0034] S51. Based on the risk types and entity identifiers in the current risk list and the potential conflict prediction list, perform matching query processing from the pre-set contingency plan rule base to obtain the initial set of intervention measures.

[0035] S52. Based on the time-to-collision value in the potential conflict prediction list, the initial set of intervention measures is filtered for response time feasibility to obtain a subset of feasible intervention measures;

[0036] S53. Based on the preset work efficiency evaluation rules, conduct multi-objective utility evaluation on a subset of feasible intervention measures and generate work adjustment suggestions.

[0037] In one embodiment, it further includes:

[0038] S61. Based on the feedback annotation data after historical alarm handling, and the dynamic digital twin scene snapshot and dynamic knowledge situation subgraph corresponding to the historical moment of the feedback annotation data, a dataset is constructed to generate a model training and knowledge mining dataset.

[0039] S62. Based on the model training and knowledge mining dataset, perform parameter optimization on the graph neural network model to obtain the updated graph neural network model parameters.

[0040] S63. Based on the analysis results of new patterns in the model training and knowledge mining dataset, the pre-constructed wharf operation knowledge graph is structurally expanded to obtain an updated wharf operation knowledge graph.

[0041] Secondly, this application also provides a digital twin-based intelligent monitoring device for port safety, comprising:

[0042] The spatiotemporal alignment and fusion module is used to perform spatiotemporal alignment and fusion based on multi-source heterogeneous raw data from the dock site to generate a dynamic digital twin scene.

[0043] The knowledge situation extraction module is used to extract and match entity relationships and events based on dynamic digital twin scenarios and pre-built dock operation knowledge graphs, and generate dynamic knowledge situation subgraphs.

[0044] The graph neural network encoding module is used to perform graph neural network encoding processing on the dynamic knowledge situation subgraph to obtain the encoded features of the nodes in the graph;

[0045] The risk and conflict analysis module is used to perform risk and conflict analysis based on the coding characteristics of nodes in the graph, and generate a current risk list and a potential conflict prediction list.

[0046] The security alarm information generation module is used to generate security alarm information based on the current risk list and the potential conflict prediction list.

[0047] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the digital twin-based intelligent monitoring method for port safety as described in the first aspect.

[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the digital twin-based intelligent monitoring method for port safety as described in the first aspect.

[0049] The aforementioned intelligent monitoring method, device, equipment, and media for port safety based on digital twins provide a spatiotemporally consistent global context for risk analysis by constructing a dynamic digital twin scenario that integrates multi-source data. Furthermore, by combining a port operation knowledge graph, the original perceived data is elevated to a dynamic knowledge situational subgraph containing domain semantics, achieving a structured representation of the "human-machine-environment" interaction relationship. Based on this, graph neural networks are used to encode and reason about the nodes and their complex relationships in the graph, effectively identifying potential risks that are difficult to determine based on simple rules and depend on equipment status, task background, and personnel intentions. This method enables the safety monitoring system to leap from the traditional perception-rule matching level to the cognition-associative reasoning level, significantly improving the accuracy of identifying high-risk behaviors and the predictability of potential conflicts, thereby achieving more accurate and intelligent safety early warnings in the complex and dynamic port operation environment. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart illustrating a digital twin-based intelligent monitoring method for port safety provided by this invention;

[0052] Figure 2 A flowchart illustrating a method for generating job adjustment suggestions in an optional embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of a digital twin-based intelligent monitoring device for port safety provided by the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] In one embodiment, such as Figure 1As shown, a digital twin-based intelligent monitoring method for port safety is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that this method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps S1 to S5:

[0056] S1. Based on the multi-source heterogeneous raw data at the dock site, perform spatiotemporal alignment and fusion to generate a dynamic digital twin scene.

[0057] Optionally, the multi-source heterogeneous raw data includes video stream data from the dock site, equipment operating parameter data (such as the position and speed data of automated guided vehicles and the lifting parameters of gantry cranes), and personnel identification data (such as personnel identification and location data read by RFID). Spatiotemporal alignment and fusion require achieving data dimensional consistency first. In the time dimension, Coordinated Universal Time (UTC) is used to uniformly calibrate the timestamps of all data. For differences in sampling frequency, linear interpolation is used to increase the sampling frequency of low-sampling-rate data to match that of high-sampling-rate data, ensuring time synchronization. In the spatial dimension, a Cartesian coordinate system is established with a pre-set fixed reference point at the dock as the origin. Coordinate transformation algorithms are used to uniformly transform various raw location data, such as GPS latitude and longitude and RFID area codes, to this coordinate system, achieving spatial uniformity. In the data fusion stage, dynamic data uses a Kalman filter algorithm to reduce measurement errors through prediction and update steps. Static data uses a weighted average algorithm, allocating weights according to the accuracy of the data source. The fused data is input into a 3D modeling tool to drive the real-time update of the position and state of entities in the model, generating a dynamic digital twin scene. The Kalman filter algorithm continuously corrects data deviations by establishing the state equation and observation equation of the dynamic system, while the linear interpolation method fills in data gaps through the linear relationship between adjacent sampling points, ensuring that the fused data can accurately reflect the actual situation on site.

[0058] S2. Based on the dynamic digital twin scenario and the pre-built dock operation knowledge graph, entity relationships and events are extracted and matched to generate a dynamic knowledge situation subgraph.

[0059] Optionally, the pre-built wharf operation knowledge graph includes an ontology layer and a data layer. The ontology layer defines core entity categories (such as personnel, automated guided vehicles, and gantry cranes), entity attributes (such as personnel identity type and equipment operating status), relationships between entities (such as personnel operating equipment and equipment traveling within an area), and event categories (such as normal equipment operation and personnel intruding into a restricted area). The data layer stores entity instance data from historical operations. Based on real-time information output from the dynamic digital twin scene, a BERT (Bidirectional Encoder Representations from Transformers)-based entity recognition model is used to process textual data, such as equipment status reports and text descriptions converted from images, outputting entity categories and instances. A CNN (Convolutional Neural Network)-based relation extraction model is used, combined with the location association information of entities in the scene, such as the distance between personnel and equipment, to output the relationship type between entities. A Transformer-based event detection model is used, inputting entity state change data, and determining event categories and participating entities through trigger word recognition and argument extraction. The extracted results are matched with the ontology layer of the knowledge graph, and elements that meet the definition are selected. A dynamic knowledge situation subgraph is constructed using a graph structure storage tool. The BERT model improves the accuracy of entity recognition through bidirectional semantic coding, while the CNN model enhances the accuracy of relationship determination through local feature extraction.

[0060] S3. Perform graph neural network encoding on the dynamic knowledge situation subgraph to obtain the encoded features of the nodes in the graph.

[0061] Optionally, the dynamic knowledge situation subgraph is first converted into a mathematical format that can be processed by a graph neural network (GNN), constructing an adjacency matrix A and a node feature matrix X: In adjacency matrix A, the corresponding element of a node with an edge (entity relationship or event association) is set to 1, otherwise it is 0; the node feature matrix X contains entity attribute features (such as equipment speed, personnel identification code) and event features (such as event duration, urgency level code). All features are normalized and mapped to a fixed interval to eliminate the influence of dimensions. A graph convolutional network (GCN) is preferred for encoding, following a hierarchical propagation rule. The feature update formula from layer l to layer l+1 is: ,in The normalized adjacency matrix is ​​calculated as follows: D is the degree matrix (the sum of the number of edges of each node), and I is the identity matrix (preserving the characteristics of each node). Let l be the feature matrix of the nodes in the l-th layer. The weight matrix is ​​a learnable matrix, trained and optimized using historical datasets. The training process employs gradient descent to minimize the loss function, where σ is the ReLU (Rectified Linear Unit) non-linear activation function. A multi-layer GCN structure is configured, with the initial layer... After multiple layered propagation calculations, the final layer node feature matrix is ​​output. Each row vector is the encoded feature of the node in the graph. This process improves the semantic and structural expressive power of the features by fusing information from the node itself and its neighborhood.

[0062] S4. Based on the coding characteristics of the nodes in the graph, perform risk and conflict analysis to generate a current risk list and a potential conflict prediction list.

[0063] Optionally, a risk rule base is pre-built, containing current risk rules and potential conflict rules. Current risk rules specify risk judgment conditions related to entity status and location, such as the distance between personnel and moving equipment being less than a safe range. Potential conflict rules define conflict judgment conditions related to equipment routes and time, such as the expected arrival time difference in the intersection area of ​​two planned routes being too small. Based on the encoding features of nodes in the graph, a logistic regression classification model is used in the risk analysis phase. This model is trained using a historical dataset (containing the node encoding features and "risk / normal" labels corresponding to historical risk events and normal events). The training process uses a cross-entropy loss function to optimize the model parameters. ,in For sample labels, The model predicts the probability of a risk and outputs the risk probability for each node combination. If the probability exceeds a preset threshold, the node combination and its corresponding risk type are added to the current risk list. The potential conflict prediction stage uses a Long Short-Term Memory (LSTM) network model. It inputs the node encoding features of a dynamic knowledge situational subgraph spanning multiple consecutive frames. The LSTM's gating mechanism (input gate, forget gate, output gate) captures the temporal variation patterns of these features, predicting future node feature trends. If the prediction results match the potential conflict rules, the corresponding node combination, conflict type, and expected occurrence time are added to the potential conflict prediction list. The list also labels the involved entity identifiers, occurrence locations, and risk levels.

[0064] S5. Generate and process security alarm information based on the current risk list and the potential conflict prediction list.

[0065] Optionally, alarm rules are preset to clarify the alarm triggering conditions and information presentation elements corresponding to different risk levels. For example, high-risk levels require the inclusion of risk / conflict type, involved entities, location of occurrence, and emergency handling suggestions, while low-risk levels can simplify the presentation elements, such as only including type and location. Based on the current risk list and potential conflict prediction list, key information is extracted, including risk / conflict type, involved entity identification, location of occurrence, risk level, and expected occurrence time (for potential conflicts). The above information is then structured and integrated, eliminating redundant content and supplementing missing elements to ensure complete and non-repetitive information. Furthermore, the integrated information is formatted according to a preset format, such as a combination of text description and location coordinate annotation, so that the processed information can be directly displayed on the terminal interface or pushed to relevant terminals via standard communication protocols. The preset alarm rules are formulated by reviewing the safety specifications for dock operations, the structured integration achieves information classification through field mapping, and the formatting process ensures a unified output format through template matching, ultimately generating clear and easy-to-understand safety alarm information.

[0066] The aforementioned intelligent monitoring method for port safety based on digital twins first generates a dynamic digital twin scenario through spatiotemporal alignment and fusion of multi-source heterogeneous data, providing basic data that closely matches the actual situation. Then, combined with a pre-built port operation knowledge graph, it extracts and matches entity relationships and events to construct a dynamic knowledge situational subgraph, achieving structured data transformation. Subsequently, it processes the subgraph through graph neural network encoding to obtain the encoded features of the fused nodes themselves and their associated information. Based on the encoded features and a pre-built rule base, it uses classification and time-series prediction models to analyze risks and conflicts, generating corresponding lists. Finally, it processes the list information according to preset alarm rules to generate safety alarm information. This method, through multi-technology collaboration, solves the problem of traditional port monitoring lacking deep semantic understanding and contextual reasoning, achieving an intelligent upgrade from "perception" to "cognition." It effectively reduces the false alarm and false negative rates of the monitoring system, improves the early warning foresight and intelligence level of port safety monitoring, and provides reliable protection for port operation safety.

[0067] In one embodiment, S2 includes:

[0068] S21. Based on the real-time state attributes of the twins in the dynamic digital twin scenario, perform spatiotemporal interaction measurement calculations to obtain the distance and proximity duration between the twins.

[0069] Optionally, the real-time state attributes of the twin include spatial location coordinates and a timestamp. The location coordinates are updated in real time based on an established Cartesian coordinate system, and the timestamp is calibrated using Coordinated Universal Time (UTC). In the calculation of spatiotemporal interaction metrics, distance is calculated using the Euclidean distance formula. Specifically, the location coordinates of the two twins at the same timestamp are extracted and substituted into the formula to obtain the real-time spatial distance. The calculation of the proximity duration requires first setting a proximity judgment condition. When the calculated distance meets the condition, the timestamp of the first time the condition is met is recorded as the start time. Subsequently, it is continuously monitored whether the distance remains within the proximity range. If it continues to meet the condition, the difference between the current timestamp and the start timestamp is used as the proximity duration. If the distance exceeds the proximity range midway, the start timestamp is reset to ensure that the duration calculation reflects the actual proximity interaction process.

[0070] S22. Based on distance and proximity duration, calculate the probability of relationship existence using a preset probability calculation model to obtain the probability value corresponding to the dynamic relationship triplet; the probability value is calculated using the following formula:

[0071]

[0072] in, This is a probability value. For distance, For the nearest duration, The preset distance threshold, The preset minimum duration threshold, This is the adjustment coefficient.

[0073] Optionally, the preset probability calculation model is a given formula for the existence probability of a relationship, where each parameter needs to be preset based on the characteristics of the dock operation scenario. Specifically, Distance thresholds are set according to the operational safety regulations for different entity types (such as personnel and automated guided vehicles, automated guided vehicles and gantry cranes). This is the minimum duration threshold for determining the validity of entity interactions. This is an adjustment coefficient calibrated using historical interaction data, used to balance the influence of duration on probability. In the formula, the first part... Used to quantify the impact of distance on the probability of a relationship existing, when d ≤ When d > 1, this part takes values ​​between 0 and 1 and increases as d decreases. The value is 0 when the time interval is reached; the latter part σ(・) is the sigmoid function, whose mathematical expression is: Used to connect duration with The difference is converted into a value between 0 and 1, thus quantifying the validity of the adjacent duration. Multiplying the two results yields the probability value corresponding to the dynamic relation triple; the closer the probability value is to 1, the higher the probability that the relation exists.

[0074] S23. Based on real-time task information and equipment status data from the terminal operation management system, perform semantic consistency verification on the dynamic relationship triples to obtain the verified dynamic relationship triples and conflict event nodes.

[0075] Optionally, real-time task information includes planned operation tasks for equipment and assigned operation instructions for personnel. Equipment status data includes the current operating mode and working status of the equipment. The above data needs to be obtained from the terminal operation management system through a standard data interface and associated with dynamic relation triples (composed of entity pairs and candidate relations). Semantic consistency verification requires the establishment of a verification rule base. The rule base is based on the terminal operation process and safety specifications. For example, if the dynamic relation triple is "personnel-approach-automated guided vehicle", then it is verified whether the current real-time task of the automated guided vehicle is "maintenance task" and whether the equipment status is "shutdown status". If both are satisfied, the relation triple is determined to be semantically consistent. If the current task of the automated guided vehicle is "transportation task" and the status is "driving status", then the relation triple is determined to be semantically conflicting, and a conflict event node is generated. The conflict event node needs to include information such as conflict type, involved twin identifiers, and timestamp of the conflict occurrence to ensure traceability of the association with related entities.

[0076] S24. Perform graph structure combination processing on the verified dynamic relation triples, conflict event nodes, and static entities and relations matched from the dock operation knowledge graph to generate a dynamic knowledge situation subgraph.

[0077] Optionally, when matching static entities and relationships from the port operation knowledge graph, associations are made through entity identifiers. For example, based on the Automated Guided Vehicle (AGV) identifier involved in the dynamic relationship triple, the static attributes (such as model and assigned work area) and static relationships (such as AGV-Assignment-Work Area) corresponding to that AGV are matched from the knowledge graph. During graph structure combination processing, the matched static entities and relationships serve as the basic framework of the graph structure. The verified dynamic relationship triples are added to the graph as edges that are updated in real time. The probability values ​​of the dynamic relationship triples can be stored as attributes of the edges. At the same time, conflict event nodes are added to the graph as independent nodes and connected to the twin nodes involved in the conflict through edges to form a complete graph structure. Preferably, a graph structure storage tool is used to implement the combination processing to ensure that the association logic between nodes and edges in the graph is clear. The final generated dynamic knowledge situation subgraph contains both static basic information and integrates real-time dynamic relationships and conflict events.

[0078] In the above embodiments, the distance and proximity duration between twins are calculated through spatiotemporal interaction metrics to provide a quantitative basis for determining the existence of relationships. Then, the probability of relationship existence is calculated based on a preset probability model to improve the objectivity and accuracy of relationship identification. Subsequently, semantic consistency verification eliminates unreasonable relationships that contradict task or device states, generating conflict event nodes to capture abnormal interactions. Finally, static entity relationships, verified dynamic relationships, and conflict nodes are integrated to generate a dynamic knowledge situation subgraph. This embodiment effectively improves the accuracy and scenario adaptability of relationships in the dynamic knowledge situation subgraph through the synergy of quantified spatiotemporal interaction, probability determination, and semantic verification, avoiding interference from invalid or erroneous relationships in subsequent analysis.

[0079] In one embodiment, S3 includes:

[0080] S31. Initialize the feature vectors based on the type and attribute information of the nodes in the dynamic knowledge situation subgraph to obtain the initial feature vectors of each node.

[0081] Optionally, node types need to be first divided into entity nodes (such as personnel, automated guided vehicles, and work areas) and event nodes (such as conflict events). The attribute information of different types of nodes differs. Entity node attributes include identity, operating status, and spatial location (calibrated to a unified coordinate system), while event node attributes include event type, timestamp, and involved entity identifiers. Feature vector initialization requires pre-setting attribute encoding rules. Numerical attributes, such as equipment operating speed and event duration, are processed using Min-Max normalization to map attribute values ​​to a fixed range to eliminate the influence of dimensions. Categorical attributes (such as personnel identity type and event type) are converted into binary vectors using one-hot encoding. Textual attributes (such as event descriptions) are converted into low-dimensional dense vectors using word embedding technology. All encoded attribute vectors of the same node are concatenated in a preset order to form the initial feature vector of that node, ensuring that the initial feature vector dimension is uniform across different types of nodes.

[0082] S32. Based on the initial feature vector and the topological structure and edge relationship type of the dynamic knowledge situation subgraph, perform relation-aware message aggregation iterative processing to obtain the iterated node feature vector.

[0083] Optionally, the topology is represented by an adjacency matrix, where elements indicate whether there is a direct connection between nodes. Edge relationships are semantically categorized into different types, such as "entity-proximity-entity," "entity-belonging-region," and "entity-participation-event," with pre-defined relationship weights for each edge type. The relationship-aware message aggregation iterative processing follows a fixed procedure: for each target node, all its neighboring nodes are first filtered according to the adjacency matrix, and then grouped by edge relationship type. For each group of neighboring nodes, the corresponding neighbor message is calculated as the product of the neighboring node's current feature vector and the corresponding edge type weight. The neighbor messages from all groups are weighted and summed to obtain the aggregated message. This aggregated message is then fused with the target node's own current feature vector through a linear transformation, followed by ReLU (Rectified Linear Unit) activation to obtain the updated feature vector of the target node. All nodes are updated synchronously according to this procedure, completing one iteration. This iteration is repeated a preset number of times, with each iteration performing the same operation based on the previous round's node feature vector and a fixed topology and edge relationship weights, ultimately yielding the iterated node feature vector.

[0084] S33. Based on the iterated node feature vectors, feature output is performed to obtain the encoded features of the nodes in the graph.

[0085] Optionally, the feature output needs to first set an iteration stopping condition. Iteration stops when the number of iterations reaches a preset value or the change in the node feature vector between two adjacent iterations is less than a preset threshold. The iterated node feature vectors undergo dimension adaptation processing, preferably implemented through a fully connected layer. The input to the fully connected layer is the iterated node feature vector, and the output is a vector of fixed dimensions. The weight parameters within the layer are optimized through training on historical datasets. During training, the mean squared error (MSE) loss function is used to minimize the difference between the output vector and the labeled feature vector. After dimension adaptation, the processed vector is directly output, which is the encoded feature of the node in the graph. This feature retains the node's own attribute information and incorporates the association information brought by the topological structure and edge relationship type.

[0086] In the above embodiment, the initial feature vector of nodes is standardized and initialized through attribute encoding rules, providing a unified foundation for iteration. Then, based on the topology and edge relationship types, a relationship-aware message aggregation iteration is performed, enabling node features to integrate neighborhood association information and relational semantics. Finally, the encoded features are output through iteration stopping judgment and dimension adaptation to ensure that the features meet the needs of subsequent analysis. This embodiment effectively improves the semantic expressiveness and structural correlation of node encoded features through phased feature processing, avoiding the feature one-sidedness caused by single attributes or ignoring relational semantics.

[0087] In one embodiment, S4 includes:

[0088] S41. Based on the encoding features of the entity nodes in the graph, the risk probability distribution is calculated using a classifier to obtain the probability distribution of each entity node corresponding to different risk categories.

[0089] Optionally, a softmax regression classifier is selected to calculate the risk probability distribution. This classifier is a commonly used model for multi-classification tasks and can be applied directly without customization. The classifier training process is based on the historical risk dataset of the dock. Each sample in the dataset contains the encoded features of the entity node and the corresponding risk category label, such as "personnel intrusion risk," "equipment collision risk," and "falling object risk." During training, the cross-entropy loss function is used to measure the deviation between the predicted value and the true label. The classifier parameters are iteratively optimized using gradient descent until the loss function value converges to a preset range. During calculation, the encoded features of the entity node in the figure are input into the trained softmax regression classifier. The classifier processes the data through linear transformation and the softmax activation function, and outputs the set of probability values ​​corresponding to all preset risk categories for that entity node, i.e., the risk probability distribution. Each probability value is between 0 and 1, and the sum of all probability values ​​is 1, representing the probability that the entity node belongs to the corresponding risk category.

[0090] S42. Based on the probability distribution and the preset judgment threshold, identify high-risk entities and generate the current risk list.

[0091] Optionally, the preset judgment thresholds need to be determined based on the port operation safety regulations and historical risk event statistics. Different thresholds can be set for different risk categories, or a uniform threshold can be used, depending on the actual safety management needs. The risk probability distribution of an entity node is compared with the preset thresholds. If the probability value of one or more risk categories corresponding to the node exceeds the corresponding threshold, the entity node is judged as a high-risk entity. Further, information on all high-risk entities is integrated to generate a current risk list. The list must include the unique identifier of the high-risk entity (such as personnel ID, equipment number), the corresponding risk category, the probability value of each risk category, and the judgment basis (i.e., the threshold value exceeded), ensuring that the list information is complete and traceable.

[0092] S43. Based on the encoding features of any two entity nodes, a conflict relationship prediction is performed using a relationship classifier to obtain the conflict relationship prediction results between entity pairs.

[0093] Optionally, a Multilayer Perceptron (MLP) is used as the relation classifier. This model consists of an input layer, hidden layers, and an output layer, with a clear structure and easy implementation. The input layer receives a concatenated vector of the encoded features of any two entity nodes, that is, the encoded features of the two nodes are combined sequentially to form a new feature vector. The hidden layer performs non-linear feature transformation using the ReLU activation function, and the output layer outputs the result using the softmax activation function. During the classifier training phase, a historical entity pair dataset is used, with sample labels indicating the conflict relationship type between entity pairs, such as no conflict, path intersection conflict, overlapping task range conflict, etc. The error is calculated using the cross-entropy loss function, and the model weights and bias parameters are optimized using gradient descent. During prediction, the concatenated vector of the encoded features of the target entity pair is input into the trained MLP. The model outputs the probability of each type of conflict relationship corresponding to the entity pair, and the conflict relationship with the highest probability is taken as the final conflict relationship prediction result.

[0094] S44. Based on the conflict relationship prediction results, assemble the conflict records to generate a potential conflict prediction list.

[0095] Optionally, the assembly of conflict records requires extracting key information based on the conflict relationship prediction results. This includes the unique identifiers of the two entity nodes involved in the conflict, the predicted conflict relationship type, and the confidence level corresponding to the prediction result (i.e., the maximum probability value output by the model). If the prediction result includes association conditions for the occurrence of the conflict, such as time windows or spatial region associations inferred from historical data, these should also be included in the record. Furthermore, the extracted information is structured, using a unified field format (e.g., entity pair identifier: [ID1, ID2]; conflict type: XXX; confidence level: XX; association condition: XXX) to organize the conflict records, ensuring consistent information format and no redundancy in each record. Finally, all structured conflict records are summarized to generate a potential conflict prediction list. This list needs to be sorted by conflict relationship type or confidence level to facilitate prioritizing high-confidence potential conflicts.

[0096] In the above embodiment, a softmax regression classifier is used to process the encoded features of entity nodes to obtain a quantifiable risk probability distribution. Then, a preset threshold is used to filter high-risk entities, generating a current risk list and achieving accurate location of existing risks. Subsequently, an MLP relationship classifier is used to predict conflict relationships between entity pairs, capturing potential risk associations. Finally, conflict records are assembled in a structured manner to generate a potential conflict prediction list, completing the prediction of future risks. This embodiment uses a mature and reproducible classification model, relies on historical data training to ensure model reliability, and improves the interpretability of results through clear threshold settings and structured record specifications, effectively enhancing the accuracy of port risk identification and the effectiveness of potential conflict prediction.

[0097] In one embodiment, prior to S44, the following is also included:

[0098] S45. Based on multiple dynamic knowledge situation subgraphs in historical time series, the future state is predicted by a spatiotemporal graph neural network model to obtain the predicted state of each entity node at a specified future time.

[0099] Optionally, multiple dynamic knowledge situation subgraphs in the historical time series need to be organized in timestamp order. Each subgraph needs to retain the encoded features of nodes, topological structure, and corresponding time information to form a time series subgraph sequence. The preferred spatio-temporal graph neural network model is the Spatio-Temporal Graph Convolutional Network (ST-GCN). This model processes data collaboratively through graph convolutions in the spatial dimension and recurrent units in the temporal dimension: the spatial dimension uses graph convolutional layers to aggregate the neighborhood features of nodes based on the subgraph topology, preserving the spatial relationships between entities; the temporal dimension uses gated recurrent units (GRUs) to process the time series subgraph sequence and capture the changing patterns of node states over time. During model training, the historical time series subgraph sequence is used as input, and the actual state of nodes at future times (such as location and operating mode) of each subgraph is used as labels. Mean Squared Error (MSE) is used as the loss function, and the model parameters are iteratively updated through the Adam optimizer until the loss function converges. After training, input the historical time series subgraph sequence to be processed, and the model can output the predicted state of each entity node at a specified future time. The predicted state includes key attributes such as spatial location and operational status.

[0100] S46. Based on the predicted state of each entity node at a specified future time, identify potential collisions and generate a set of potential future conflicts.

[0101] Optionally, in the predicted state of each entity node at a future specified time, the core is to extract spatial location attributes and entity type information. Spatial location is determined based on the established Cartesian coordinate system, and entity type includes categories such as personnel, automated guided vehicles (IGVs), and gantry cranes. Potential collision identification requires pre-setting collision judgment rules, which are formulated according to entity type combinations. For example, the collision judgment rule for personnel and IGV is "the Euclidean distance between their predicted positions is less than a preset safety threshold," and the collision judgment rule for IGV and gantry crane is "the predicted position of IGV falls within the area covered by the gantry crane's rotation radius." Parameters such as the safety threshold and rotation radius need to be determined according to the dock operation safety regulations. All entity node pairs are traversed, and the spatial positional relationship of the entity pair at the same future specified time is calculated. If the corresponding collision judgment rule is met, the entity pair is marked as a potential collision object, and the entity pair identifier, predicted collision time, and entity type are recorded, assembling to form a set of future potential conflicts.

[0102] S47. Calculate the time-to-collision value based on the set of future potential conflicts to obtain the time-to-collision value corresponding to each future potential conflict; the time-to-collision value is the time interval from the current moment to the predicted time of the collision.

[0103] Optionally, calculating the Time To Collision (TTC) value requires obtaining two key timestamps: the current time (UTC) when this step was performed, and the predicted collision time (a future specified time recorded when potential collisions were identified). The TTC value is calculated as the difference between the two, using the following formula: ,in To predict the timestamp of the collision, This is the timestamp of the current moment. If the calculated result is negative, indicating that the predicted collision time is earlier than the current moment due to data errors, the TTC is set to 0, signifying that the conflict is in an emergency state. Each potential future conflict must be associated with a corresponding TTC value, which is then added to each record in the conflict set.

[0104] S48. Based on the time to collision value, perform risk level quantification to generate quantified future conflict information.

[0105] Optionally, the risk level quantification process requires pre-setting risk level classification rules. These rules are based on the TTC value and combined with the risk weights of entity type combinations. Specifically, the range of TTC values ​​is first defined, with the range defined by referencing the minimum effective response time for the terminal's emergency response. For example, a TTC between 0 and the shortest response time is considered "extremely high risk," while a TTC between the shortest response time and twice the shortest response time is considered "high risk," and so on, dividing the risk into multiple levels. Simultaneously, risk weights are assigned to different entity type combinations. For instance, the collision weight between personnel and IGVs is higher than the collision weight between IGVs. These weights are determined by analyzing historical accident statistics from the terminal. When calculating the quantified risk, a basic risk level is first determined based on the TTC value. Then, the risk weights of the entity type combinations are used to fine-tune the basic level. For example, if the weight exceeds a threshold, the level is increased by one level. Finally, future conflict information is generated, including entity pair identifiers, predicted collision times, TTC values, and quantified risk levels.

[0106] In the above embodiment, the historical time-series subgraph is first processed using a spatiotemporal graph neural network model to achieve accurate prediction of the future state of entity nodes. Then, based on the predicted state and preset rules, potential future collisions are identified, forming a conflict set. Subsequently, the time-to-collision value for each conflict is calculated to clarify the urgency of the time dimension. Finally, the risk level is quantified by combining the TTC value and entity risk weight to generate structured future conflict information. This embodiment starts with historical data time-series modeling and gradually realizes the prediction, identification, time quantification, and risk classification of future conflicts. It not only provides the ability to predict in advance for port safety management but also provides a clear basis for emergency resource allocation through risk quantification, effectively extending the emergency response window and significantly reducing the probability of future safety accidents.

[0107] In an optional embodiment, such as Figure 2 As shown, it also includes:

[0108] S51. Based on the risk types and entity identifiers in the current risk list and the potential conflict prediction list, perform matching query processing from the pre-set contingency plan rule base to obtain the initial set of intervention measures.

[0109] Optionally, the pre-built contingency plan rule base needs to be constructed in advance based on terminal safety regulations, historical accident handling cases, and operational process standards. Its core structure is indexed by "risk type-entity combination," with each index corresponding to one or more structured intervention measure entries. The entry content includes the measure name, execution process, required resources, and preset execution time. During the matching query processing, key query conditions are first extracted from the current risk list and potential conflict prediction list, including risk type (such as "personnel intrusion into IGV operating area" and "equipment collision risk") and entity identifier (such as the personnel ID and IGV number involved). Then, using Structured Query Language (SQL) or graph database query statements, with "consistent risk type and matching entity combination" as the filtering condition, the corresponding intervention measure entries are retrieved from the contingency plan rule base. All retrieved entries are deduplicated and integrated to form the initial set of intervention measures.

[0110] S52. Based on the time-to-collision value in the potential conflict prediction list, the initial set of intervention measures is filtered for response time feasibility to obtain a subset of feasible intervention measures.

[0111] Optionally, the Time to Collision (TTC) value in the potential conflict prediction list clearly defines the emergency time boundary of the conflict. The response time feasibility filtering requires first extracting the preset execution time of each measure in the initial set of intervention measures. This time is an inherent attribute of the intervention measure entries in the contingency plan rule base and is determined by historical execution data and workflow analysis. The filtering logic uses "preset execution time of measure ≤ TTC" as the core judgment criterion. Specifically, it iterates through each intervention measure in the initial set and compares its preset execution time with the TTC of the corresponding potential conflict. If the time is less than or equal to the TTC, it indicates that the measure can be executed before the conflict occurs, and the measure is retained; if the time is greater than the TTC, it indicates that the measure cannot take effect in time, and the measure is removed. The remaining measures constitute a subset of feasible intervention measures.

[0112] S53. Based on the preset work efficiency evaluation rules, conduct multi-objective utility evaluation on a subset of feasible intervention measures and generate work adjustment suggestions.

[0113] Optionally, the preset operational efficiency evaluation rules need to cover three core dimensions: safety effectiveness, operational delay cost, and resource utilization rate. Safety effectiveness is quantified by the probability of risk reduction achieved through measures; operational delay cost is quantified by the duration of operational pauses or route adjustments caused by the measures; and resource utilization rate is quantified by the number of equipment (such as dispatched vehicles) and personnel required by the measures. In multi-objective utility evaluation, weights are first assigned to the three dimensions. These weights are determined using the Analytic Hierarchy Process (AHP) combined with historical terminal operational data and safety management priorities. Then, each measure in the subset of feasible intervention measures is scored separately across the three dimensions (scores mapped to fixed intervals), according to the formula: Calculate the overall utility value. The overall utility value, , , Weights for each dimension, , , To score each dimension, select one or two measures with the highest overall utility value, supplement their execution steps, required resource scheduling paths, and job adjustment nodes, and generate job adjustment suggestions.

[0114] In the above embodiment, initial intervention measures corresponding to the risks are first obtained through matching with the contingency plan rule base to ensure the safety and adaptability of the measures; then, measures that cannot be implemented in a timely manner are filtered out based on TTC (Time-to-Care) to ensure the time feasibility of intervention; finally, the optimal measures that balance safety and efficiency are screened through multi-dimensional utility evaluation to generate implementable operational adjustment suggestions. This embodiment forms a complete closed loop from risk identification to intervention implementation, from measure matching to feasibility screening to utility optimization. It avoids interference with operations by ineffective measures and balances safety assurance and production efficiency, providing timely and practical decision support for port safety management.

[0115] In one embodiment, it further includes:

[0116] S61. Based on the feedback annotation data after historical alarm handling, and the dynamic digital twin scene snapshot and dynamic knowledge situation subgraph corresponding to the historical moment of the feedback annotation data, a dataset is constructed to generate a model training and knowledge mining dataset.

[0117] Optionally, the feedback annotation data consists of historical alarm handling result identifiers, including category labels such as effective handling, false alarms, and missed alarms. The dynamic digital twin scene snapshot is a static data set of the twin's location and status at a historical moment, and the dynamic knowledge situation subgraph is the entity-relationship-event structured data at the corresponding moment. When constructing the dataset, the feedback annotation data is associated one-to-one with the scene snapshot and knowledge situation subgraph at the corresponding historical moment according to the timestamp, forming a three-dimensional data unit of "annotation-snapshot-subgraph". Subsequently, the three-dimensional data units are cleaned to remove units with mismatched timestamps or missing data fields. Then, the cleaned data is divided into a model training set and a knowledge mining set according to a preset ratio. The division process adopts a time series slicing method to avoid training bias caused by data overlap at different time periods, and finally generates a model training and knowledge mining dataset with aligned structure and balanced categories.

[0118] S62. Based on the model training and knowledge mining dataset, the parameters of the graph neural network model are optimized to obtain the updated graph neural network model parameters.

[0119] Optionally, the task type of the graph neural network needs to be clearly defined before optimization. For risk classification tasks, the cross-entropy loss function is used; for state prediction tasks, the mean squared error loss function is used. Using the model training set as input, the graph neural network calculates the prediction result through forward propagation. The prediction result is compared with the true labels in the feedback labeled data to obtain the current loss value. The Adam optimizer is used for backpropagation based on the loss value to calculate the gradients of each parameter (such as the weight matrix and bias vector), and the parameters are adjusted according to a preset learning rate to reduce the loss. After a certain number of iterations, the model performance is verified using a knowledge mining set. If the validation set loss does not decrease for several consecutive rounds, the iteration stops, and the model parameters at this point are saved as the updated parameters. The entire process requires recording an iteration log, including the training loss and validation loss for each round, to facilitate tracking the effect of parameter adjustments.

[0120] S63. Based on the analysis results of new patterns in the model training and knowledge mining dataset, the pre-constructed wharf operation knowledge graph is structurally expanded to obtain an updated wharf operation knowledge graph.

[0121] Optionally, pattern extraction is first performed on the three-dimensional data units in the knowledge mining set. This involves statistically analyzing the entity pairs, relation types, and event categories that do not match the pre-built knowledge graph ontology layer, such as undefined relationships like "drone inspection equipment - collaborative operation - gantry crane." The extracted candidate patterns are then frequency-filtered, retaining those that occur frequently and conform to the dock operation logic. The validity of these patterns is then verified against the latest dock operation specifications and safety standards, eliminating any occasional invalid data combinations. Once valid, corresponding entity categories, relation types, and event categories are added to the knowledge graph ontology layer. Entity instances and relation instances corresponding to the pattern are added to the data layer. Simultaneously, the graph's inference rules are updated, such as adding a constraint like "collaborative operation must meet equipment status synchronization," ensuring that the expanded graph is consistent with the original structure and logic. Finally, the updated dock operation knowledge graph is obtained.

[0122] In the above embodiment, a high-quality dataset is first constructed by associating historical alarm feedback, twin scene snapshots, and knowledge subgraphs, providing a foundation for model optimization and knowledge mining. Then, the dataset is used to optimize the graph neural network parameters through gradient descent and validation convergence mechanisms, improving model prediction accuracy. Finally, the knowledge graph structure is expanded by analyzing effective new patterns in the dataset, enhancing scenario adaptability. This embodiment enables the dock safety monitoring system to continuously adapt to changes in operational scenarios, gradually reducing false alarm and missed alarm rates, and improving the intelligence and reliability of long-term monitoring.

[0123] In the aforementioned intelligent monitoring method, device, equipment, and media for terminal safety based on digital twins, a dynamic digital twin scenario is generated through spatiotemporal alignment and fusion of multi-source heterogeneous raw data, providing basic data that closely reflects the actual situation on-site for monitoring. Then, combined with a pre-constructed terminal operation knowledge graph, entity relationships and events are extracted and matched through spatiotemporal interaction measurement, probability calculation, and semantic verification to generate a dynamic knowledge situation subgraph. Subsequently, node feature initialization and relationship-aware message aggregation iteration are performed on this subgraph, and node features integrating semantic and structural information are obtained through graph neural network encoding. Based on the encoded features, risk probability calculation, conflict relationship prediction, and future state spatiotemporal prediction are carried out to generate a current risk list, a potential conflict prediction list, and quantified future conflict information. Further, intervention measures are matched with a contingency plan rule base, and operational adjustment suggestions are generated through response time feasibility filtering and operational efficiency evaluation. Finally, a dataset is constructed using feedback annotation data after historical alarm handling, the graph neural network parameters are optimized, and the terminal operation knowledge graph is expanded, forming a closed-loop iteration of data-model-knowledge. This technical solution visualizes the on-site scene through dynamic digital twin scenarios and achieves deep semantic association and contextual reasoning through the collaboration of knowledge graphs and graph neural networks. It effectively solves the problems of high false alarm rate, high risk of missed alarm, and inability to upgrade from "perception" to "cognition" caused by the reliance on visual perception and simple spatial rules in the existing monitoring technologies. It realizes intelligent early warning and continuous optimization of port safety monitoring.

[0124] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0125] Based on the same inventive concept, this application also provides an apparatus for implementing the aforementioned intelligent monitoring method for terminal safety based on digital twins. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the intelligent monitoring apparatus for terminal safety based on digital twins provided below can be found in the limitations of the intelligent monitoring method for terminal safety based on digital twins described above, and will not be repeated here.

[0126] In one exemplary embodiment, such as Figure 3 As shown, a digital twin-based intelligent monitoring device 10 for port safety is provided, comprising:

[0127] The spatiotemporal alignment and fusion module 11 is used to perform spatiotemporal alignment and fusion based on multi-source heterogeneous raw data from the dock site to generate a dynamic digital twin scene.

[0128] The knowledge situation extraction module 12 is used to extract and match entity relationships and events based on the dynamic digital twin scenario and the pre-built dock operation knowledge graph, and generate a dynamic knowledge situation subgraph.

[0129] Graph neural network encoding module 13 is used to perform graph neural network encoding processing on the dynamic knowledge situation subgraph to obtain the encoded features of the nodes in the graph;

[0130] Risk and conflict analysis module 14 is used to perform risk and conflict analysis based on the coding characteristics of nodes in the graph, and generate a current risk list and a potential conflict prediction list.

[0131] The security alarm information generation module 15 is used to generate security alarm information based on the current risk list and the potential conflict prediction list.

[0132] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the aforementioned digital twin-based intelligent monitoring method for port safety.

[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a digital twin-based intelligent monitoring method for port safety as described above.

[0134] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0135] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A digital twin-based intelligent monitoring method for port safety, characterized in that, The method includes: S1. Based on the multi-source heterogeneous raw data from the dock site, perform spatiotemporal alignment and fusion to generate a dynamic digital twin scene; S2. Based on the dynamic digital twin scenario and the pre-built dock operation knowledge graph, entity relationships and events are extracted and matched to generate a dynamic knowledge situation subgraph. S3. Perform graph neural network encoding processing on the dynamic knowledge situation subgraph to obtain the encoding features of the nodes in the graph; S4. Based on the coding characteristics of the nodes in the graph, perform risk and conflict analysis to generate a current risk list and a potential conflict prediction list. S5. Based on the current risk list and the potential conflict prediction list, perform security alarm information generation processing to generate security alarm information.

2. The method according to claim 1, characterized in that, S2 includes: S21. Based on the real-time state attributes of the twins in the dynamic digital twin scenario, perform spatiotemporal interaction measurement calculations to obtain the distance and proximity duration between the twins; S22. Based on the distance and the duration of proximity, the probability of relationship existence is calculated using a preset probability calculation model to obtain the probability value corresponding to the dynamic relationship triplet; the probability value is calculated using the following formula: in, This is a probability value. For distance, For the nearest duration, The preset distance threshold, The preset minimum duration threshold, This is the adjustment coefficient; S23. Based on real-time task information and equipment status data from the dock operation management system, perform semantic consistency verification on the dynamic relationship triplet to obtain the verified dynamic relationship triplet and conflict event nodes. S24. Perform graph structure combination processing on the verified dynamic relationship triples, the conflict event nodes, and the static entities and relationships matched from the dock operation knowledge graph to generate the dynamic knowledge situation subgraph.

3. The method according to claim 1, characterized in that, S3 includes: S31. Based on the type and attribute information of the nodes in the dynamic knowledge situation subgraph, the feature vectors are initialized to obtain the initial feature vectors of each node; S32. Based on the initial feature vector and the topological structure and edge relationship type of the dynamic knowledge situation subgraph, perform relation-aware message aggregation iterative processing to obtain the iterated node feature vector; S33. Based on the iterated node feature vector, perform feature output to obtain the encoded features of the nodes in the graph.

4. The method according to claim 1, characterized in that, S4 includes: S41. Based on the encoding features of the entity nodes in the figure, the risk probability distribution is calculated by a classifier to obtain the probability distribution of each entity node corresponding to different risk categories. S42. Based on the probability distribution and the preset judgment threshold, identify high-risk entities and generate the current risk list; S43. Based on the encoding features of any two entity nodes, a conflict relationship prediction is performed using a relationship classifier to obtain the conflict relationship prediction results between entity pairs. S44. Based on the conflict relationship prediction results, assemble the conflict records to generate the potential conflict prediction list.

5. The method according to claim 4, characterized in that, Before S44, it also includes: S45. Based on multiple dynamic knowledge situation subgraphs in historical time series, the future state is predicted by a spatiotemporal graph neural network model to obtain the predicted state of each entity node at a specified future time. S46. Based on the predicted state of each entity node at the specified future time, perform potential collision identification to generate a set of potential future conflicts; S47. Calculate the time-to-collision value based on the set of future potential conflicts to obtain the time-to-collision value corresponding to each future potential conflict; the time-to-collision value is the time interval from the current moment to the predicted time of the collision. S48. Based on the time-to-collision value, perform risk level quantification processing to generate quantified future conflict information.

6. The method according to claim 1, characterized in that, Also includes: S51. Based on the risk types and entity identifiers in the current risk list and the potential conflict prediction list, perform matching query processing from the preset contingency plan rule base to obtain an initial set of intervention measures; S52. Based on the time-to-collision value in the potential conflict prediction list, the initial set of intervention measures is filtered for response time feasibility to obtain a subset of feasible intervention measures; S53. Based on the preset work efficiency evaluation rules, perform multi-objective utility evaluation on the subset of feasible intervention measures and generate work adjustment suggestions.

7. The method according to claim 1, characterized in that, Also includes: S61. Based on the feedback annotation data after historical alarm handling, and the dynamic digital twin scene snapshot and dynamic knowledge situation subgraph corresponding to the historical moment of the feedback annotation data, a dataset is constructed to generate a model training and knowledge mining dataset. S62. Based on the model training and knowledge mining dataset, perform parameter optimization on the graph neural network model to obtain updated graph neural network model parameters. S63. Based on the analysis results of the new patterns in the model training and knowledge mining dataset, the pre-constructed wharf operation knowledge graph is structurally expanded to obtain an updated wharf operation knowledge graph.

8. A digital twin-based intelligent monitoring device for port safety, characterized in that, The device includes: The spatiotemporal alignment and fusion module is used to perform spatiotemporal alignment and fusion based on multi-source heterogeneous raw data from the dock site to generate a dynamic digital twin scene. The knowledge situation extraction module is used to extract and match entity relationships and events based on the dynamic digital twin scenario and the pre-built dock operation knowledge graph, and generate a dynamic knowledge situation subgraph. The graph neural network encoding module is used to perform graph neural network encoding processing on the dynamic knowledge situation subgraph to obtain the encoded features of the nodes in the graph; The risk and conflict analysis module is used to perform risk and conflict analysis based on the coding characteristics of the nodes in the graph, and generate a current risk list and a potential conflict prediction list. The security alarm information generation module is used to generate security alarm information based on the current risk list and the potential conflict prediction list.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images