Power transformation infrastructure construction risk identification method and system, and medium

Abstract

The invention relates to the field of power transformation infrastructure security and protection, in particular to a power transformation infrastructure risk identification method and system and a medium, and the method comprises the steps: collecting multi-modal data of a construction site, the multi-modal data comprising video monitoring data, operation monitoring data, equipment state data and environment monitoring data; a BIM model is adopted as a three-dimensional digital base map of a construction site, fusion modeling is carried out on the multi-modal data and the BIM model, and the BIM model after fusion modeling is obtained; inputting the multi-modal data into a pre-trained AI large model, extracting safety risk characteristics and violation behavior modes of the construction site through the AI large model, and identifying safety risks and violation behaviors in real time by the AI large model based on the extracted safety risk characteristics and violation behavior modes; and in response to the AI large model, identifying security risks and violation behaviors in real time, generating alarm information, and positioning and displaying the alarm information through the BIM model after fusion modeling.

Description

Technical Field

[0001] The present disclosure relates to the field of substation infrastructure security, and more specifically, to a substation infrastructure risk identification method, system, and medium. Background Art

[0002] In recent years, with the increasing voltage levels, complexity, and scale of substation infrastructure projects, traditional construction site safety management methods have become unable to meet the efficiency and precision requirements of modern construction management. While existing smart construction site systems have achieved a certain degree of on-site monitoring and data collection, their overall level of intelligence still needs improvement. Due to a lack of effective coordination between systems, a unified and efficient safety risk management system has yet to be established.

[0003] First, traditional construction sites lack intuitive display of alarm information or early warning information. Management personnel, construction personnel, etc. are often unable to passively receive alarm information or early warning information, and are unable to fully and intuitively understand specific safety risks or violations from the entire construction site in a three-dimensional form. The rectification of the construction site is particularly passive, and it is difficult to quickly and accurately understand the location, time, and status of safety risks or violations.

[0004] Secondly, current construction site safety supervision relies on manual inspections, making it difficult to detect hidden dangers and violations in a timely manner. Furthermore, the deployment of various sensing devices (such as video surveillance and environmental sensors) and user display devices has formed relatively independent data silos, necessitating the organic integration of on-site data.

[0005] Thirdly, existing systems mostly use a rule-matching approach to identify risks and violations at construction sites, with poor accuracy and generalization capabilities. The unit models for load identification and diagnosis often act as a whole that is difficult to divide and collaborate. Moreover, after deployment, it is difficult to achieve automatic optimization based on the specific risk situations encountered during implementation. Additional personnel and redeployment are required, which incurs human and material costs, making it difficult to adapt to the dynamically changing scenario requirements in the construction environment.

[0006] In addition, after hidden dangers or violations are discovered, human intervention is required. The existing system lacks an automatic tracking and display mechanism, making it difficult to supervise the results of rectification work carried out by managers, and there is a safety risk of incomplete and incomplete rectification.

[0007] For example, the prior art CN118658279A discloses obtaining video surveillance data and equipment and environmental monitoring data, obtaining human posture sequences and image sequences of personnel, equipment and environmental markers based on the video surveillance data, and combining the equipment and environmental monitoring data sequences to realize multimodal data fusion analysis of personnel, equipment and environment. However, its fusion analysis is based on convolutional neural networks to fuse multimodal data. It only realizes the fusion analysis of multimodal data collected by sensing devices, and does not organically integrate multimodal data with three-dimensional display models, user display devices and other construction site data. It also forms a situation where sensing devices, three-dimensional display models and user display devices are independent of each other. It does not realize the organic integration of construction site data, form a unified and efficient safety risk management system, or form an effective supervision mechanism for the rectification work of management personnel.

[0008] Therefore, there is an urgent need to build an efficient and intelligent construction site safety management system that can integrate multi-source data, improve recognition accuracy, and achieve real-time early warning and closed-loop management. Summary of the Invention

[0009] The technical problem to be solved by the present disclosure is to address the above-mentioned shortcomings and provide a method, system and medium for identifying risks in substation infrastructure construction, so as to solve the technical problem that the existing technology cannot comprehensively and intuitively understand specific safety risks or violations in the entire construction site in a three-dimensional form. The present disclosure adopts the following technical solutions: In a first aspect, the present disclosure provides a method for identifying risks in substation infrastructure, the method comprising: Collecting multimodal data of the construction site, the multimodal data including video surveillance data, operation monitoring data, equipment status data, and environmental monitoring data; Using the BIM model as a three-dimensional digital base map of the construction site, fusing the multimodal data with the BIM model to obtain the fused BIM model, thereby achieving comprehensive modeling of the construction site; Inputting the multimodal data into a pre-trained AI big model, extracting safety risk characteristics and violation behavior patterns at the construction site through the AI ​​big model, and the AI ​​big model identifying safety risks and violation behaviors at the construction site in real time based on the extracted safety risk characteristics and violation behavior patterns; In response to the AI ​​large model identifying safety risks and violations at the construction site in real time, corresponding alarm information is generated and the alarm information is positioned and displayed in three dimensions through the BIM model after fusion modeling. During implementation, through the fusion modeling of the multimodal data and the BIM model, a BIM model that fully models the construction site can be obtained, which can then assist in visually displaying alarm information, safety risks, and violations through comprehensive modeling of the construction site, and can assist in the rectification work corresponding to the alarm information, which can be continuously displayed in three dimensions through the BIM model until the management personnel have thoroughly completed the rectification work, thereby forming an effective, unified, and efficient safety risk management system and rectification supervision mechanism.

[0010] The safety risk characteristics and violation patterns extracted from construction sites by the trained AI model can serve as the basis for subsequent AI model identification of safety risks and violations at the construction site. This method achieves comprehensive modeling of the construction site through the combined modeling of multimodal data and BIM models. This integration of BIM technology and the AI ​​model can identify safety risks and violations in substation infrastructure, track their rectification, and more.

[0011] Preferably, the BIM model is a high-precision BIM model.

[0012] Preferably, the method of collecting the multimodal data of the construction site specifically includes: Collect video surveillance data and / or operation monitoring data from the construction site through video surveillance equipment; Collect equipment status data at the construction site through equipment sensors; Environmental monitoring data from the construction site is collected through environmental sensors.

[0013] Preferably, the environmental sensors include temperature sensors, humidity sensors, vibration sensors and the like.

[0014] Preferably, before adopting the BIM model as the three-dimensional digital base map of the construction site and fusing the multimodal data with the BIM model to obtain the BIM model, the method further includes: preprocessing the multimodal data.

[0015] Preferably, the multimodal data is preprocessed, including operations such as image enhancement, denoising, or feature extraction. Preprocessing the multimodal data can provide high-quality data support for subsequent integration with the BIM model and training and deployment of the AI ​​large model. The multimodal data is preferably preprocessed before being integrated with the BIM model for fusion modeling and training of the AI ​​large model.

[0016] Preferably, the preprocessed multimodal data can be used to train the AI ​​big model or participate in the training of the AI ​​big model.

[0017] Preferably, the AI ​​large model includes: a multimodal analysis model for training based on the multimodal data to extract safety risk characteristics and violation behavior patterns of the construction site; The diagnostic model is used to identify the safety risks and illegal behaviors at the construction site in real time based on the extracted safety risk characteristics and illegal behavior patterns at the construction site.

[0018] Preferably, the multimodal analysis model is deployed on an intranet cloud or a controller or a server.

[0019] Preferably, the diagnostic model is deployed in the edge device of the construction site. During implementation, the AI ​​large model can be divided into a multimodal analysis model and a diagnostic model with distinct functions and deployed separately to achieve division of labor and cooperation between the two and separate deployment. The diagnostic model can be deployed in the edge device closer to the construction site, while the multimodal analysis model can make full use of the powerful computing power and resources of the intranet cloud or controller or server. Separate deployment can avoid the recognition and analysis models from forming a bloated and inflexible whole. The separate deployment of the multimodal analysis model and the diagnostic model can also enable the multimodal analysis model to be continuously optimized and updated according to the analysis data, and the diagnostic model can perform diagnostic identification work based on the results of the updated optimization of the multimodal analysis model, thereby adapting to the dynamically changing scenario requirements in the construction environment.

[0020] Preferably, the safety risks and violations identified in real time by the AI ​​big model specifically include: Personnel violations: Risky behaviors of personnel violating safety regulations at the construction site, including not wearing safety helmets, not wearing safety belts when working at height, and standing under the crane arm; Equipment safety risks: The equipment risk status at the construction site, including abnormal operating status of mechanical equipment, idle equipment, etc. Operation plan risks: Unplanned operations, out-of-scope operations, and dangerous operations are identified by comparing the actual operation conditions at the construction site with the planned operation conditions.

[0021] Preferably, after the AI ​​large model identifies safety risks and violations at the construction site in real time in response to the AI ​​large model, generates corresponding warning information, and positions and displays the warning information in three dimensions through the fused BIM model, the method further includes: The generated alarm information is pushed to the terminal device of the construction site manager, and the AI ​​big model continuously tracks the rectification progress of the construction site.

[0022] Preferably, the generated alarm information is pushed to the terminal device of the construction site manager, and the AI ​​big model continuously tracks the rectification progress of the construction site, specifically including: Alarm push: Push the generated alarm information to the terminal device of the construction site manager. The alarm information includes alarm type, occurrence time, screenshots and alarm content; On-site rectification: Construction site management personnel carry out rectification and disposal of the construction site based on the alarm information, and record the rectification process and results; Correction tracking: The AI ​​large model is set to continuously track and monitor the progress of the correction and treatment of the construction site. In response to identifying that the safety risks and violations corresponding to the alarm information no longer exist, the BIM model stops positioning and displaying the alarm information, thereby ensuring that the safety risks and violations are completely resolved. The continuous tracking and identification of the AI ​​large model and the continuous three-dimensional display of the alarm information by the BIM model until it is identified that the rectification has been completed can form an effective supervision mechanism for the rectification work, avoid incomplete, incomplete, and slow-down rectification, and strengthen the avoidance mechanism of safety risks and violations at the construction site.

[0023] Preferably, the diagnostic model is a lightweight diagnostic model.

[0024] Preferably, after pushing the generated alarm information to the terminal device of the construction site manager, the method may further include: Record alarm information and rectification records, perform data analysis and optimization, and feed the results of this analysis and optimization back to the AI ​​model. The AI ​​model then optimizes based on this feedback to improve its performance and recognition accuracy. Feeding this data analysis and optimization back to the AI ​​model for optimization avoids repeated deployments and enables adaptation to dynamically changing scenarios in the construction environment.

[0025] Preferably, the data analysis and optimization results are fed back to the multimodal analysis model to enable the multimodal analysis model to extract or update the safety risk characteristics and violation behavior patterns. In this way, the diagnostic model can carry out diagnostic identification work based on the latest extracted or updated safety risk characteristics and violation behavior patterns, avoiding excessive coupling of the multimodal analysis model and the diagnostic model, and helping to adapt to the dynamically changing scenario requirements in the construction environment. Of course, if necessary, it is also possible to set the data analysis and optimization results to be fed back to the diagnostic model together. This feedback mechanism can flexibly set specific feedback to the specific model unit in the AI ​​large model.

[0026] In a second aspect of the present disclosure, a system for identifying risks in substation infrastructure is provided, the system comprising: Data acquisition module: used to collect multimodal data of the construction site, the multimodal data including video monitoring data, operation monitoring data, equipment status data and environmental monitoring data; BIM business module: used to use the BIM model as the three-dimensional digital base map of the construction site, fuse the multimodal data with the BIM model to obtain the fused BIM model, so as to achieve comprehensive modeling of the construction site, thereby providing business support for the three-dimensional visualization of safety risks and violations at the construction site through the BIM model; Model business module: used to input the multimodal data into a pre-trained AI big model, extract the safety risk characteristics and violation behavior patterns of the construction site through the AI ​​big model, and the AI ​​big model identifies the safety risks and violations of the construction site in real time based on the extracted safety risk characteristics and violation behavior patterns; Alarm service module: used to respond to the AI ​​large model to identify safety risks and violations at the construction site in real time, generate corresponding alarm information, and position and display the alarm information in three-dimensional form through the BIM model after fusion modeling.

[0027] Preferably, the model business module includes: A cloud computing unit can be used for deployment, training, management, and optimization of a multimodal analysis model, wherein the multimodal analysis model is trained based on the multimodal data to extract safety risk characteristics and violation behavior patterns of the construction site; The edge computing unit can be used to deploy a diagnostic model, which can be used to realize real-time identification of the safety risks and illegal behaviors based on the extracted safety risk characteristics and illegal behavior patterns.

[0028] Preferably, the system may further include: The data preprocessing module can be used to perform preprocessing operations on the multimodal data of the construction site; Preferably, the data acquisition module can also be used to realize real-time video monitoring of the construction site, superimpose AI recognition frames and alarm information.

[0029] Preferably, the system may further include: The tracking module is used to push the generated alarm information to the terminal device of the construction site manager and continuously track the progress of the rectification of the construction site through the AI ​​big model. Based on the AI ​​big model, the tracking module 500 continuously tracks the alarm information until the alarm information is completely rectified.

[0030] Preferably, the system may further include: The data analysis module is used to record alarm information and rectification records, perform data analysis and optimization, and feed back the data analysis and optimization results to the AI ​​large model. The AI ​​large model is optimized based on this feedback, which can dynamically improve the performance and recognition accuracy of the AI ​​large model. The data analysis module performs statistical analysis on safety risks and violations at the construction site, provides trend results (safety risk trends, violations, etc.) derived from the statistical analysis, and feeds them back to the model business module, assisting the model business module in optimizing the AI ​​large model. The data analysis module is preferably connected to the model business module.

[0031] Preferably, the system may further include: The front-end display module provides a user interface that intuitively displays multimodal construction site data, statistical analysis, alarm records, relevant BIM models, and 3D visualization of alarm information using the BIM models. Multimodal data includes video surveillance data, operational monitoring data, equipment status data, and environmental monitoring data. The front-end display module can be a graphical user interface, display screen, or the like.

[0032] In one embodiment, the system further comprises: The data storage module is used to store video data, sensor data, BIM model data, and alarm records from the construction site. The data storage module can be a database, etc.

[0033] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying risks of substation infrastructure as described above is implemented.

[0034] In a fourth aspect of the present disclosure, an electronic device is provided, which includes a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the above-mentioned method for identifying risks of substation infrastructure.

[0035] Beneficial effects of the present disclosure: The present disclosure proposes a method, system and medium for identifying risks in substation infrastructure, which realizes accurate identification and dynamic monitoring of safety risks and violations at the construction site by deeply integrating BIM model structural information with the semantic understanding and image recognition capabilities of AI big models, thereby improving the safety management level and response efficiency of substation construction project sites, and conducting more comprehensive screening and rectification of various human or non-human risk factors at the construction site. The present disclosure uses multimodal data and BIM models to comprehensively model the construction site, and based on the combination of BIM technology and AI big models, it can identify, track and rectify safety risks and violations of substation infrastructure. The present disclosure can solve the problems of the lack of a unified and efficient safety risk management system for substation infrastructure, the lack of data integration after the deployment of various sensing devices, and the poor accuracy and generalization ability of the rule matching method in identifying risks and violations at the construction site. Compared with the existing technology, the present disclosure has the following advantages: (A) Improving recognition accuracy: By introducing a large AI model, a multimodal analysis model is used to train multimodal data from construction sites and extract safety risk characteristics and violation behavior patterns. A diagnostic model is then used to identify and monitor safety risks and violations in real time based on these safety risk characteristics and violation behavior patterns. This significantly improves the recognition accuracy of safety risks and violations at construction sites. The initial target recognition accuracy is no less than 75%, and will gradually increase to over 85% over time.

[0036] (B) Reduce operation and maintenance costs: Intelligent equipment can replace manual labor to achieve efficient and comprehensive hidden danger inspections, reduce the workload of manual inspections, and reduce operation and maintenance costs.

[0037] (C) Improve management: Through the AI ​​big model, the progress of rectification of safety risks and violations at the construction site is continuously tracked and monitored. The display of alarm information will not be stopped until the rectification is complete. This can achieve an upgrade from passive rectification to active prevention. Through real-time monitoring, data analysis and intelligent guidance, the safety, quality and progress management level of the construction site can be comprehensively improved.

[0038] (D) Data visualization: Based on the three-dimensional visualization function of the BIM model, alarm information can be located and displayed, and safety risks and violations at the construction site can be intuitively displayed, allowing managers to quickly locate problem areas.

[0039] (E) Model Optimization: Relying on the optimization capabilities of large AI models, the recognition accuracy and efficiency of this disclosure in specific scenarios can be continuously optimized based on the training results of multimodal analysis models on multimodal data. For example, model tuning can be performed for specific risk points in substation civil construction, electrical installation stages, and cable tunnels. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are intended to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the accompanying drawings: Figure 1 This is a flowchart diagram of a method for identifying risks of substation infrastructure in Example 1 of the present disclosure.

[0041] Figure 2 This is a flowchart diagram of a method for identifying risks of substation infrastructure in Example 1 of the present disclosure. Steps S5 and S6 are optional steps.

[0042] Figure 3 This is a structural framework of a substation infrastructure risk identification system in Example 2 of the present disclosure. Figure 1 .

[0043] Figure 4 This is a structural framework of a substation infrastructure risk identification system in Example 2 of the present disclosure. Figure 2 —Tracking business module and data analysis module are optional.

[0044] Figure 5 This is a logical diagram of a method for identifying risks in substation infrastructure construction described in the present disclosure. DETAILED DESCRIPTION

[0045] The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0046] The following detailed descriptions are all exemplary descriptions and are intended to provide further detailed descriptions of the present disclosure. Unless otherwise specified, all technical terms used in the present disclosure have the same meanings as those commonly understood by those skilled in the art to which this application belongs. The terms used in the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure.

[0047] Example 1: like Figure 1 As shown, the present disclosure provides a method for identifying risks in substation infrastructure, the method comprising: S1 collects multimodal data of the construction site, wherein the multimodal data includes video surveillance data, operation monitoring data, equipment status data, and environmental monitoring data; S2 uses the BIM model as a three-dimensional digital base map of the construction site, fuses the multimodal data with the BIM model to obtain the fused BIM model, and thus realizes comprehensive modeling of the construction site; S3 inputs the multimodal data into a pre-trained AI big model, extracts safety risk characteristics and violation behavior patterns at the construction site through the AI ​​big model, and the AI ​​big model identifies safety risks and violation behaviors at the construction site in real time based on the extracted safety risk characteristics and violation behavior patterns; S4 responds to the AI ​​large model's real-time identification of safety risks and violations at the construction site, generates corresponding alarm information, and uses the fused modeled BIM model to position and display the alarm information in three dimensions. During implementation, through the fusion modeling of the multimodal data and BIM model, a BIM model that fully models the construction site can be obtained, which can then assist in visually displaying alarm information, safety risks, and violations through comprehensive modeling of the construction site. It can also assist in the rectification work corresponding to the alarm information, which can be continuously displayed in three dimensions through the BIM model until the management personnel have thoroughly completed the rectification work, thereby forming an effective, unified, and efficient safety risk management system and rectification supervision mechanism.

[0048] The safety risk characteristics and violation patterns extracted from construction sites by the trained AI model can serve as the basis for subsequent AI model identification of safety risks and violations at the construction site. This method achieves comprehensive modeling of the construction site through the combined modeling of multimodal data and BIM models. This integration of BIM technology and the AI ​​model can identify safety risks and violations in substation infrastructure, track their rectification, and more.

[0049] In one embodiment, the BIM model is preferably a high-precision BIM model.

[0050] In one embodiment, the method of collecting multimodal data of the construction site specifically includes: Collect video surveillance data and / or operation monitoring data from the construction site through video surveillance equipment; Collect equipment status data at the construction site through equipment sensors; Environmental monitoring data from the construction site is collected through environmental sensors.

[0051] In one embodiment, the environmental sensor includes a temperature sensor, a humidity sensor, a vibration sensor, and the like.

[0052] In one embodiment, before S2, the method further includes: preprocessing the multimodal data.

[0053] In one embodiment, preprocessing the multimodal data includes operations such as image enhancement, denoising, or feature extraction. Preprocessing the multimodal data can provide high-quality data support for subsequent integration with the BIM model and training and deployment of the AI ​​large model. The multimodal data is preferably preprocessed before being integrated with the BIM model for fusion modeling and training of the AI ​​large model.

[0054] In one embodiment, the AI ​​big model includes: A multimodal analysis model, configured to be trained based on the multimodal data to extract safety risk characteristics and violation behavior patterns of the construction site; The diagnostic model is used to identify the safety risks and illegal behaviors at the construction site in real time based on the extracted safety risk characteristics and illegal behavior patterns at the construction site.

[0055] In one embodiment, the multimodal analysis model is deployed on an intranet cloud or a controller or a server.

[0056] In one embodiment, the diagnostic model is deployed in the edge device of the construction site. During implementation, the AI ​​large model can be divided into a multimodal analysis model and a diagnostic model with distinct functions and deployed separately to achieve division of labor and cooperation between the two and separate deployment. The diagnostic model can be deployed in the edge device closer to the construction site, while the multimodal analysis model can make full use of the powerful computing power and resources of the intranet cloud or controller or server. Separate deployment can avoid the recognition and analysis models from forming a bloated and inflexible whole. The multimodal analysis model and the diagnostic model are deployed separately, and the multimodal analysis model can be continuously optimized and updated according to the analysis data, and the diagnostic model can perform diagnostic identification work based on the results of the updated optimization of the multimodal analysis model, thereby adapting to the dynamically changing scenario requirements in the construction environment.

[0057] In one embodiment, the safety risks and violations identified in real time by the AI ​​big model specifically include: Personnel violations: Risky behaviors of personnel violating safety regulations at the construction site, including not wearing safety helmets, not wearing safety belts when working at height, and standing under the crane arm; Equipment safety risks: The equipment risk status at the construction site, including abnormal operating status of mechanical equipment, idle equipment, etc. Operation plan risks: Unplanned operations, out-of-scope operations, and dangerous operations are identified by comparing the actual operation conditions at the construction site with the planned operation conditions.

[0058] In one embodiment, Figure 2As shown, after S4 responds to the AI ​​large model identifying safety risks and violations at the construction site in real time, generating corresponding alarm information, and positioning and displaying the alarm information in three dimensions through the BIM model after fusion modeling, the method further includes: S5. Push the generated alarm information to the terminal device of the construction site manager, and the AI ​​big model continuously tracks the rectification progress of the construction site.

[0059] In one embodiment, the generated alarm information is pushed to the terminal device of the construction site manager, and the AI ​​large model continuously tracks the rectification progress of the construction site, specifically including: Alarm push: Push the generated alarm information to the terminal device of the construction site manager. The alarm information includes the alarm type, occurrence time, screenshot and alarm content; On-site rectification: Construction site management personnel carry out rectification and disposal of the construction site based on the alarm information, and record the rectification process and results; Correction tracking: The AI ​​large model is set to continuously track and monitor the progress of the correction and treatment of the construction site. In response to identifying that the safety risks and violations corresponding to the alarm information no longer exist, the BIM model stops positioning and displaying the alarm information, thereby ensuring that the safety risks and violations are completely resolved. The continuous tracking and identification of the AI ​​large model and the continuous three-dimensional display of the alarm information by the BIM model until it is identified that the rectification has been completed can form an effective supervision mechanism for the rectification work, avoid incomplete, incomplete, and slow-down rectification, and strengthen the avoidance mechanism of safety risks and violations at the construction site.

[0060] In one embodiment, the diagnostic model is preferably a lightweight diagnostic model.

[0061] In one embodiment, Figure 2 As shown, after the generated alarm information is pushed to the terminal device of the construction site manager and the AI ​​large model continues to track the rectification progress of the construction site, the method may further include: S6. Record alarm information and rectification records, perform data analysis and optimization, and feed the results of the data analysis and optimization back to the AI ​​model. The AI ​​model then optimizes based on the feedback to improve its performance and recognition accuracy. Feeding the data analysis and optimization results back to the AI ​​model for optimization avoids repeated deployments and enables adaptation to dynamically changing scenarios in the construction environment.

[0062] In one embodiment, the data analysis and optimization results are preferably fed back to the multimodal analysis model to enable the multimodal analysis model to extract or update the safety risk characteristics and violation behavior patterns. In this way, the diagnostic model can carry out diagnostic identification work based on the latest extracted or updated safety risk characteristics and violation behavior patterns, avoiding excessive coupling between the multimodal analysis model and the diagnostic model, and helping to adapt to the dynamically changing scenario requirements in the construction environment. Of course, if necessary, it is also possible to set the data analysis and optimization results to be fed back to the diagnostic model together. This feedback mechanism can flexibly set specific feedback to the specific model unit in the AI ​​large model.

[0063] In one embodiment, when integrating the multimodal data with the BIM model, the BIM model may be represented by a property graph G = (V, E, A) to preserve the spatial and semantic topological relationships between components: Node set V: Each construction component (such as beams, columns, slabs, etc.) is treated as a graph node. Graph node attributes include the component's size, coordinates, material information, construction stage, etc. Edge set E: Edges are used to represent the spatial connection relationship between components (such as "connection", "containment", "adjacency") and the dependency relationship in the construction process (such as "construction sequence"); Adjacency matrix A: used to represent the connection strength between nodes, which can be initialized as: Binary adjacency (whether directly connected); Or a weighted value calculated based on geometric distance, reflecting the intensity of spatial influence between components. This graph structure truly reflects the component topology, spatial constraints, and construction logic in the BIM model.

[0064] In one embodiment, in order to extract the spatial semantic features between components in the graph structure of the BIM model, a graph attention network (GAT) can be used to perform layer-by-layer feature learning of component nodes in the BIM model.

[0065] In one embodiment, a graph attention network (GAT) may be used to learn layer-by-layer features of component nodes in a BIM model, which may specifically include the following steps: (Step A) Update the component node characteristics, refer to the following formula 1: ; (Formula 1) In formula 1: : represents the features of node i in the l+1th layer; : represents the activation function; N(i): represents the set of all adjacent nodes connected to node i; j∈N(i): represents the node j belongs toi Neighbor nodes of : represents the characteristics of the neighbor node j in the lth layer; : represents the attention coefficient of neighbor node j to node i in the lth layer; : represents the learnable weight matrix of layer l; (Step B) Calculate the attention coefficient, referring to the following formula 2: ; (Formula 2) In formula 2: α ij : represents the learnable attention coefficient (score); N(i): represents the set of all adjacent nodes connected to node i; k∈N(i): represents the node k belongs to i Neighbor nodes of α T : represents the transpose of vector α; W : Learnable linear transformation matrix; h i : represents the feature vector node i ; h j : represents the feature vector node j; h k : represents the feature vector node k; : represents vector splicing; ‌LeakyReLU : represents the activation function; This mechanism allows the model to automatically identify the risk transmission paths and strengths between key components; (Step C) Global graph pooling can be performed to aggregate the final features of all component nodes to obtain the structural vector representation of the BIM model as shown in Formula 3: ; (Formula 3) in: : Vectorized representation of BIM model structure, which can be used for subsequent fusion modeling and risk analysis with multimodal data; : Average pooling, obtaining a global representation of the entire graph; : The feature representation of each node output by the last layer (layer L) of the graph neural network; L : number of GNN layers; : Contains the set of feature vectors output by all nodes in the graph at the last layer; : The total number of nodes in the graph.

[0066] In one embodiment, the fusion modeling method of the multimodal data and the BIM model can be as follows: (Step I) Cross-modal feature alignment: Refer to the following formula 4 to uniformly map the different modal features to the embedding space of the same dimension; ； (Formula 4); In formula 4: : The aligned features of each modality unify the dimensions and facilitate subsequent fusion; 、 : is the mode-specific linear mapping parameter; : The original feature vector extracted from each modality (BIM, Video, Sensor); m : Indicates the index of the modality, which can be BIM, Video or Sensor; BIM : represents the BIM model; Video : Represents video frame data, which can be extracted through CNN; Sensor : represents time series sensor data, which can be extracted by LSTM;

[0067] (Step II) Adaptive Attention Fusion Mechanism: Design a Graph Attention Network (GAT) to perform weighted fusion of features from each modality in multimodal data, and learn the contribution of different modalities to the current risk prediction: ； (Formula 5) ； (Formula 6) In Formula 5 and Formula 6: : modality attention weight; v : Learnable context vector; v T :vector v The transpose of n :represent BIM , Video , Sensor ; m : Indicates the index of the modality, which can be BIM, Video or Sensor; : is the global attention layer parameter; R: set of real numbers; d: dimension; Z fusion : The fused multimodal feature vector.

[0068] In one embodiment, the AI ​​big model identifies the safety risks and illegal behaviors and generates corresponding warning information, which can be achieved by referring to the following formula 7. Input a classifier to calculate the risk probability of a specific component in a construction site or BIM model: ; (Formula 7) In formula 7: : is the output risk prediction probability value, which can be used to trigger the AI ​​model to respond and identify safety risks and violations at the construction site; : is the Sigmoid activation function; b p : A learnable bias term used for the final prediction; W p : The learnable weight matrix used for the final prediction; Z fusion : The fused multimodal feature vector. This risk prediction and alert output method can output a probability value between 0 and 1, with a set trigger threshold (e.g., 0.5). When the probability value exceeds the trigger threshold, a risk alert is triggered—indicates that the AI ​​model has identified safety risks and violations at the construction site.

[0069] Example 2: Embodiment 2 of the present disclosure provides a system for identifying risks in substation infrastructure, the system comprising: Data acquisition module 100: used to collect multimodal data of the construction site, the multimodal data including video monitoring data, operation monitoring data, equipment status data and environmental monitoring data; BIM business module 200: used to use the BIM model as a three-dimensional digital base map of the construction site, fuse the multimodal data with the BIM model to obtain the fused BIM model, so as to achieve comprehensive modeling of the construction site, thereby providing business support for the three-dimensional visualization of safety risks and violations at the construction site through the BIM model; Model business module 300: used to input the multimodal data into a pre-trained AI large model, extract the safety risk characteristics and violation behavior patterns of the construction site through the AI ​​large model, and the AI ​​large model identifies the safety risks and violations of the construction site in real time based on the extracted safety risk characteristics and violation behavior patterns; Alarm service module 400: used to respond to the AI ​​large model to identify safety risks and violations at the construction site in real time, generate corresponding alarm information, and position and display the alarm information in three-dimensional form through the BIM model after fusion modeling.

[0070] In one embodiment, the model business module 300 includes: The cloud computing unit 301 can be used for the deployment, training, management, and optimization of a multimodal analysis model. The multimodal analysis model is trained based on the multimodal data to extract safety risk characteristics and violation behavior patterns of the construction site. The edge computing unit 302 can be used to deploy a diagnostic model, which can be used to realize real-time identification of the safety risks and illegal behaviors based on the extracted safety risk features and illegal behavior patterns.

[0071] In one embodiment, the data acquisition module 100 can also be used to implement real-time video monitoring of the construction site, superimpose AI recognition frames and alarm information.

[0072] In one embodiment, the system may further include: The tracking module 500 is used to push the generated alarm information to the terminal device of the construction site manager and continuously track the progress of the rectification of the construction site through the AI ​​big model. Based on the AI ​​big model, the tracking module 500 continuously tracks the alarm information until the alarm information is completely rectified.

[0073] In one embodiment, the system may further include: The data analysis module 600 is used to record alarm information and rectification records, perform data analysis and optimization, and feed back the data analysis and optimization results to the AI ​​large model. The AI ​​large model is optimized based on the feedback, which can dynamically improve the performance and recognition accuracy of the AI ​​large model. The data analysis module 600 performs statistical analysis on safety risks and violations at the construction site, provides trend results (safety risk trends, violations, etc.) obtained from the statistical analysis, and feeds them back to the model business module 300, assisting the model business module 300 in optimizing the AI ​​large model. The data analysis module 600 is preferably connected to the model business module 300.

[0074] In one embodiment, the system may further include: The front-end display module provides a user interface that intuitively displays multimodal construction site data, statistical analysis, alarm records, relevant BIM models, and 3D visualization of alarm information using the BIM models. Multimodal data includes video surveillance data, operational monitoring data, equipment status data, and environmental monitoring data. The front-end display module can be a graphical user interface, display screen, or the like.

[0075] In one embodiment, the system may further include: The data storage module is used to store video data, sensor data, BIM model data, and alarm records from the construction site. The data storage module can be a database, etc.

[0076] During implementation, the data acquisition module 100, the BIM business module 200, the model business module 300, the alarm business module 400, the tracking business module 500 and the data analysis module 600 can be used to execute S1, S2, S3, S4, S5 and S6 in Example 1 respectively. The front-end display module provides users with user interaction interface services, and can cooperate with other modules to provide users with relevant data display or operation menu display, etc., so that users can intuitively view or operate the system. The data storage module provides data storage and other services for the business of each module. It is worth noting that the system described in Example 2 is only a system implementation of the substation infrastructure risk identification method, and does not limit the substation infrastructure risk identification method to rely on the system described in Example 2.

[0077] Example 3: Embodiment 3 of the present disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for identifying risks of substation infrastructure as described in embodiment 1 is implemented. Or implement the substation infrastructure risk identification system as described in Example 2.

[0078] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0079] Example 4: Embodiment 4 of the present disclosure provides an electronic device comprising a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the method for identifying risks of substation infrastructure described in embodiment 1.

[0080] Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory and read-only memory (ROM).

[0081] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0083] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0085] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0086] In summary, the methods, systems, and media for identifying risks in substation infrastructure provided by Examples 1-4 of the present disclosure achieve accurate identification and dynamic monitoring of safety risks and violations at the construction site by deeply integrating BIM model structural information with the semantic understanding and image recognition capabilities of the AI ​​big model, thereby improving the safety management level and response efficiency of substation construction project sites, and conducting more comprehensive screening and rectification of various human or non-human risk factors at the construction site. The present disclosure uses multimodal data and BIM models to comprehensively model the construction site, and based on the combination of BIM technology and AI big models, it can identify, track, and rectify safety risks and violations of substation infrastructure.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and not to limit them. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present disclosure can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present disclosure should be included in the scope of protection of the claims of the present disclosure.

Claims

1. A method for identifying risks in substation infrastructure construction, characterized in that: The method comprises: Collecting multimodal data of the construction site, the multimodal data including video surveillance data, operation monitoring data, equipment status data, and environmental monitoring data; Using the BIM model as a three-dimensional digital base map of the construction site, fusing the multimodal data with the BIM model to obtain the fused BIM model; Inputting the multimodal data into a pre-trained AI big model, extracting safety risk characteristics and violation behavior patterns at the construction site through the AI ​​big model, and the AI ​​big model identifying safety risks and violation behaviors at the construction site in real time based on the extracted safety risk characteristics and violation behavior patterns; In response to the AI ​​large model identifying safety risks and violations at the construction site in real time, corresponding alarm information is generated and the alarm information is positioned and displayed in three dimensions through the BIM model after fusion modeling.

2. The method for identifying risks of substation infrastructure construction according to claim 1, wherein: The method of collecting multimodal data of the construction site specifically includes: Collect video surveillance data and / or operation monitoring data from the construction site through video surveillance equipment; Collect equipment status data at the construction site through equipment sensors; Collect environmental monitoring data from the construction site through environmental sensors; The environmental sensors include a temperature sensor, a humidity sensor, and a vibration sensor.

3. The method for identifying risks of substation infrastructure construction according to claim 1, wherein: Before adopting the BIM model as the three-dimensional digital base map of the construction site and fusing the multimodal data with the BIM model to obtain the BIM model, the method further includes: The multimodal data is preprocessed.

4. The method for identifying risks of substation infrastructure construction according to claim 3, wherein: The multimodal data is preprocessed, including image enhancement, denoising, or feature extraction operations.

5. The method for identifying risks of substation infrastructure construction according to claim 1, wherein: The AI ​​big model includes: A multimodal analysis model, configured to be trained based on the multimodal data to extract safety risk characteristics and violation behavior patterns of the construction site; The diagnostic model is used to identify the safety risks and illegal behaviors at the construction site in real time based on the extracted safety risk characteristics and illegal behavior patterns at the construction site.

6. The method for identifying risks of substation infrastructure construction according to claim 5, characterized in that: The multimodal analysis model is deployed on the intranet cloud or the controller or the server; The diagnostic model is deployed in the edge device at the construction site.

7. The method for identifying risks of substation infrastructure construction according to claim 1, wherein: The safety risks and violations identified in real time by the AI ​​model specifically include: Personnel violations: Risky behaviors of personnel violating safety regulations at the construction site, including not wearing safety helmets, not wearing safety belts when working at height, and standing under the crane arm; Equipment safety risks: The equipment risk status at the construction site, including abnormal operating status of mechanical equipment and idle equipment; Operation plan risk: Unplanned operations, out-of-scope operations, and dangerous operations identified by comparing the actual operation conditions at the construction site with the planned operation conditions.

8. The method for identifying risks of substation infrastructure construction according to claim 1, wherein: After generating the corresponding alarm information and positioning and displaying the alarm information in a three-dimensional form through the fused BIM model, the method further includes: Alarm push: Push the generated alarm information to the terminal device of the construction site manager. The alarm information includes the alarm type, occurrence time, screenshot and alarm content; On-site rectification: Construction site management personnel carry out rectification and disposal of the construction site based on the alarm information, and record the rectification process and results; Correction tracking: The AI ​​large model is set to continuously track and monitor the progress of the correction process at the construction site. In response to identifying that the safety risks and violations corresponding to the alarm information no longer exist, the BIM model stops positioning and displaying the alarm information, thereby ensuring that the safety risks and violations are completely resolved.

9. A substation infrastructure risk identification system, characterized by: The system comprises: A data acquisition module (100) is used to acquire multimodal data of the construction site, wherein the multimodal data includes video monitoring data, operation monitoring data, equipment status data and environmental monitoring data; BIM business module (200): used for using the BIM model as a three-dimensional digital base map of the construction site, fusing the multimodal data with the BIM model to obtain the fused BIM model, so as to achieve comprehensive modeling of the construction site; Model business module (300): used for inputting the multimodal data into a pre-trained AI big model, extracting the safety risk characteristics and violation behavior patterns of the construction site through the AI ​​big model, and the AI ​​big model identifying the safety risks and violation behaviors of the construction site in real time based on the extracted safety risk characteristics and violation behavior patterns; Alarm service module (400): used for responding to the AI ​​large model to identify safety risks and illegal behaviors at the construction site in real time, generating corresponding alarm information and positioning and displaying the alarm information in three-dimensional form through the BIM model after fusion modeling.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images