A multi-modal data display method of a BIM model

Abstract

The application relates to a BIM model multi-modal data display method and relates to the technical field of data interaction display. The BIM model multi-modal data display method obtains three-dimensional point clouds and multi-angle image data of each device, extracts a structure attribute set, constructs an industrial plant BIM model, and analyzes to obtain a device abnormal state perception index; in combination with device motion behavior data and running voiceprint signal data, behavior abnormal coordination indexes and voiceprint abnormal recognition indexes are extracted to form a running evaluation set; the running evaluation set and the device abnormal state perception index are comprehensively analyzed, and finally a device heteromeric risk index of each device graph element is calculated, the BIM model of the industrial plant is displayed in linkage through the device heteromeric risk index of each device graph element, so that coupling modeling, risk quantification and dynamic display among multi-modal abnormalities are realized, and the visual depth, abnormal recognition precision and state evolution tracking capability of BIM in device operation and maintenance are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of data interaction and display technology, specifically a method for displaying multimodal data of a BIM model. Background Technology

[0002] BIM, or Building Information Modeling, has been widely used in the planning, construction, and operation and maintenance management of industrial plants. BIM models can intuitively present information such as equipment distribution and space utilization at the spatial structure level, providing important support for the visual management and equipment deployment of industrial sites. Traditional BIM systems mainly display the geometric position and labeled attributes of equipment during the operation and maintenance phase, but it is difficult to reflect the behavior changes, structural displacements, abnormal sound patterns or vibrations during equipment operation in real time. Even if some systems attempt to access sensor data, they mostly only perform numerical overlay or alarm pop-ups, lacking the ability to model and analyze the inherent relationships between data. This results in equipment elements and multi-model data existing in silos, making it difficult to form a unified perception-driven data view, which affects the application effect of BIM models in operation and maintenance scenarios such as intelligent diagnosis and fault diagnosis.

[0003] Existing technologies, such as the patent application with publication number CN118885650A, disclose a BIM-based multimodal data display method and system, which includes separating BIM model data, assembling the separated BIM models to obtain an assembled BIM model for rendering; processing multimodal single-point data to integrate multimodal single-point data; binding the integrated multimodal single-point data with the rendering BIM model; and displaying the bound multimodal single-point data in a linked manner. The BIM-based multimodal data display method of this invention includes BIM model data-model separation, multimodal single-point data processing, and multimodal single-point data-model binding. It proposes two integration schemes for integrating two different types of multimodal data with BIM models: single-point multimodal data and homogeneous multi-point multimodal data, breaking down BIM data silos and combining multimodal data with BIM data.

[0004] Based on the above findings, the limitations of existing technologies include at least the following problems: Existing technologies fail to establish the ability to model the correlation between multimodal operational data such as behavioral changes, structural disturbances, and acoustic anomalies during equipment operation. They also lack the ability to transform multi-source dynamic data into a quantifiable, comparable, and decision-making risk assessment mechanism. For example, in an industrial plant equipment operation and maintenance scenario, a rotating device simultaneously exhibits the following anomalies: slight eccentricity of its internal rotor leads to unstable operating vibration frequencies; the inertial measurement unit reports a continuous shift in its attitude; and simultaneously, the acoustic signature signal collected by the acoustic acquisition module shows high-amplitude energy leakage near the dominant frequency. For such operating states where all three modes simultaneously exhibit abnormal characteristics, existing technologies struggle to perform joint modeling and analysis, thus ignoring the synergistic effects and risk enhancement trends among multiple anomalies. This results in coarse-grained and inaccurate anomaly identification, and an inability to form a detailed expression of the anomaly type, severity level, and evolution trend. Summary of the Invention

[0005] This invention addresses the technical problems existing in the prior art by providing a multimodal data display method for BIM models. It solves the problem that the existing technology is difficult to use for multimodal anomaly-driven element display linkage, thus making it difficult to express the entire process of equipment state evolution.

[0006] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A method for multimodal data display of a BIM model, comprising the following steps: acquiring three-dimensional point cloud data and image data of each piece of equipment in a designated industrial plant, and inputting them into a pre-trained attribute recognition model for attribute extraction processing to obtain a structural attribute set for each piece of equipment in the designated industrial plant, and constructing a BIM model of the industrial plant, including several equipment primitives; inputting the structural attribute set of each piece of equipment in the designated industrial plant into the industrial plant BIM model for comprehensive analysis to obtain the equipment anomaly perception index of each piece of equipment in the industrial plant BIM model. It also acquires real-time operational data for each piece of equipment within the designated industrial plant, including equipment motion behavior data and operational acoustic signal data, and inputs this data into the industrial plant BIM model for feature analysis. This yields an operational evaluation set for each piece of equipment in the industrial plant BIM model, including an abnormal behavior coordination index and an abnormal acoustic signal recognition index. A comprehensive analysis of the operational evaluation set and the abnormal perception index of each piece of equipment in the industrial plant BIM model is then performed to obtain the equipment heterogeneity risk index for each piece of equipment in the industrial plant BIM model. Based on the equipment heterogeneity risk index, each piece of equipment in the industrial plant BIM model is then displayed in a linked manner.

[0007] Furthermore, the specific formula for calculating the equipment heterogeneity risk index of a certain piece of equipment in the BIM model of an industrial plant is as follows: ;in, This refers to the equipment heterogeneity risk index for a specific piece of equipment in an industrial plant BIM model. This refers to the anomaly perception index of a specific piece of equipment in the BIM model of an industrial plant. These are the heterosensing adjustment coefficients stored in the database. This refers to the behavior anomaly coordination index of a certain piece of equipment in the BIM model of an industrial plant. These are the behavioral adjustment coefficients stored in the database. This refers to the voiceprint anomaly recognition index for a specific piece of equipment in an industrial plant BIM model. The voiceprint adjustment coefficients are stored in the database. These are the structural adjustment coefficients stored in the database.

[0008] Furthermore, the three-dimensional point cloud data specifically includes the voxel value and three-dimensional coordinates of each voxel point, the image data specifically includes the pixel value, two-dimensional coordinates, and corresponding depth information value of each pixel point, the structural attribute set includes a geometric attribute set, a structural construction attribute set, a material attribute set, and a structural risk attribute set, and the attribute recognition model is specifically a multi-modal fusion recognition network, which includes an encoding input layer, a feature fusion layer, a semantic construction layer, a risk modeling layer, and an attribute output layer.

[0009] Further, the specific steps for obtaining the structural attribute set of each piece of equipment within the designated industrial plant are as follows: In the encoding input layer of the multi-modal fusion recognition network, the 3D point cloud data and image data from each angle of each piece of equipment within the designated industrial plant are received and encoded separately to obtain the point cloud feature vector and image feature vector of each piece of equipment within the designated industrial plant; In the feature fusion layer of the multi-modal fusion recognition network, the point cloud feature vector and image feature vector of each piece of equipment within the designated industrial plant are fused and enhanced to obtain the fused feature vector of each piece of equipment within the designated industrial plant; In the semantic construction layer of the multi-modal fusion recognition network, the fused feature vector of each piece of equipment within the designated industrial plant is... Attribute semantic mapping is performed to obtain the attribute semantic vector of each piece of equipment in the designated industrial plant. In the risk modeling layer of the multi-mode fusion recognition network, risk expansion processing is performed on the attribute semantic vector of each piece of equipment in the designated industrial plant to obtain the attribute risk semantic vector of each piece of equipment in the designated industrial plant. In the attribute output layer of the multi-mode fusion recognition network, attribute prediction processing is performed on the attribute risk semantic vector of each piece of equipment in the designated industrial plant to obtain the geometric attribute set, structural construction attribute set, material attribute set, and structural risk attribute set of each piece of equipment in the designated industrial plant. The structural risk attribute set includes the obstacle risk index, stability risk index, corrosion risk index, and fatigue risk index.

[0010] Furthermore, the specific steps for obtaining the equipment anomaly perception index of each equipment element in the industrial plant BIM model are as follows: Read the obstruction risk index, stability risk index, corrosion risk index, and fatigue risk index of each piece of equipment in the industrial plant, and perform comprehensive analysis to obtain the equipment anomaly perception index of each piece of equipment in the industrial plant; perform correlation processing on the equipment anomaly perception index of each piece of equipment in the industrial plant based on the industrial plant BIM model to obtain the equipment anomaly perception index of each equipment element in the industrial plant BIM model.

[0011] Furthermore, the specific steps for obtaining the operational evaluation set of each equipment element in the industrial plant BIM model are as follows: Based on the industrial plant BIM model, the operational data of each piece of equipment in the industrial plant is correlated and processed to obtain the operational data of each equipment element in the industrial plant BIM model; the operational data of each equipment element in the industrial plant BIM model are comprehensively analyzed to obtain the behavior anomaly coordination index and voiceprint anomaly recognition index of each equipment element in the industrial plant BIM model.

[0012] Furthermore, the equipment motion behavior data includes vibration frequency values, inertial offset response index, attitude offset values, multi-axis phase difference index, and inertial offset loop area values. The specific steps to obtain the behavior anomaly coordination index of each equipment element in the industrial plant BIM model are as follows: The equipment motion behavior data of each equipment element in the industrial plant BIM model are comprehensively analyzed to obtain a behavior evaluation set for each equipment element in the industrial plant BIM model, including inertial configuration index and disturbance imbalance index; and the behavior evaluation set of each equipment element in the industrial plant BIM model is comprehensively analyzed to obtain the behavior anomaly coordination index of each equipment element in the industrial plant BIM model.

[0013] Furthermore, the specific formula for calculating the behavior anomaly coordination index of a certain equipment element in the industrial plant BIM model is as follows: ;in, This is the behavior anomaly coordination index for a specific equipment element in an industrial plant BIM model. This refers to the disturbance imbalance index of a specific equipment element in the BIM model of an industrial plant. These are the disturbance adjustment coefficients stored in the database. This refers to the inertial configuration index of a specific equipment element in an industrial plant BIM model. The inertial adjustment coefficients are stored in the database. These are the inhibition and regulation coefficients stored in the database.

[0014] Furthermore, the specific steps for obtaining the behavior evaluation set of each equipment element in the industrial plant BIM model are as follows: Based on the genetic algorithm, the inertial offset response index and attitude offset value of each equipment element in the industrial plant BIM model are comprehensively analyzed to obtain the inertial configuration index of each equipment element in the industrial plant BIM model; Based on the genetic algorithm, the vibration frequency value, multi-axis phase difference index, and inertial offset loop area value of each equipment element in the industrial plant BIM model are comprehensively analyzed to obtain the disturbance imbalance index of each equipment element in the industrial plant BIM model.

[0015] Furthermore, the operational acoustic signal data specifically refers to the sound pressure amplitude value at each instant. The specific steps for obtaining the acoustic anomaly identification index of each equipment element in the industrial plant BIM model are as follows: Based on the Fast Fourier Transform, the operational acoustic signal data of each equipment element in the industrial plant BIM model is comprehensively analyzed to obtain the acoustic evaluation index set of each equipment element in the industrial plant BIM model, including the acoustic energy shift index and the abnormal bandwidth expansion index; and based on the genetic algorithm, the acoustic evaluation index set of each equipment element in the industrial plant BIM model is comprehensively analyzed to obtain the acoustic anomaly identification index of each equipment element in the industrial plant BIM model.

[0016] The beneficial effects of this invention are as follows: Based on the structural attributes, behavioral data, and voiceprint signals of industrial equipment, a multi-source information fusion equipment element operation evaluation system is constructed. Specifically, by introducing a structural attribute recognition network and a real-time operation data processing mechanism, a behavioral anomaly coordination index, a voiceprint anomaly recognition index, and an equipment anomaly perception index are established. Furthermore, an equipment heterogeneous risk index is constructed by integrating these indices. This drives the color linkage of equipment element status in the BIM model, the synchronous unfolding of indicator display and subordinate information, thereby achieving continuous perception and visual expression of status changes. Moreover, the element linkage mechanism supports risk level color-coded display and interactive response, and can unfold detailed operation evaluation information when the user clicks. This realizes the closed-loop linkage of equipment status from attribute modeling, operation data collection, risk assessment, and element display, thereby achieving coupled modeling, risk quantification, and dynamic display of multi-mode anomalies. Consequently, it effectively improves the visualization depth, anomaly identification accuracy, and status evolution tracking capability of the BIM system in equipment operation and maintenance. Attached Figure Description

[0017] Figure 1 This is a flowchart of a multimodal data display method for a BIM model according to the present invention.

[0018] Figure 2 This is a flowchart illustrating the specific steps involved in obtaining the structural attribute set of each piece of equipment within a designated industrial plant in a multimodal data display method for a BIM model according to the present invention.

[0019] Figure 3This is a flowchart illustrating the specific steps involved in obtaining the behavioral anomaly coordination index of each equipment element in an industrial plant BIM model using a multimodal data display method for BIM models according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0022] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0023] Please see Figure 1This invention provides a technical solution: a method for displaying multimodal data of a BIM model, comprising the following steps: acquiring 3D point cloud data and image data of each piece of equipment within a designated industrial plant, and inputting them into a pre-trained attribute recognition model for attribute extraction processing to obtain a structural attribute set for each piece of equipment within the designated industrial plant, and constructing a BIM model of the industrial plant, including several equipment primitives; inputting the structural attribute set of each piece of equipment within the designated industrial plant into the industrial plant BIM model for comprehensive analysis to obtain the equipment anomaly perception index of each piece of equipment primitive in the industrial plant BIM model; and acquiring the operating data of each piece of equipment within the designated industrial plant in real time (in... It is important to note that "real-time" does not mean instantaneous, but rather refers to a very short time window (e.g., 200ms). This includes equipment motion behavior data and operational acoustic signal data, which are input into the industrial plant BIM model for feature analysis. This yields an operational evaluation set for each equipment element in the industrial plant BIM model, including an abnormal behavior coordination index and an abnormal acoustic signal recognition index. A comprehensive analysis of the operational evaluation set and equipment anomaly perception index for each equipment element in the industrial plant BIM model is then performed to obtain the equipment heterogeneity risk index for each equipment element. Based on the equipment heterogeneity risk index, each equipment element in the industrial plant BIM model is then displayed in a linked manner.

[0024] The 3D point cloud data specifically includes the voxel value and 3D coordinates of each voxel (with the center of the industrial plant as the origin of the coordinate system; the X-axis represents the horizontal spatial distribution direction of the industrial plant, pointing in front of the equipment or left-right in the space; the Y-axis represents the vertical spatial extension direction of the industrial plant, pointing to the right of the equipment or front-back in the space; and the Z-axis represents the vertical height change of the industrial plant, pointing directly upwards). The image data specifically includes the pixel value, 2D coordinates (with the top left corner of the image as the origin; the X-axis represents the horizontal direction of the image, increasing to the right and reflecting the pixel column index; and the Y-axis represents the vertical direction of the image, increasing downwards and reflecting the pixel row index), and corresponding depth information values ​​of each pixel. The structural attribute set includes geometric attribute set, structural construction attribute set, material attribute set, and structural risk attribute set. The attribute recognition model is specifically a multi-modal fusion recognition network (fused with PointNet++ and ResNet-50). The multi-modal fusion recognition network includes an encoding input layer, a feature fusion layer, a semantic construction layer, a risk modeling layer, and an attribute output layer.

[0025] The geometric attribute set includes: length, width, height, volume, surface area, profile curvature, and spatial envelope size.

[0026] The structural construction attribute set includes: installation method, support structure type, connection type, number of interfaces, and spatial distribution.

[0027] The material attribute set includes: main material type, surface treatment method, material thickness grade, and aging grade.

[0028] The encoding input layer is used to extract geometric and visual features from device point cloud and image data.

[0029] The feature fusion layer is used to align, fuse, and enhance point cloud features with image features into a unified fused feature vector.

[0030] The semantic construction layer is used to transform the fused features into vector representations with device attribute semantics.

[0031] The risk modeling layer is used to enhance the feature dimensions related to equipment structural risks based on attribute semantics.

[0032] The attribute output layer is used to output a complete set of attributes, including geometry, construction, materials, and structural risks.

[0033] The specific steps for constructing a BIM model of an industrial plant are as follows:

[0034] Based on the geometric attribute set, the spatial dimensions and outer contour data of each equipment component are used as input for 3D modeling to generate the corresponding geometric model. The spatial envelope dimensions are used to accurately lay out the installation position, occupation range and spatial relationship of each component in the BIM model. Then, based on the structural attribute set, the connection logic and support method between each equipment element and the plant structure or other equipment are defined. The connection relationship between the equipment and components such as pipes, foundations and supports is generated in the BIM platform, and the automatic layout of system-level equipment pipelines is realized according to the spatial distribution of interfaces. Finally, based on the material attribute set, the material, surface treatment and degradation status of each equipment are bound to the BIM element as component attributes for subsequent material list generation and visual rendering (such as metal / spraying effect simulation). The three attribute sets together support the automatic conversion of the identified industrial equipment into 3D equipment elements with visible structure, solvable connection and controllable material, and bind them to the industrial plant BIM model.

[0035] The specific steps for linking and displaying each equipment element in the industrial plant BIM model based on the equipment heterogeneity risk index are as follows: The equipment heterogeneity risk index of each equipment element in the industrial plant BIM model is compared with a preset equipment heterogeneity risk index threshold range. If the equipment heterogeneity risk index of each equipment element in the industrial plant BIM model is lower than the lower limit of the preset equipment heterogeneity risk index threshold range, the equipment element is displayed in green. If the equipment heterogeneity risk index of each equipment element in the industrial plant BIM model is within the preset equipment heterogeneity risk index threshold range, the equipment element is displayed in green. The display is yellow; if the equipment heterogeneity risk index of each equipment element in the industrial plant BIM model is higher than the upper limit of the preset equipment heterogeneity risk index threshold range, the equipment element will be displayed in red; and when the user selects or clicks on any equipment element in the industrial plant BIM model, the system will simultaneously expand the operation status details window associated with the equipment element, and display the corresponding original operation data summary and the various index values ​​calculated by the equipment in the window, including but not limited to the behavior anomaly coordination index, voiceprint anomaly recognition index, equipment anomaly perception index and the finally generated equipment heterogeneity risk index.

[0036] The specific formula for calculating the equipment heterogeneity risk index of a certain piece of equipment in the BIM model of an industrial plant is as follows: ;in, This refers to the equipment heterogeneity risk index for a specific piece of equipment in an industrial plant BIM model. This refers to the anomaly perception index of a specific piece of equipment in the BIM model of an industrial plant. These are the heterosensing adjustment coefficients stored in the database. This refers to the behavior anomaly coordination index of a certain piece of equipment in the BIM model of an industrial plant. These are the behavioral adjustment coefficients stored in the database. This refers to the voiceprint anomaly recognition index for a specific piece of equipment in an industrial plant BIM model. The voiceprint adjustment coefficients are stored in the database. These are the structural adjustment coefficients stored in the database.

[0037] It needs to be explained that the specific form of the tanh function is as follows: ,in, is a natural constant, and in this implementation example, it can be taken as 2.71, and its domain is ( , ), with a range of ( , ).

[0038] , , , The following steps can be taken to obtain the following: Using historical data, combined with the equipment anomaly perception index, behavior anomaly coordination index, and voiceprint anomaly recognition index, statistical regression analysis is performed to quantify the specific impact of each factor on the equipment heterogeneity risk index, thereby fitting initial weight values. Secondly, sensitivity analysis is used to adjust the value range of each coefficient and observe its impact on the equipment heterogeneity risk assessment results to ensure the stability and rationality of the model. Based on the characteristics of the equipment and the actual situation, the initially fitted coefficients are corrected and optimized, and finally, coefficient values ​​applicable to specific equipment are determined.

[0039] Specifically, such as Figure 2 As shown, the specific steps to obtain the structural attribute set of each piece of equipment within the industrial plant are as follows:

[0040] In the encoding input layer of the multi-modal fusion recognition network, 3D point cloud data of each device within a designated industrial plant and image data from each angle are received and encoded separately. (The point cloud data is input into a voxel feature extraction sub-network built on the PointNet++ network structure. First, the original voxel point set is spatially downsampled using the Farthest Point Sampling algorithm to retain representative backbone voxel points, reducing redundancy and maintaining the global geometric distribution. Subsequently, a local neighborhood is constructed around each sampling point, and the data is processed by Ball...) The query, spherical neighborhood search, selects a set of voxel points within a fixed radius to form local substructures. Within each local substructure, relative position encoding aligns the positions of neighboring points with the center point. Then, voxel values ​​and coordinate information are input as joint features to the local feature learning module, which includes a three-layer pointwise multilayer perceptron (MLP) to extract local spatial geometric relationships and point-to-point structural distribution features. Subsequently, max pooling is used to aggregate local features, obtaining a local representation for each central sampling point. These multi-level local features are stacked layer by layer to construct a global point cloud feature tensor. In the final output layer, a fixed-length point cloud feature vector is obtained through fully connected mapping, used to characterize the volumetric morphology, edge structure, support contour, and local structural variations of the device components. Image data includes image sequences covering multiple viewpoints of the device. Each image is input as an independent frame sequentially into an image feature extraction subnetwork built on ResNet-50. This subnetwork employs a shared... The weighted structure ensures the consistency and comparability of features from multiple perspectives. Each image first extracts low-level visual features such as color edges and surface textures through the first two convolutional modules, and then enters the residual block structure. Material texture patterns, boundary continuity, and color difference distribution information are extracted from multiple residual units. At the same time, the depth values ​​and spatial positions in the depth channel are fused into the intermediate feature map through positional encoding to enhance the representation of complex surface states such as uneven areas, corrosion spots, and contamination boundaries. All residual block output features are processed by global average pooling to generate fixed-dimensional image feature vectors. Then, the image feature vectors from all perspectives are input to the angle fusion module. Through attention-weighted fusion or angle smoothing averaging, a unified image feature vector is generated to comprehensively express the surface material type, color consistency, and surface degradation signs of equipment components, thus obtaining the point cloud feature vector and image feature vector for each piece of equipment in the specified industrial plant.

[0041] In the feature fusion layer of the multimodal fusion recognition network, the point cloud feature vectors and image feature vectors of each device in the designated industrial plant are fused and enhanced. (First, the two types of feature vectors are linearly transformed to make their dimensions consistent, and this is used as input to construct a fusion representation. During the fusion process, the depth information value of each pixel in the image data is introduced. The fusion weight coefficient matrix is ​​generated according to the matching relationship between the depth value and the spatial coordinates in the point cloud structure. This matrix is ​​used to guide the enhancement of channels in the image feature vector that correspond to the geometric structure. Image interference features that do not have spatial consistency are filtered out. Then, the point cloud features and the weighted image features are fused with cross-modal attention. The semantic correlation between the image channel and the point cloud channel is modeled using the cross-attention mechanism. The feature backpropagation and stability optimization are performed through the residual enhancement module. The fusion result is then adaptively adjusted for the importance of various semantic features through the channel attention mechanism. Finally, the fused feature vector of each device in the designated industrial plant is output as the input basis for component attribute recognition and structural risk inference.)

[0042] In the semantic construction layer of the multimodal fusion recognition network, the fusion feature vector of each device in the set industrial plant is subjected to attribute semantic mapping processing (the fusion feature vector is input to the attribute semantic mapping module, which is composed of a multi-layer perceptron network, i.e., MLP, which gradually enhances the semantic expressive ability of the features through continuous nonlinear transformation, and constructs a mapping channel between the fusion features and the device attributes. Subsequently, spatial position encoding is introduced in the feature propagation process, and the relative coordinate information of the device in the plant is embedded as the position into the feature expression to strengthen the correspondence between geometric attributes and spatial semantics. A local association modeling mechanism is introduced to construct logical adjacency relationships in the feature vector to simulate the combined semantics between the internal connection method, installation structure and morphological constraints of the device. Then, through feature channel grouping and interactive attention mechanism, the potential information under different semantic dimensions is classified, summarized and selectively retained to construct a high-order semantic space representation including geometric shape, construction form, material texture and layout features), to obtain the attribute semantic vector of each device in the set industrial plant.

[0043] In the risk modeling layer of the multi-modal fusion recognition network, the attribute semantic vector of each device in the set industrial plant is subjected to risk expansion processing (the attribute semantic vector is input into the risk modeling module, and adaptive reweighting is performed on each feature channel within it. Based on the changes in the feature value distribution between channels, the response of semantic components highly correlated with risk factors is automatically enhanced. Risk factors include, but are not limited to, unstable equipment connection, surface corrosion, structural fatigue, and spatial occlusion. Subsequently, the enhanced features are compressed and normalized through a nonlinear activation function to retain the differential expression between features. The enhanced features are then fused with the original attribute semantic vector using a residual connection mechanism to ensure that the integrity of the original semantics such as geometric attributes, structural attributes, and material attributes is maintained while embedding risk dimension information). This yields the attribute risk semantic vector of each device in the set industrial plant.

[0044] In the attribute output layer of the multimodal fusion recognition network, attribute prediction processing is performed on the attribute risk semantic vector of each device in the industrial plant. (The semantic vector is mapped to four independent output branches through channel partitioning and weight allocation, corresponding to the geometric attribute channel, structural construction attribute channel, material attribute channel, and structural risk attribute channel, respectively. In the geometric attribute channel, feature dimensions reflecting the volume shape and boundary structure of the device are extracted from the semantic vector. These dimensions mainly come from the local spatial structure features learned by the point cloud channel during the encoding stage, such as neighborhood density and boundary position change rate. A regression sub-network is constructed through a multilayer perceptron (MLP) to map these spatial structure semantics into explicit geometric parameters.) Numerical values ​​are used to obtain a set of geometric attributes, including: length, width, and height, corresponding to the boundary dimensions of the equipment along the principal axis in three-dimensional space; volume: derived by fitting a combination of boundary scale and morphological complexity features; surface area: estimated based on point cloud density and boundary irregularity; contour curvature: fitted by the edge curvature using the trend of inter-point structural changes; spatial envelope size: calculated by the minimum three-dimensional envelope box of the fitted boundary point set, outputting the structural enclosure range for BIM layout reference; in the structural construction attribute channel, semantic features related to the equipment support form, connection method, and interface configuration are extracted. These semantics are jointly encoded from the image and point cloud fusion features, reflecting the connection area. The boundary continuity, support location features, and topological morphology are input into a classification subnetwork. A Softmax strategy is used to perform multi-label recognition of various structural features, outputting a set of structural attributes, including: installation method (ground-mounted, suspended, embedded, or suspended); support structure type (classified as single-point, multi-point, or suspended based on the ratio of the equipment's center of gravity to the underlying support structure); connection type (classified as rigid, flexible, plug-in, or snap-fit ​​connection based on boundary cues); number and spatial distribution of interfaces (identified by feature indexing mechanism to determine the number of equipment ports and their distribution vector in three-dimensional space); and in the material attribute channel, image feature vectors generated by the image coding network are extracted, focusing on color changes, surface texture, and edges. Semantic components related to clarity are used to construct a joint output structure for classification and regression. Branch processing is performed on the shared basic feature extraction layer to output a set of material attributes, including: main material type: material classification is performed based on information such as color distribution and edge clarity, such as carbon steel, stainless steel, aluminum alloy, etc.; surface treatment method: based on the overall contrast and texture continuity in the image, it is judged as painting, electroplating, polishing, sandblasting, etc.; material thickness level: based on the semantic representation of the trend of device boundary changes, it is estimated as thin, medium or thick through regression model; aging level: based on the surface fading, texture perturbation and other features encoded in the image channel, a continuous score in the range of 0-1 is output to reflect the degree of degradation of the material surface condition.In the structural risk attribute channel, enhanced feature dimensions, derived from the risk modeling layer, are extracted from the attribute risk semantic vector. These semantic features are automatically focused on key factors related to potential equipment risks through the front-end network, such as weak installation structures, insufficient boundary contact area, surface damage, or spatial obstruction. The risk attribute channel employs a regression structure, establishing prediction branches for different categories of risk factors. Through feature compression and normalization, a continuous risk index within the range of 0 to 1 is generated for each risk factor, outputting a set of structural risk attributes, including: an obstruction risk index: based on the equipment's location in the spatial layout, interface orientation, and envelope dimensions, assessing whether the equipment obstructs passageways or hinders maintenance operations, reflecting... Spatial accessibility risk; Stability risk index: combining the equipment's support structure, connection type, and force distribution semantics, inferring its overturning or vibration resistance capabilities, representing the structural stability risk of the equipment under operation or external force interference; Corrosion risk index: derived from visual features such as surface color difference changes and edge texture disturbances encoded in the image channel, quantifying the potential risks of corrosion, rust, or oxidation degradation on the surface; Fatigue risk index: based on the combined characteristics of sharp corner areas, local stress concentration features, or areas with thin material thickness in the geometric shape, regression assessment of the potential fatigue failure risk of the structure after long-term operation, obtaining the geometric attribute set, structural construction attribute set, material attribute set, and structural risk attribute set for each piece of equipment in the set industrial plant;

[0045] The structural risk attribute set includes the obstacle risk index, stability risk index, corrosion risk index, and fatigue risk index.

[0046] The pre-training process of the multi-modal fusion recognition network is as follows:

[0047] A multimodal image dataset of labeled industrial equipment was obtained, including 3D point cloud data and multi-angle image data of each equipment component, with accurate annotation information (such as equipment bounding boxes, support connection labels, material type annotations, and local structural risk level annotations). The original data was formatted, point cloud voxelized, and image standardized and enhanced, and divided into multimodal training and validation sets to ensure balanced coverage of different component types, installation methods, and surface materials.

[0048] The multimodal fusion recognition network is initialized by randomly initializing all trainable weights using the Kaiming method. At the same time, some encoder weights pre-trained on general 3D model libraries (such as ShapeNet) and image classification datasets (such as ImageNet) are loaded to enhance the expressive power of the multimodal input channels and the cross-modal transfer performance of the model.

[0049] The network is pre-trained based on a multi-modal training set, with a training cycle set (e.g., 100 rounds). In each round of training, forward propagation (point cloud and image data are input into the network, which is then processed through encoding, fusion, semantic construction, and attribute prediction modules to output geometric, structural, material, and risk attribute results) is performed sequentially. Multi-task loss function calculation (including geometric regression loss, structural construction cross-entropy loss, material classification loss, and risk index regression loss, which are integrated into a total loss using a weighted strategy) and backpropagation and parameter update (the network parameters are iteratively updated using the AdamW optimizer combined with gradient clipping, learning rate hot start, and the Cosine Decay strategy).

[0050] During training, the model is evaluated on a multi-model validation set after each round, and the prediction results of each attribute are output and the accuracy indicators are calculated, including the mean squared error (MSE) of geometric attributes, the accuracy and IoU of construction and material classification, the mean deviation (MAE) of risk prediction and the F1 score, etc. At the same time, the training loss and indicator change curves are plotted to monitor the model convergence progress. If the performance no longer improves after several consecutive rounds of validation, the Early Stopping mechanism is activated to automatically terminate the training in advance to prevent overfitting.

[0051] Finally, the converged multi-mode fusion identification network model weight file is saved and exported as a deployment format to support component attribute identification and structural risk analysis tasks in subsequent industrial plant BIM systems.

[0052] The specific steps for obtaining the equipment anomaly perception index of each equipment element in the industrial plant BIM model are as follows: First, read the obstruction risk index, stability risk index, corrosion risk index, and fatigue risk index of each piece of equipment within the designated industrial plant, and perform a comprehensive analysis to obtain the equipment anomaly perception index for each piece of equipment within the designated industrial plant. Then, based on the industrial plant BIM model, perform association processing on the equipment anomaly perception index of each piece of equipment within the designated industrial plant (i.e., obtain the unique identification information of each piece of equipment, which is a preset equipment number, and standardize the identification format to meet the matching requirements with the equipment elements in the industrial plant BIM model). Finally, perform component element analysis on the industrial plant BIM model to extract the equipment elements belonging to the equipment. For each element instance of a class, its basic attribute fields are obtained, including element number, system affiliation, component type, installation location, etc. An element attribute structure index is established, and an extended field is added to each target equipment element in the BIM model to carry external state values ​​and receive the equipment anomaly perception index corresponding to that element. Then, based on the mapping relationship between equipment identification information and element number, the equipment anomaly perception index of each device is written to the extended field of the corresponding equipment element in the BIM model, forming a one-to-one data binding relationship. Finally, after all equipment element state values ​​are written, the BIM model is synchronized as a whole to obtain the equipment anomaly perception index of each equipment element in the industrial plant BIM model.

[0053] The specific formula for calculating the abnormal perception index of a certain piece of equipment in a designated industrial plant is as follows: ;in, To set the anomaly perception index for a specific piece of equipment within an industrial plant. To determine the obstacle risk index of a specific piece of equipment within an industrial plant, The resistance adjustment coefficient is stored in the database. To determine the stability risk index of a specific piece of equipment within an industrial plant. These are the stability adjustment coefficients stored in the database. To determine the corrosion risk index of a specific piece of equipment within an industrial plant, The corrosion adjustment coefficient is stored in the database. To determine the fatigue risk index of a specific piece of equipment within an industrial plant. The fatigue adjustment coefficients are stored in the database. These are the interaction adjustment coefficients stored in the database.

[0054] It needs to be explained that, , , , , The following steps can be taken to obtain the initial impact weights of each variable (obstruction risk index, stability risk index, corrosion risk index, and fatigue risk index) on the equipment anomaly perception index based on historical data and statistical regression analysis. Then, the range of coefficient values ​​is adjusted using sensitivity analysis to assess the stability and applicability of these parameters to the formula output. Next, the weights are further fitted through model optimization (such as machine learning algorithms) to ensure that the formula accurately reflects the anomaly perception state of the actual equipment. The coefficients are then fine-tuned based on different equipment characteristics to ensure that they are applicable to specific equipment anomaly perception assessment needs.

[0055] The following is a specific implementation example for calculating the abnormality perception index of a certain piece of equipment in a set industrial plant. The available data includes the obstacle risk index, stability risk index, corrosion risk index, and fatigue risk index of three randomly selected pieces of equipment in the set industrial plant, as shown in Table 1:

[0056] Table 1. Example of setting risk attribute set data for equipment sequence structure within an industrial plant.

[0057]

[0058] The database stores the resistance adjustment coefficient. Approximately 0.294;

[0059] Stability adjustment coefficients stored in the database Approximately 0.306;

[0060] Corrosion adjustment coefficients stored in the database Approximately 0.216;

[0061] Fatigue adjustment coefficients stored in the database Approximately 0.349;

[0062] Interaction adjustment coefficients stored in the database Approximately 1.573;

[0063] Substituting the data from Table 1 and the aforementioned adjustment coefficients into the specific formula for calculating the abnormality perception index of a certain piece of equipment within an industrial plant, we obtain:

[0064] Set the anomaly perception index of the first piece of equipment in the industrial plant as √(0.231^0.294×0.318^0.306×0.213^0.216×0.438^0.349) / exp(-1.573×0.231×0.318×0.213×0.438)≈0.502;

[0065] Set the anomaly perception index of the second piece of equipment in the industrial plant as √(0.128^0.294×0.289^0.306×0.267^0.216×0.364^0.349) / exp(-1.573×0.128×0.289×0.267×0.364)≈0.479;

[0066] Set the anomaly perception index of the third device in the industrial plant to √(0.157^0.294×0.374^0.306×0.183^0.216×0.516^0.349) / exp(-1.573×0.57×0.374×0.183×0.516)≈0.485.

[0067] In this implementation scheme, deep analysis and semantic construction of multimodal raw data of industrial equipment are performed to comprehensively extract multidimensional attribute information such as the geometric structure, connection structure, surface material, and potential structural risks of the equipment. Secondly, a multimodal recognition network integrating PointNet++ and ResNet-50 is adopted to introduce multi-angle image texture semantics while maintaining the details of the point cloud structure. This enables the final recognition result to not only have accurate spatial structure expression capabilities, but also effectively reflect the surface treatment process and degradation state of the equipment. Through the hierarchical abstraction of the semantic construction layer and the risk modeling layer, a mapping mechanism from raw perception data to high-order structural attributes is established. Finally, the output attribute set provides semantic closed-loop support for subsequent equipment status assessment, BIM component generation, and intelligent linkage display, thereby enhancing the stability, interpretability, and scalability of attribute recognition. This makes it suitable for structural modeling and multimodal data visualization application scenarios of heterogeneous equipment in complex industrial environments.

[0068] Specifically, the steps to obtain the operational evaluation set of each equipment element in the industrial plant BIM model are as follows: Based on the industrial plant BIM model, the operational data of each piece of equipment in the designated industrial plant is correlated (the logic is consistent with the correlation processing of the equipment anomaly perception index of each piece of equipment in the designated industrial plant), to obtain the operational data of each equipment element in the industrial plant BIM model; the operational data of each equipment element in the industrial plant BIM model are comprehensively analyzed to obtain the behavior anomaly coordination index and voiceprint anomaly recognition index of each equipment element in the industrial plant BIM model.

[0069] The equipment motion behavior data includes vibration frequency values, inertial offset response index, attitude offset values, multi-axis phase difference index, and inertial offset loop area values. The specific steps to obtain the behavior anomaly coordination index of each equipment element in the industrial plant BIM model are as follows: The equipment motion behavior data of each equipment element in the industrial plant BIM model are comprehensively analyzed to obtain a behavior evaluation set for each equipment element in the industrial plant BIM model, including inertial configuration index and disturbance imbalance index; and the behavior evaluation set of each equipment element in the industrial plant BIM model is then comprehensively analyzed to obtain the behavior anomaly coordination index of each equipment element in the industrial plant BIM model.

[0070] Among them, the vibration frequency value is the mechanical vibration generated by the reciprocating, rotating or colliding of the parts during the operation of the equipment, which can be obtained by vibration sensors.

[0071] The inertial bias response index is the offset intensity of the inertial response of a device during operation (such as the asymmetry or deviation behavior exhibited by the device in its inertial response). It can be obtained by acquiring the angular velocities of the three axes through a three-axis gyroscope and the acceleration values ​​of the three axes through a three-axis accelerometer, and then averaging them to obtain the mean angular velocity and mean acceleration values. Then, it is standardized and weighted based on the standardized results. The result is the inertial bias response index.

[0072] Attitude offset is the change in spatial attitude angle of the overall structure of the equipment during operation. It can be obtained by a 9-axis IMU (three-axis accelerometer, three-axis gyroscope, and three-axis magnetometer) to acquire the acceleration, angular velocity, and geomagnetic direction signals of the equipment in the X, Y, and Z axes. The raw data collected by the IMU is input into the embedded attitude calculation module. The three types of signals are fused and calculated using complementary filtering, Mahony filtering, or Madgwick algorithm to calculate the current pitch angle and roll angle of the equipment. After the equipment is first installed, the pitch angle and roll angle at that moment are recorded and marked as pitch reference angle and roll reference angle. The current pitch angle and roll angle are then compared with the pitch reference angle and roll reference angle respectively to obtain the current pitch angle difference and roll angle difference. Then, the standardization process is performed, and the result is weighted based on the standardization result. The result is the attitude offset value.

[0073] The multi-axis phase difference index quantifies the degree of phase synchronization deviation between vibration signals in each axis of an equipment during operation. It can be obtained by acquiring vibration acceleration signals in the X, Y, and Z directions of the equipment through a triaxial accelerometer and performing high-frequency synchronous sampling (≥1kHz). Then, the triaxial acceleration signals are bandpass filtered to select the main operating frequency range of the equipment (e.g., 10~500Hz) to remove DC offset and high-frequency interference. Subsequently, the main frequency phase of the triaxial signals is extracted using short-time Fourier transform (STFT) or Hilbert transform, and the real-time phase difference between any two axes is calculated. The results are weighted and processed to obtain the multi-axis phase difference index.

[0074] The inertial offset loop area value represents the relative change trend and energy asymmetry characteristics between the angular velocity and linear acceleration responses during equipment operation. It can be obtained by synchronously acquiring angular velocity and linear acceleration signals through an inertial measurement unit (i.e., a three-axis gyroscope and a three-axis accelerometer), and using the X-axis position main axis signal output. Then, the currently acquired angular velocity is used as the X-axis coordinate and the acceleration is used as the Y-axis coordinate to construct the current inertial state point on a two-dimensional feature plane, which forms an inertial trajectory arc with the point at the previous moment. This arc is continuously connected over time to form a dynamic open loop trajectory. Then, the area of ​​the current data point pair (i.e., the current inertial state point and the inertial state point at the previous moment) is calculated using infinitesimal elements. The trapezoidal method or vector cross product method is used to solve the incremental area, and the result is the inertial offset loop area value (which is 0 at the first moment).

[0075] The specific steps to obtain the behavior evaluation set of each equipment element in the industrial plant BIM model are as follows: Based on a genetic algorithm, a comprehensive analysis of the inertial offset response index and attitude offset value of each equipment element in the industrial plant BIM model is performed (i.e., standardization is performed first, and weighting is performed based on the standardization results; the weight coefficients corresponding to the inertial offset response index and attitude offset value are obtained through a genetic algorithm). Specifically, a weight coefficient search space is constructed, and the weight coefficients corresponding to the inertial offset response index and attitude offset value are denoted as A and B respectively, and they are set to satisfy the normalization constraint condition, i.e., A+B=1, while limiting their value range to 0-1 to ensure the effectiveness and distinguishability of the weight distribution. Then, based on the pre-acquired historical structure sample set, a fitness evaluation mechanism for the genetic algorithm is constructed. The historical inertial offset response index and attitude offset value of multiple equipment elements are used as input samples, combined with their known inertial configuration state labels, to construct the sample prediction error as the objective function. Specifically, the inertial configuration index calculated under each weight is compared with the reference label value. The mean squared error is used as the fitness function, i.e., the evaluation criterion for the quality of the weight combination. Next, the genetic algorithm population is initialized, with several sets of weight coefficient combinations that meet the constraints used as the initial population individuals, forming the first generation of candidate solutions. During the iteration of the genetic algorithm, individual selection, crossover, and mutation operations are performed respectively. The selection operation uses a roulette wheel or tournament method based on fitness ranking to retain excellent solutions. The crossover operation generates new individuals by randomly exchanging parent weight fragments at crossover points. The mutation operation enhances population diversity by applying small perturbations to individual weight values, preventing them from getting trapped in local optima. Subsequently, the weight coefficient combination is iteratively updated based on the fitness values ​​of each individual in each generation of the population. The optimization process terminates when the preset maximum number of iterations is reached or the fitness change of the population for several consecutive generations is lower than the convergence threshold. Finally, the weight combination with the best fitness is extracted as the optimal weight coefficient for the inertial bias response index and attitude offset value, resulting in the inertial configuration index (measuring the degree of mismatch between the inertial direction distribution and structural configuration of the equipment during operation) for each equipment element in the industrial plant BIM model.

[0076] Based on a genetic algorithm, a comprehensive analysis is performed on the vibration frequency value, multi-axis phase difference index, and inertial offset loop area value of each equipment element in the industrial plant BIM model. (The analysis is first standardized, and then weighted based on the standardized results. The weight coefficients corresponding to the vibration frequency value, multi-axis phase difference index, and inertial offset loop area value are obtained through a genetic algorithm, and the logic is consistent with that of the weight coefficients corresponding to the inertial offset response index and attitude offset value, which are obtained through a genetic algorithm.) This yields the disturbance imbalance index of each equipment element in the industrial plant BIM model.

[0077] The specific formula for calculating the behavior anomaly coordination index of a certain equipment element in an industrial plant BIM model is as follows: ;in, This is the behavior anomaly coordination index for a specific equipment element in an industrial plant BIM model. This refers to the disturbance imbalance index of a specific equipment element in the BIM model of an industrial plant. These are the disturbance adjustment coefficients stored in the database. This refers to the inertial configuration index of a specific equipment element in an industrial plant BIM model. The inertial adjustment coefficients are stored in the database. These are the inhibition and regulation coefficients stored in the database.

[0078] It needs to be explained that in the formula This factor is used to adjust the inhibitory effect of the inertial configuration index on the disturbance imbalance index, so as to avoid the abnormal behavior coordination index being too high or too low.

[0079] , , The following steps can be used to obtain the initial influence weights of each variable (disturbance imbalance index, inertial configuration index) on the behavioral abnormality coordination index based on historical data through statistical regression analysis. Then, the range of values ​​of the coefficients is adjusted using sensitivity analysis to evaluate the stability and applicability of these parameters to the formula output. Next, the weights are further fitted through model optimization (such as multi-objective optimization) to ensure that the formula can accurately reflect the abnormal state of actual behavioral abnormality coordination.

[0080] In this implementation scheme, five types of physically interpretable behavioral characteristic indicators are extracted from high-frequency inertial measurement data acquired during equipment operation. A genetic algorithm is then used to achieve the optimal weighted combination of multiple parameters, thereby constructing two intermediate evaluation quantities: the inertial configuration index and the disturbance imbalance index. Furthermore, a behavioral anomaly coordination index is derived through nonlinear coupling relationships. This allows for the characterization of the equipment's comprehensive behavioral performance in terms of attitude stability, structural dynamic response, and vibration coordination, thereby quantifying abnormal offsets and imbalance trends during equipment movement. This step also achieves the fusion modeling of multi-dimensional inertial behavioral parameters, resulting in higher stability, robustness, and dynamic recognition capabilities. This provides high-resolution behavioral characteristic support for the operational status assessment, risk modeling, and multi-mode display of equipment elements in industrial plant BIM models, effectively enhancing the model's granularity and reliability in identifying complex equipment dynamic anomalies.

[0081] Specifically, the operating acoustic signal data refers to the sound pressure amplitude at each instant within the time window. The specific steps for obtaining the acoustic anomaly recognition index of each equipment element in the industrial plant BIM model are as follows: Based on the Fast Fourier Transform, the operating acoustic signal data of each equipment element in the industrial plant BIM model is comprehensively analyzed to obtain the acoustic evaluation index set of each equipment element in the industrial plant BIM model, including the acoustic energy offset index and the abnormal bandwidth expansion index; and based on the genetic algorithm, the acoustic evaluation index set of each equipment element in the industrial plant BIM model is comprehensively analyzed (i.e., first, standardization is performed, and then weighting is performed based on the standardization results, and the weight coefficients corresponding to the acoustic energy offset index and the abnormal bandwidth expansion index are obtained through the genetic algorithm, and the logic is consistent with that of obtaining the weight coefficients corresponding to the inertial bias response index and the attitude offset value through the genetic algorithm), to obtain the acoustic anomaly recognition index of each equipment element in the industrial plant BIM model.

[0082] The specific steps for obtaining the acoustic signature evaluation index set for each equipment element in the industrial plant BIM model are as follows: The operational acoustic signature signal data of each equipment element in the industrial plant BIM model is weighted using a window function, employing a Hamming window to suppress spectral leakage. A Fast Fourier Transform (FFT) is then performed on the windowed operational acoustic signature signal data of each equipment element in the industrial plant BIM model to obtain the frequency domain energy spectrum of each equipment element, including the acoustic energy at each frequency point, used to characterize the energy distribution of the acoustic signature signal at each frequency. A weighted average is then performed on the acoustic energy at each frequency point of each equipment element in the industrial plant BIM model to obtain the spectral centroid value of each equipment element in the industrial plant BIM model. A reference spectral centroid value for each equipment element in the industrial plant BIM model is also obtained (by obtaining historical spectral centroid values ​​from several historical time periods and averaging them), and a ratio is performed (i.e., the absolute value of the spectral centroid value compared to the reference spectral centroid value). The acoustic energy offset index of each equipment element in the industrial plant BIM model is obtained by (difference / reference spectrum centroid value). The acoustic energy of each frequency point of each equipment element in the industrial plant BIM model is sorted in ascending order, and the cumulative energy ratio of each frequency point is calculated in turn. That is, the sum of the energy from the lowest frequency point to the current frequency point accounts for the proportion of the total energy in the entire frequency domain. By calculating the cumulative energy ratio of each frequency point in turn, a complete frequency cumulative energy distribution curve can be constructed. Two specific frequency points are counted, namely the lower boundary frequency point where the cumulative energy reaches 5% and the upper boundary frequency point where the cumulative energy reaches 95%. The frequency difference between the two is calculated and called the main energy distribution bandwidth of each equipment element in the industrial plant BIM model. It represents the main coverage area of ​​the equipment acoustic energy in the spectrum. The frequency point with the largest energy value is counted to determine the main frequency position. Finally, the ratio of the main energy distribution bandwidth of each equipment element in the industrial plant BIM model to the corresponding main frequency value is calculated to obtain the abnormal bandwidth expansion index.

[0083] In this implementation scheme, by converting the raw sound pressure signal during equipment operation into a frequency domain energy spectrum, efficient modeling and anomaly identification of acoustic behavior characteristics are achieved. This significantly improves the fine-grained perception capability of the industrial plant BIM system for equipment status changes. Specifically, through window function weighting and fast Fourier transform, the energy distribution of the acoustic signal at each frequency point can be accurately extracted, and highly expressive acoustic evaluation indices such as the spectral centroid and the dominant frequency bandwidth can be constructed. This effectively characterizes the energy shift characteristics and frequency diffusion trend during equipment operation. Furthermore, a genetic algorithm is introduced to weight and integrate multiple acoustic evaluation indices to construct a unified acoustic anomaly identification index. This enables adaptive fusion analysis among multiple indicators and enhances the robust identification capability for complex acoustic anomaly states, thereby providing a high-precision acoustic decision-making basis for the linkage of graphic element display.

[0084] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0090] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for multi-modal data presentation of a BIM model, characterized in that, The method comprises the following steps: Obtain three-dimensional point cloud data of each device in the designated industrial plant, and image data of each angle, and input the three-dimensional point cloud data and the image data into a pre-trained attribute recognition model for attribute extraction processing, wherein the three-dimensional point cloud data specifically includes voxel values and three-dimensional coordinates of each voxel point, and the image data specifically includes pixel values, two-dimensional coordinates and corresponding depth information values of each pixel point, and the attribute recognition model specifically includes a multi-modal fusion recognition network, and the multi-modal fusion recognition network comprises an encoding input layer, a feature fusion layer, a semantic construction layer, a risk modeling layer and an attribute output layer, so as to obtain a structural attribute set of each device in the designated industrial plant, wherein the structural attribute set comprises a geometric attribute set, a structural construction attribute set, a material attribute set and a structural risk attribute set, and the structural risk attribute set specifically comprises an obstacle risk index, a stability risk index, a corrosion risk index and a fatigue risk index. In the encoding input layer of the multi-modal fusion recognition network, the three-dimensional point cloud data of each device in the designated industrial plant and the image data of each angle are received and encoded to obtain point cloud feature vectors and image feature vectors of each device in the designated industrial plant. In the feature fusion layer of the multi-modal fusion recognition network, the point cloud feature vectors and the image feature vectors of each device in the designated industrial plant are fused and enhanced to obtain a fusion feature vector of the corresponding device. In the semantic construction layer of the multi-modal fusion recognition network, the fusion feature vector of each device in the designated industrial plant is subjected to attribute semantic mapping processing to obtain an attribute semantic vector of the corresponding device. In the risk modeling layer of the multi-modal fusion recognition network, the attribute semantic vector of each device in the designated industrial plant is subjected to risk expansion processing to obtain an attribute risk semantic vector of the corresponding device. In the attribute output layer of the multi-modal fusion recognition network, the attribute risk semantic vector of each device in the designated industrial plant is subjected to attribute prediction processing to obtain a geometric attribute set, a structural construction attribute set, a material attribute set and a structural risk attribute set of the corresponding device. The structural risk attribute set comprises an obstacle risk index, a stability risk index, a corrosion risk index and a fatigue risk index, and a BIM model of the industrial plant is constructed, comprising a plurality of device graph elements. The structural attribute set of each device in the designated industrial plant is input into the BIM model of the industrial plant for comprehensive analysis to obtain a device anomaly perception index of each device graph element in the BIM model of the industrial plant, wherein the device anomaly perception index is specifically obtained by reading the obstacle risk index, the stability risk index, the corrosion risk index and the fatigue risk index of each device in the designated industrial plant and performing comprehensive analysis on the obstacle risk index, the stability risk index, the corrosion risk index and the fatigue risk index. Based on the BIM model of the industrial plant, the device anomaly perception index of each device in the designated industrial plant is subjected to correlation processing to obtain a device anomaly perception index of each device graph element in the BIM model of the industrial plant. Real-time running data of each device in the designated industrial plant, including device motion behavior data and running voiceprint signal data, are input into the BIM model of the industrial plant for feature analysis to obtain a running evaluation set of each device graph element in the BIM model of the industrial plant, including a behavior abnormality coordination index and a voiceprint abnormality recognition index. ​ Comprehensive analysis is performed on the operation evaluation set and the equipment anomaly sensing index of each equipment graphic element in the industrial plant BIM model to obtain an equipment heterogeneity risk index of each equipment graphic element in the industrial plant BIM model; Based on the equipment heterogeneity risk index, each equipment graphic element in the industrial plant BIM model is displayed in linkage.

2. The method of claim 1, wherein, The specific formula for calculating the equipment heterogeneity risk index of a certain equipment in the industrial plant BIM model is as follows: ； wherein, 、 、 、 are, in sequence, an equipment heterogeneous risk index, an equipment heterogeneous state perception index, an abnormal behavior coordination index, and a voiceprint abnormality identification index of a certain equipment in the industrial plant BIM model, 、 、 、 are, in sequence, a heterogeneous state perception adjustment coefficient, a behavior adjustment coefficient, a voiceprint adjustment coefficient, and a structure adjustment coefficient stored in the database.

3. The method of Claim 1, wherein, The specific steps for obtaining the operation evaluation set of each equipment graphic element in the industrial plant BIM model are as follows: Based on the industrial plant BIM model, the operation data of each equipment in the set industrial plant are associated to obtain the operation data of each equipment graphic element in the industrial plant BIM model; The behavior abnormal coordination index and the voiceprint abnormal recognition index of each equipment graphic element in the industrial plant BIM model are obtained by respectively performing comprehensive analysis on the operation data of each equipment graphic element in the industrial plant BIM model.

4. The method of multi-modal data presentation of a BIM model according to claim 3, characterized in that, The specific steps for obtaining the behavior abnormal coordination index of each equipment graphic element in the industrial plant BIM model are as follows: The behavior evaluation set of each equipment graphic element in the industrial plant BIM model is obtained by respectively performing comprehensive analysis on the equipment motion behavior data of each equipment graphic element in the industrial plant BIM model, including the inertia configuration index and the disturbance imbalance index. The behavior abnormal coordination index of each equipment graphic element in the industrial plant BIM model is obtained by performing comprehensive analysis on the behavior evaluation set of each equipment graphic element in the industrial plant BIM model.

5. The method of multi-modal data presentation of a BIM model according to claim 4, characterized in that, The specific formula for calculating the behavior abnormal coordination index of a certain equipment graphic element in the industrial plant BIM model is as follows: ； wherein, 、 、 are respectively the behavior abnormal coordination index, the disturbance imbalance index, and the habit configuration index of a certain equipment graphic element in the industrial plant BIM model, 、 、 are respectively the disturbance adjustment coefficient, the habit adjustment coefficient, and the inhibition adjustment coefficient stored in the database.

6. The method of claim 4, wherein, The specific steps for obtaining the behavior evaluation set of each equipment graphic element in the industrial plant BIM model are as follows: Based on the genetic algorithm, the inertia bias response index and the attitude offset value of each equipment graphic element in the industrial plant BIM model are comprehensively analyzed to obtain the inertia configuration index of each equipment graphic element in the industrial plant BIM model. Based on the genetic algorithm, the vibration frequency value, the multi-axis phase difference index, and the inertia offset loop area value of each equipment graphic element in the industrial plant BIM model are comprehensively analyzed to obtain the disturbance imbalance index of each equipment graphic element in the industrial plant BIM model.

7. The method of Claim 3, wherein, The specific steps for obtaining the voiceprint abnormal recognition index of each equipment graphic element in the industrial plant BIM model are as follows: Based on the fast Fourier transform, the operation voiceprint signal data of each equipment graphic element in the industrial plant BIM model are comprehensively analyzed to obtain a voiceprint evaluation index set of each equipment graphic element in the industrial plant BIM model, including the voiceprint energy offset index and the abnormal bandwidth expansion index. Based on the genetic algorithm, the voiceprint evaluation index set of each equipment graphic element in the industrial plant BIM model is comprehensively analyzed to obtain the voiceprint abnormal recognition index of each equipment graphic element in the industrial plant BIM model.

Images