Cross-domain knowledge fusion system and method based on multiple features

Abstract

The invention discloses a multi-feature-based cross-domain knowledge fusion system and method, and the system comprises an equipment layer and an edge intelligent layer which comprise an exclusive feature extraction tower and an equipment portrait engine, and carries out the compression of feature vectors through an auto-encoder or a full-connection network, and obtains a compressed feature package; the cross-domain fusion center is used for sequentially performing entity extraction, relation extraction, space-time alignment and knowledge fusion on the compressed feature packet and auxiliary data from other domains to obtain fusion data and storing / updating the fusion data; and the decision application layer comprises a display terminal used for displaying fusion data, and a scheduling unit used for providing a predictive maintenance scheme early warning unit and cross-device scheduling. According to the system provided by the invention, various types of sensors can be integrated, and more comprehensive and multi-dimensional data acquisition is realized; the richness and accuracy of the data are improved, and a firmer foundation is provided for subsequent feature extraction and knowledge fusion.

Description

Technical Field

[0001] The present invention relates to the field of knowledge fusion technology, and in particular to a knowledge fusion system for large-area, discrete, heterogeneous devices / equipment located in different positions and spaces, and specifically to a cross-domain knowledge fusion system and method based on multiple features. Background Art

[0002] Knowledge fusion is the process of integrating and unifying knowledge from diverse sources and structures, aiming to eliminate redundancy and conflict and build a consistent and complete knowledge system. Knowledge graphs, on the other hand, are a technology for representing knowledge using graph structures. They organize information in the form of entity, attribute, and relationship triplets, visually demonstrating the connections between knowledge. The combination of these two technologies can effectively improve the understandability and usability of knowledge.

[0003] Knowledge graphs are currently widely used in search engines, intelligent question-answering, and recommendation systems. In industries like healthcare and finance, knowledge graphs assist in disease diagnosis and risk prediction. In the internet sector, they power semantic search and personalized recommendations. With the development of multimodal technologies, knowledge graphs are increasingly integrating data from multiple sources, such as text and images, expanding their application scenarios.

[0004] The main technical challenges facing existing knowledge fusion processes include: heterogeneity in multi-source data makes alignment difficult, and the accuracy of entity disambiguation and relationship conflict resolution still needs to be improved; the efficiency of dynamic knowledge updates is low, making it difficult to adapt to real-time changes; there is a lack of universal methods for cross-language and cross-domain knowledge fusion; and the high storage and computational overhead of large-scale knowledge graphs restricts practical application effectiveness. These challenges urgently need to be overcome through algorithm optimization and hardware collaboration. Summary of the Invention

[0005] In order to solve the problems of alignment difficulties, entity disambiguation and relationship conflicts caused by the heterogeneity of multi-source data in the prior art, the present application provides a cross-domain knowledge fusion system and method based on multiple features, which is used to solve the knowledge fusion problems of different structural types and different fields. The present invention has a significant effect on the monitoring, management and decision-making basis of large-scale, large-area, discrete and heterogeneous entities located in different positions and spaces compared with the prior art. The present invention can realize the identification and display of massive entity data through cross-domain data collection, feature extraction and knowledge fusion. Through the feature extraction tower and shared feature projection layer, multi-level and multi-dimensional feature extraction and fusion are realized, which improves the accuracy and efficiency of feature extraction and helps to more accurately describe the status and behavior patterns of equipment. At the same time, cross-domain knowledge fusion is realized through the cross-domain fusion center, and the graph is updated and stored through the innovative confidence incremental learning mechanism, which provides a basis and support for subsequent predictive maintenance and cross-device scheduling.

[0006] In order to achieve the above objectives, the technical solutions adopted in this application are: The present invention provides a cross-domain knowledge fusion system based on multiple features, including The device layer includes multiple sensors installed on devices of different types and locations to collect data from different devices. ,in, m Represents the type of installed equipment, n Represents the sequence of installed equipment; the sensor Collect and obtain the corresponding raw data , and sent to the Edge intelligence layer for feature extraction; The edge intelligence layer, including a dedicated feature extraction tower and a device profiling engine, converts the raw data sent from the device layer into The original data is processed by a dedicated feature extraction tower deployed on the device. Perform feature extraction and send the extracted feature vector to the device profiling engine. Compress the feature vector through an autoencoder or a fully connected network to obtain a compressed feature package. The cross-domain fusion center combines the compressed feature packets sent from the edge intelligence layer and auxiliary data from the environment domain, supply chain domain, operation and maintenance domain, and regulatory domain. Perform entity extraction, relationship extraction, spatiotemporal alignment, knowledge fusion in sequence, obtain fused data and store / update it; The decision-making application layer includes a display terminal for displaying fused data, and a scheduling unit for providing a warning unit for predictive maintenance solutions and cross-device scheduling.

[0007] In order to realize multi-field and multi-type equipment information collection, the sensor It is a multimodal sensor or a variety of single-modal sensors. The multimodal sensor refers to a sensor that is integrated with sensors suitable for different modalities and calls different modes to collect different information. Different equipment, different environments, and different data collection can be achieved by simply changing or not changing the multimodal sensor. Compared with single-modal sensors, multimodal sensors have higher integration, stronger processing capabilities, a wider range of collected data types, and more application scenarios, but the cost of the sensor is also higher; single-modal sensors refer to sensors provided for specific equipment or devices and / or environments. The type of information they collect is determined for specific equipment. Compared with multimodal sensors, they collect fewer types of data and have lower compatibility, but they are more targeted and have lower costs.

[0008] To improve the original data To improve the accuracy and efficiency of feature extraction, preferably, the exclusive feature extraction tower includes a feature tower A based on Conv1D-LSTM network, a feature tower B based on CNN network, a feature tower C based on Transformer and a feature tower D based on GNN tower, as well as the final high-dimensional feature vector output by feature tower AD. Shared feature projection layer mapped to a unified space; for device types , input data ,in, T is the time step, C is the number of channels; Among them, the final high-dimensional feature vector output by feature tower A The calculation formula is as follows: ; Among them, in the convolutional layer Represents the output feature map after convolution, Represents the weight matrix of the convolution kernel; The first To Time step data; The bias term of the convolution operation; Activation function; in LSTM layer , are the weight and bias of the LSTM layer respectively.

[0009] Further preferably, the final high-dimensional feature vector output by the feature tower D is The calculation formula is as follows: ; in, ; ; In the above formula, is an aggregate function, is the representation of node v at layer l, is the weight matrix of the lth layer, is the representation of node u at the l-1 layer, and MEAN is the average operation of the representation of neighboring nodes; is the activation function, represents the set of neighbors of node v, and E represents the set of edges in the graph.

[0010] Further preferably, the shared feature projection layer inputs the final high-dimensional feature vector The eigenvector obtained after projection The calculation formula is as follows: in, is the projected eigenvector, is the shared weight matrix, is the shared bias.

[0011] Preferably, the device portrait engine includes a multimodal feature fusion unit for fusing feature vectors sent by a dedicated feature extraction tower, a spatiotemporal state encoding unit for spatiotemporal data processing, a portrait generation unit for device portrait generation, device remaining life prediction and health index calculation, and a feature compression encoding unit for compressing feature vectors into 128-dimensional compressed feature packages; the spatiotemporal state encoding unit adopts an improved spatiotemporal LSTM that introduces a device aging factor, wherein the forget gate The expression is: in, is the aging factor, is the hidden state at time t-1, is the LSTM bias vector, is the LSTM weight matrix, is the Sigmoid activation function; " stands for element-by-element multiplication, is the hidden state at time t; other gating mechanisms are the same as the existing spatiotemporal LSTM.

[0012] Preferably, the cross-domain fusion center includes an entity extraction unit, which is used to respectively receive the 128-dimensional compressed feature package sent from the device portrait engine and data from other domains including the environment domain, supply chain domain, operation and maintenance domain, and supervision domain; and respectively extract device entities from the 128-dimensional compressed feature package, and extract entities related to the device entities from the other domain data; a relationship extraction unit, which establishes an entity data packet according to predefined rules and relationship extraction models; a spatiotemporal alignment unit, which standardizes the data of the entity data packet and establishes spatiotemporal relationships; a knowledge fusion unit, which is used to fuse the newly extracted knowledge with the existing knowledge graph to handle conflicts and redundancies; a storage and update unit, which stores the knowledge after fusion by the knowledge fusion unit in the graph database to form a new spatiotemporal knowledge graph.

[0013] The present invention also provides a cross-domain knowledge fusion method, which is implemented based on the above-mentioned multi-feature-based cross-domain knowledge fusion system and specifically includes the following steps: Step STP100, by including the sensor Collect cross-domain raw data including at least one of the equipment domain, environment domain, supply chain domain, operation and maintenance domain, and supervision domain and convert the raw data into Perform feature extraction and output a compressed feature package consisting of fixed-length vectors; Step STP200, performing spatiotemporal alignment, entity extraction, relationship extraction, knowledge fusion and storage update steps on the compressed feature package in sequence through the cross-domain fusion center; The spatiotemporal alignment step is implemented using a spatiotemporal normalization model, which is: ; ; Among them, GeoID is the geometry library for generating geographic identifiers, level=18‌ represents the resolution of S2CellId, lat‌ and lon respectively represent the specific location on the earth; t represents the current time, t0 represents the start time, Δt represents the time window size, TimeSlot‌ represents the relative position of the current time within the time window; SpatioTempKey‌ represents the spatiotemporal key that combines geographic location and time information to uniquely identify a spatiotemporal point, 10 10 It is the separator between GeoID and TimeSlot, and there will be no conflict when combining them; The entity extraction step uses a unified entity model to compress the feature package and convert it into a unified entity model with spatiotemporal consistency. The unified entity model Entity is as follows: Where ‌ID‌ represents the unique identifier of the entity, ‌Type‌ represents the type of the entity, ‌Attributes‌ represents the attributes of the entity, and ‌SpatioTempKey‌ represents the spatiotemporal key obtained from the spatiotemporal alignment step; The relationship extraction step includes performing relationship reasoning based on GNN message passing based on relationship discovery of the hypergraph, obtaining a relationship set including implicit relationships and explicit relationships and forming a knowledge graph; The storage and update step uses an incremental learning mechanism with confidence settings to update and store the formed knowledge graph. The incremental learning mechanism is expressed as: in, represents the spectrum at time t, The graph of the previous moment, represents the incremental map, τ represents the confidence threshold; confidence The calculation formula is as follows: in, Represents the confidence level of the data, represents the model confidence, Temporal consistency, α, β, γ are weight parameters.

[0014] Beneficial effects: 1. The system provided by the present invention can integrate various types of sensors to achieve more comprehensive and multi-dimensional data collection; it improves the richness and accuracy of data and provides a more solid foundation for subsequent feature extraction and knowledge fusion.

[0015] ‌2. This invention uses a dedicated feature extraction tower, including feature towers based on Conv1D-LSTM, CNN, Transformer, and GNN, and a shared feature projection layer, to achieve multi-level and multi-dimensional feature extraction and fusion; improve the accuracy and efficiency of feature extraction, and help to more accurately describe device status and behavior patterns.

[0016] ‌3. This paper introduces a device aging factor into the spatiotemporal state encoding unit, improving the traditional spatiotemporal LSTM model. This more accurately reflects the aging process of devices over time, improving the accuracy of device remaining life prediction and health index calculation.

[0017] ‌4. The present invention realizes the fusion of cross-domain knowledge through a cross-domain fusion center, and adopts an incremental learning mechanism with set confidence to update and store the knowledge graph; it realizes the effective integration and utilization of cross-domain knowledge, improves the timeliness and accuracy of the knowledge graph, and provides strong support for predictive maintenance and cross-device scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 It is a structural block diagram of the feature extraction tower of the present invention.

[0020] Figure 2 It is a system structure diagram of the present invention.

[0021] Figure 3 The five-domain federation of the system of the present invention is just a schematic diagram.

[0022] Figure 4 It is a flow chart of the present invention.

[0023] Figure 5 This is the flow chart of constructing the spatiotemporal knowledge graph of the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without making any creative efforts shall fall within the scope of protection of the present application.

[0026] Example 1: The present invention provides a cross-domain knowledge fusion system based on multiple features. Figure 2-Figure 3 As shown, specifically including The device layer includes multiple sensors installed on devices of different types and locations to collect data from different devices. ,in, m Represents the type of installed equipment, n Represents the sequence of installed equipment; the sensor Collect and obtain the corresponding original data , and sent to the Edge intelligence layer for feature extraction; The edge intelligence layer, including a dedicated feature extraction tower and a device profiling engine, converts the raw data sent from the device layer into The original data is processed by a dedicated feature extraction tower deployed on the device. Perform feature extraction and send the extracted feature vector to the device profiling engine. Compress the feature vector through an autoencoder or a fully connected network to obtain a compressed feature package. The cross-domain fusion center combines the compressed feature packets sent from the edge intelligence layer and auxiliary data from the environment domain, supply chain domain, operation and maintenance domain, and regulatory domain. Perform entity extraction, relationship extraction, spatiotemporal alignment, knowledge fusion in sequence, obtain fused data and store / update it; The decision-making application layer includes a display terminal for displaying fused data, and a scheduling unit for providing a warning unit for predictive maintenance solutions and cross-device scheduling.

[0027] In order to better explain and illustrate this system, this embodiment uses different types of collection vehicles, armored vehicles, drones, and transport vehicles that are distributed all over the country as entity objects for specific description. Of course, the four entity devices used in this embodiment are just one example of the present invention to facilitate understanding, including but not limited to the above entities. Other entity devices can also be used and should not be understood in a specific way. m = 1, 2, 3, 4 represents the type of device; n ∈ 1, 2, 3 ... N, then the sensor of the collection vehicle is represented as , … Similarly, the sensors of the armored vehicle are represented as , … ; The drone’s sensor is represented by , … The transport vehicle's sensor is represented by , … ; For any sensor of the collection vehicle The collected information includes vibration, temperature, rotation speed, speed, Beidou positioning location information, etc.; the sensors of any armored vehicle The collected information includes speed, tire pressure, position, speed, etc.; any drone’s sensor The collected information includes battery power, speed, image, etc.; the sensors of any transport vehicle The information collected includes vibration, temperature, rotation speed, speed, Beidou positioning location information, etc. The information categories collected by the above different sensors are only listed for the convenience of understanding. The information actually collected includes but is not limited to the above information categories. The actual collected data depends on the settings and requirements of the system and is not limited to the above information data. In order to reduce the data processing pressure of the system and improve the efficiency of overall data processing and sharing, edge computing is adopted in this embodiment. Different exclusive feature extraction towers are configured according to different device types to extract the corresponding raw data collected by the corresponding sensors. It is worth noting that the original data The subscript mn in the table should correspond to the sensor providing the data. The corresponding sensor The collected raw data is recorded as , and so on. The advantage of the edge computing approach is that the cross-domain fusion center receives compressed feature packets that have already been processed. This is a qualitative leap in the construction of system hardware and the communication pressure, significantly reducing the amount of data transmission. Because the data has been processed by edge computing, the feature vector extraction and compression are completed, making the subsequent fusion processing faster, more efficient, and more accurate. This lays the technical foundation for subsequent display, call, and timely decision-making. Figure 2 As shown, the compressed feature packets sent through the edge intelligence layer and the auxiliary data from the environment domain, supply chain domain, operation and maintenance domain, and regulatory domain respectively. , where i represents the domain and j represents the order of the auxiliary data. In this embodiment, the environmental domain, supply chain domain, operation and maintenance domain, and regulatory domain correspond to i=1, 2, 3, 4; j∈1, 2, 3…+∞. This establishes entity-slicing relationships based on predefined system rules and models. For example, the relationship between the collection vehicle, armored vehicle, drone, and transport vehicle in this embodiment and the rotational speed, speed, and Beidou positioning coordinates in the compressed feature package; the relationship between the physical device and the environment, such as weather information at a specific location; and the relationship between the operation and maintenance work order, including maintenance records and data on upcoming repairs; as well as the relationship between supply chain data and regulatory data. This enables the system to grasp information from different domains and unify and correlate them, providing a basis for subsequent scientific decision-making. However, since devices are located in different regions and times, the location and time information of the entities need to be standardized. For example, Beidou positioning coordinates can be converted into geographic area codes, and time can be converted into timestamps in a unified time zone. Spatiotemporal relationships can then be established, such as when a device is located at a certain location at a certain point in time. Finally, all the spatiotemporal data is fused into a knowledge graph encompassing data from the environmental, supply chain, operations, and regulatory domains. Semantic conflicts and redundancies are resolved, resulting in a fused spatiotemporal knowledge graph that incorporates both temporal and spatial dimensions. This allows for better management of physical devices, ensuring that all managed physical devices are consistently maintained in a scientific and healthy state. For example, changes in a device's state are associated with specific time and location, and environmental events (such as typhoons) also have spatiotemporal attributes. Maintenance work orders can be used to predict potential device failures, and combined with supply chain data, the probability of timely parts delivery can be determined when a device has a high probability of failure. This creates a knowledge graph capable of managing a theoretically unlimited number of physical devices and providing real-time visibility into the location and status of all devices. The state described here encompasses both the current operating state and the probability of the next event occurring. This enables real-time monitoring, surveillance, early warning, and cross-device scheduling, significantly improving the utilization efficiency of the entire system, reducing internal consumption costs, and predicting the impact of sudden failures of physical equipment in advance, as well as timely occurrence of high-probability failure events. It can also conduct efficient troubleshooting based on supply chain data and operation and maintenance work order data, significantly improving management efficiency and the accuracy and scientificity of decision-making. Figure 3As shown, the core node of any physical device can be associated with the environment node, operation and maintenance node and supply chain node according to its operating status and location information, to ensure that the physical device is in the process of operation and the prediction of the state that may be about to be reached or generated. The factors of the environment, operation and maintenance, and supply chain can be automatically captured; for special physical equipment, it is beneficial to design airspace supervision for drones, etc., and it can also understand whether the flight airspace involved in a certain drone equipment involves the possibility of airspace supervision, as well as the specific time, range, etc., so as to effectively ensure the normal operation of the drone, and will not cause new problems due to the triggering of supervision due to the established operation plan. If a sudden failure occurs in the physical device, the operation and maintenance node can be matched in time according to the fault code, and the component ID can be matched and shared with the supply chain node according to the feedback operation and maintenance node data. Adopt Figure 3 The hierarchical federated architecture shown achieves the unity of data isolation and knowledge sharing. As an optional implementation method, it can be further combined with ADA-FE dynamic feature compression, CrossD-KF hypergraph fusion, and KGRL knowledge-driven decision algorithm to utilize cross-domain knowledge graphs to achieve full-link innovation of data perception-knowledge construction-decision generation.

[0028] Example 2: This embodiment is based on the first embodiment to realize the multi-field and multi-type equipment information collection. It is a multimodal sensor or a variety of single-modal sensors. The multimodal sensor refers to a sensor that is integrated with sensors suitable for different modalities and calls different modes to collect different information. Different equipment, different environments, and different data collection can be achieved by simply changing or not changing the multimodal sensor. Compared with single-modal sensors, multimodal sensors have higher integration, stronger processing capabilities, a wider range of collected data types, and more application scenarios, but the cost of the sensor is also higher; single-modal sensors refer to sensors provided for specific equipment or devices and / or environments. The type of information they collect is determined for specific equipment. Compared with multimodal sensors, they collect fewer types of data and have lower compatibility, but they are more targeted and have lower costs.

[0029] To improve the original data For feature extraction accuracy and efficiency improvement, see the manual Figure 1 As shown, the exclusive feature extraction tower includes feature tower A based on Conv1D-LSTM network, feature tower B based on CNN network, feature tower C based on Transformer and feature tower D based on GNN tower, as well as the final high-dimensional feature vector output by feature tower AD. Shared feature projection layer mapped to a unified space; for device types , input data ,in,T is the time step, C is the number of channels; in order to avoid ambiguity, it should be noted that the feature tower AD here refers to the feature tower A, feature tower B, feature tower C and feature tower D, and the final high-dimensional feature vector is the eigenvector , the eigenvector , the eigenvector and eigenvectors , is just an abbreviation for the convenience of expression.

[0030] Among them, the final high-dimensional feature vector output by feature tower A The calculation formula is as follows: ; Among them, in the convolutional layer Represents the output feature map after convolution, Represents the weight matrix of the convolution kernel; The first To Time step data; The bias term of the convolution operation; Activation function; in LSTM layer , are the weight and bias of the LSTM layer respectively.

[0031] In this embodiment, the final high-dimensional feature vector output by the feature tower D is The calculation formula is as follows: ; in, ; ; In the above formula, is an aggregate function, For node v in l The layer representation, For the l The weight matrix of the layer, For node u in the l -1 layer representation, MEAN is the average operation of the representation of neighboring nodes; is the activation function, represents the set of neighbors of node v, and E represents the set of edges in the graph.

[0032] Further preferably, the shared feature projection layer inputs the final high-dimensional feature vector The eigenvector obtained after projection The calculation formula is as follows: in, is the projected eigenvector, is the shared weight matrix, is the shared bias.

[0033] Preferably, the device portrait engine includes a multimodal feature fusion unit for fusing feature vectors sent by a dedicated feature extraction tower, a spatiotemporal state encoding unit for spatiotemporal data processing, a portrait generation unit for device portrait generation, device remaining life prediction and health index calculation, and a feature compression encoding unit for compressing feature vectors into 128-dimensional compressed feature packages; the spatiotemporal state encoding unit adopts an improved spatiotemporal LSTM that introduces a device aging factor, wherein the forget gate The expression is: in, is the aging factor, is the hidden state at time t-1, is the LSTM bias vector, is the LSTM weight matrix, is the Sigmoid activation function; " stands for element-by-element multiplication, is the hidden state at time t; other gating mechanisms are the same as the existing spatiotemporal LSTM.

[0034] In this embodiment, the cross-domain fusion center includes an entity extraction unit, which is used to respectively receive the 128-dimensional compressed feature package sent by the device portrait engine and data from other domains including the environment domain, supply chain domain, operation and maintenance domain, and supervision domain; and extract device entities from the 128-dimensional compressed feature package, and extract entities related to the device entities from the other domain data; a relationship extraction unit, which establishes an entity data packet according to predefined rules and relationship extraction models; a spatiotemporal alignment unit, which standardizes the data of the entity data packet and establishes spatiotemporal relationships; a knowledge fusion unit, which is used to fuse the newly extracted knowledge with the existing knowledge graph to handle conflicts and redundancies; a storage and update unit, which stores the knowledge after fusion by the knowledge fusion unit in the graph database to form a new spatiotemporal knowledge graph.

[0035] Example 3: The present invention also provides a cross-domain knowledge fusion method, which is implemented based on the above-mentioned multi-feature-based cross-domain knowledge fusion system. Figure 4-Figure 5 As shown, the specific steps include: Step STP100, by including the sensor Collect cross-domain raw data including at least one of the equipment domain, environment domain, supply chain domain, operation and maintenance domain, and supervision domain and convert the raw data into Perform feature extraction and output a compressed feature package consisting of fixed-length vectors; Step STP200, performing spatiotemporal alignment, entity extraction, relationship extraction, knowledge fusion and storage update steps on the compressed feature package in sequence through the cross-domain fusion center; The spatiotemporal alignment step is implemented using a spatiotemporal normalization model, which is: ; ; Among them, GeoID is the geometry library for generating geographic identifiers, level=18‌ represents the resolution of S2CellId, lat‌ and lon respectively represent the specific location on the earth; t represents the current time, t0 represents the start time, Δt represents the time window size, TimeSlot‌ represents the relative position of the current time within the time window; SpatioTempKey‌ represents the spatiotemporal key that combines geographic location and time information to uniquely identify a spatiotemporal point, 10 10 It is the separator between GeoID and TimeSlot, and there will be no conflict when combining them; The entity extraction step uses a unified entity model to compress the feature package and convert it into a unified entity model with spatiotemporal consistency. The unified entity model Entity is as follows: Where ‌ID‌ represents the unique identifier of the entity, ‌Type‌ represents the type of the entity, ‌Attributes‌ represents the attributes of the entity, and ‌SpatioTempKey‌ represents the spatiotemporal key obtained from the spatiotemporal alignment step; The relationship extraction step includes performing relationship reasoning based on GNN message passing based on relationship discovery of the hypergraph, obtaining a relationship set including implicit relationships and explicit relationships and forming a knowledge graph; The storage and update step uses an incremental learning mechanism with confidence settings to update and store the formed knowledge graph. The incremental learning mechanism is expressed as: in, represents the spectrum at time t, The graph of the previous moment, represents the incremental map, τ represents the confidence threshold; confidence The calculation formula is as follows: in, Represents the confidence level of the data, represents the model confidence, Temporal consistency, α, β, γ are weight parameters.

[0036] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A multi-feature-based cross-domain knowledge fusion system, characterized by: include The device layer includes multiple sensors installed on devices of different types and locations to collect data from different devices. ,in, m Represents the type of installed equipment, n Represents the sequence of installed equipment; the sensor Collect and obtain the corresponding original data , and sent to the Edge intelligence layer for feature extraction; The edge intelligence layer, including a dedicated feature extraction tower and a device profiling engine, converts the raw data sent from the device layer into The original data is processed by a dedicated feature extraction tower deployed on the device. Perform feature extraction and send the extracted feature vector to the device profiling engine. Compress the feature vector through an autoencoder or a fully connected network to obtain a compressed feature package. The cross-domain fusion center combines the compressed feature packets sent from the edge intelligence layer and auxiliary data from the environment domain, supply chain domain, operation and maintenance domain, and regulatory domain. Perform entity extraction, relationship extraction, spatiotemporal alignment, knowledge fusion in sequence, obtain fused data and store / update it; The decision-making application layer includes a display terminal for displaying fused data, and a scheduling unit for providing a warning unit for predictive maintenance solutions and cross-device scheduling.

2. The multi-feature-based cross-domain knowledge fusion system according to claim 1, characterized in that: The sensor It is a multimodal sensor or multiple single-modal sensors.

3. The multi-feature-based cross-domain knowledge fusion system according to claim 1, characterized in that: The exclusive feature extraction towers include feature tower A based on Conv1D-LSTM network, feature tower B based on CNN network, feature tower C based on Transformer and feature tower D based on GNN tower, as well as the final high-dimensional feature vector output by feature tower AD. A shared feature projection layer that maps to a unified space; For equipment types , input data ,in, T is the time step, C is the number of channels; Among them, the final high-dimensional feature vector output by feature tower A The calculation formula is as follows: ； Among them, in the convolutional layer Represents the output feature map after convolution, Represents the weight matrix of the convolution kernel; The first To Time step data; The bias term of the convolution operation; Activation function; in LSTM layer , are the weight and bias of the LSTM layer respectively.

4. The multi-feature-based cross-domain knowledge fusion system according to claim 3 is characterized in that: The final high-dimensional feature vector output by the feature tower D The calculation formula is as follows: ； in, ; ; In the above formula, is an aggregate function, is the representation of node v at layer l, is the weight matrix of the lth layer, is the representation of node u at the l-1 layer, and MEAN is the average operation of the representation of neighboring nodes; is the activation function, represents the set of neighbors of node v, and E represents the set of edges in the graph.

5. The multi-feature-based cross-domain knowledge fusion system according to claim 4 is characterized in that: The shared feature projection layer converts the input final high-dimensional feature vector The eigenvector obtained after projection The calculation formula is as follows: in, is the projected eigenvector, is the shared weight matrix, is the shared bias.

6. The multi-feature-based cross-domain knowledge fusion system according to claim 1, characterized in that: The device portrait engine includes a multimodal feature fusion unit for fusing feature vectors sent by a dedicated feature extraction tower, a spatiotemporal state encoding unit for spatiotemporal data processing, a portrait generation unit for device portrait generation, device remaining life prediction and health index calculation, and a feature compression encoding unit for compressing feature vectors into 128-dimensional compressed feature packages; the spatiotemporal state encoding unit adopts an improved spatiotemporal LSTM that introduces a device aging factor, in which the forget gate The expression is: in, is the aging factor, is the hidden state at time t-1, is the LSTM bias vector, is the LSTM weight matrix, is the Sigmoid activation function;" " stands for element-by-element multiplication, is the hidden state at time t; other gating mechanisms are the same as the existing spatiotemporal LSTM.

7. The multi-feature-based cross-domain knowledge fusion system according to claim 6, characterized in that: The cross-domain fusion center includes an entity extraction unit, which is used to receive 128-dimensional compressed feature packages sent by the device profiling engine and data from other domains including the environment domain, supply chain domain, operation and maintenance domain, and supervision domain; and extract device entities from the 128-dimensional compressed feature packages and entities related to the device entities from the other domain data; a relationship extraction unit, which establishes entity data packets based on predefined rules and relationship extraction models; and a spatiotemporal alignment unit, which standardizes the data in the entity data packets and establishes spatiotemporal relationships. Knowledge fusion unit, used to fuse newly extracted knowledge with existing knowledge graphs and handle conflicts and redundancies; The storage update unit stores the knowledge fused by the knowledge fusion unit into the graph database to form a new spatiotemporal knowledge graph.

8. A cross-domain knowledge fusion method, implemented based on the multi-feature-based cross-domain knowledge fusion system according to any one of claims 1 to 7, comprising the following steps: Step STP100, by including the sensor Collect cross-domain raw data including at least one of the equipment domain, environment domain, supply chain domain, operation and maintenance domain, and supervision domain and convert the raw data into Perform feature extraction and output a compressed feature package consisting of fixed-length vectors; Step STP200, performing spatiotemporal alignment, entity extraction, relationship extraction, knowledge fusion and storage update steps on the compressed feature package in sequence through the cross-domain fusion center; The spatiotemporal alignment step is implemented using a spatiotemporal normalization model, which is: ; ; Among them, GeoID is the geometry library for generating geographic identifiers, level=18‌ represents the resolution of S2CellId, lat‌ and lon respectively represent the specific location on the earth; t represents the current time, t0 represents the start time, Δt represents the time window size, TimeSlot‌ represents the relative position of the current time within the time window; SpatioTempKey‌ represents the spatiotemporal key that combines geographic location and time information to uniquely identify a spatiotemporal point, 10 10 It is the separator between GeoID and TimeSlot, and there will be no conflict when combining them; The entity extraction step uses a unified entity model to compress the feature package and convert it into a unified entity model with spatiotemporal consistency. The unified entity model Entity is as follows: Where ‌ID‌ represents the unique identifier of the entity, ‌Type‌ represents the type of the entity, ‌Attributes‌ represents the attributes of the entity, and ‌SpatioTempKey‌ represents the spatiotemporal key obtained from the spatiotemporal alignment step; The relationship extraction step includes performing relationship reasoning based on GNN message passing based on relationship discovery of the hypergraph, obtaining a relationship set including implicit relationships and explicit relationships and forming a knowledge graph; The storage and update step uses an incremental learning mechanism with confidence settings to update and store the formed knowledge graph. The incremental learning mechanism is expressed as: in, represents the spectrum at time t, The graph of the previous moment, represents the incremental map, τ represents the confidence threshold; confidence The calculation formula is as follows: in, Represents the confidence level of the data, represents the model confidence, Temporal consistency, α, β, γ are weight parameters.

Images