Knowledge thematic generation method and system

Abstract

The invention provides a knowledge thematic generation method and system, which can effectively mine deep implicit association among multi-source heterogeneous data and efficiently and accurately generate knowledge themes. The method comprises the following steps: acquiring multi-source heterogeneous data; processing the multi-source heterogeneous data to obtain at least two structured entities; determining a comprehensive association degree between any two entities in the at least two entities, performing clustering division on the at least two entities based on the comprehensive association degree between any two entities to obtain at least one entity group, and screening out each target entity group from the at least one entity group according to a first preset condition, the comprehensive correlation degree is obtained by carrying out weighted fusion on semantic similarity, topological correlation degree and time sequence compactness; and generating a corresponding knowledge topic based on each target entity group.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and knowledge automatic management, and particularly relates to a knowledge topic generation method and system. BACKGROUND

[0002] With the deepening of enterprise informatization construction and the popularization of Internet of Things technology, a large amount of multi-source heterogeneous data (referring to a data set from different sources, with different structures and formats) is generated in various industries, including base station logs, device alarms, operation and maintenance work orders, operation and maintenance reports, and knowledge graph data. In the scenarios of intelligent operation and maintenance, enterprise knowledge management, and decision support, how to quickly extract valuable and systematic knowledge from multi-source heterogeneous data and automatically generate knowledge topics for specific topics or problems has become a key requirement to improve operational efficiency and intelligent level. Knowledge topics can aggregate, sort and present scattered but related knowledge points, thereby providing strong information support for fault root cause analysis and business insight mining.

[0003] However, the current knowledge topic generation scheme relying on traditional knowledge bases usually implements simple association and classification of knowledge through artificial pre-set rules, and its processing logic is relatively rigid. This scheme not only relies heavily on expert experience and has high labor costs, but also due to the limitations of rules, it is difficult to effectively mine deep implicit associations between multi-source heterogeneous data, resulting in low efficiency and insufficient accuracy of the generated knowledge topics.

[0004] Therefore, there is an urgent need for a method that can efficiently and accurately generate knowledge topics. SUMMARY

[0005] The present application provides a knowledge topic generation method and system, which can effectively mine deep implicit associations between multi-source heterogeneous data and efficiently and accurately generate knowledge topics.

[0006] In a first aspect, a knowledge topic generation method is provided, the method comprising: obtaining multi-source heterogeneous data; processing the multi-source heterogeneous data to obtain at least two structured entities; determining a comprehensive correlation degree between any two entities of the at least two entities, the comprehensive correlation degree being obtained by weighted fusion based on semantic similarity, topological correlation degree and time sequence tightness between the two entities, the semantic similarity being a similarity between the entities in terms of semantics, the topological correlation degree being a correlation between the entities in terms of topological structure, and the time sequence tightness being a tightness between the entities in terms of time; clustering and dividing the at least two entities based on the comprehensive correlation degree between any two entities to obtain at least one entity group; selecting target entity groups from the at least one entity group according to a first preset condition; generating a corresponding knowledge topic based on each target entity group.

[0007] In a possible design, determining the comprehensive correlation degree between any two entities of the at least two entities comprises: obtaining a semantic feature vector of text description information of each entity based on a natural language processing model; and determining a semantic similarity between any two entities based on the semantic feature vector of each entity; determining a topological correlation degree between any two entities based on attributes and / or connection relationships of corresponding nodes of the entities in the knowledge graph data; determining a time sequence closeness of any two entities based on time attributes of the entities; weighting the semantic similarity between any two entities by a first weight coefficient, weighting the topological correlation degree between any two entities by a second weight coefficient, weighting the time sequence closeness between any two entities by a third weight coefficient, and summing the weighted semantic similarity, topological correlation degree and time sequence closeness to obtain the comprehensive correlation degree, the first weight coefficient, the second weight coefficient and the third weight coefficient being dynamically adjusted coefficients.

[0008] In a possible design, determining the topological correlation degree between any two entities based on attributes and / or connection relationships of corresponding nodes of the entities in the knowledge graph data comprises: obtaining an embedding representation matrix, an adjacency matrix and a topological weight matrix of the corresponding nodes of the entities in the knowledge graph data, the topological weight matrix being a matrix composed of network topological feature parameters; performing fusion processing on the embedding representation matrix, the adjacency matrix and the topological weight matrix of the corresponding nodes of the entities based on a topological perception graph convolution network model, to update the embedding representation matrix of the corresponding nodes of the entities, the updated embedding representation matrix containing semantic information and topological information; determining the topological correlation degree between any two entities according to the updated embedding representation matrix of the corresponding nodes of the two entities.

[0009] In a possible design, generating a corresponding knowledge topic based on each target entity group comprises: generating a title of the knowledge topic based on text description information of each entity in the target entity group; constructing an association evidence chain of the knowledge topic according to the comprehensive correlation degree and the association path among the entities in the target entity group; If the group correlation degree and / or the business risk indicator of the target entity group meet the second preset condition, a treatment suggestion is matched or generated from a preset treatment suggestion library, the title, the associated evidence chain and the treatment suggestion are integrated, a structured knowledge special report is generated and displayed, and the group correlation degree is the average of the comprehensive correlation degrees of each entity pair in the target entity group, and each entity pair is formed based on any two entities in the target entity group. Otherwise, the title and the associated evidence chain are integrated, and a structured knowledge special report is generated and displayed.

[0010] In a possible design, determining the time sequence tightness of any two entities based on the time attribute of the entity includes: Based on the adaptive sliding window mechanism, the size of the current processing window is dynamically determined according to the event inflow density; According to the size of the current processing window, the tightness of the time difference of the occurrence of the two entities is determined using a Gaussian kernel function, that is, the time sequence tightness.

[0011] In a possible design, before determining the comprehensive correlation degree between any two entities of the at least two entities, the method further includes: screening the at least two entities, and the screening step includes: Real-time acquisition of the data stream of the entity identified from the message queue of the message middleware; Grouping the data stream according to the entity identifier of the entity, and dividing the data with the same entity identifier into the same processing group; Based on the network load indicator, the length of the current processing window is dynamically determined, and the starting boundary time point of the processing window is determined based on the event timestamp, and a sliding time window instance is generated in combination with the length of the current processing window; In each sliding time window instance, the dynamic correlation aggregation processing is performed on the entities in each processing group after grouping, and the dynamic correlation aggregation processing is based on a time decay factor, a cross-domain adjustment factor and a feature filtering threshold, and outputs an aggregation score reflecting the dynamic correlation degree between entities; Delete the entity pair formed by the two entities whose aggregation score is less than the first preset threshold.

[0012] In a possible design, the knowledge graph data is stored in a memory graph database, a distributed graph database cluster and a columnar distributed database based on the latest date of data access, wherein the memory graph database is used to store hot data, the distributed graph database cluster is used to store warm data, and the columnar distributed database is used to store cold data.

[0013] In a possible design, the distributed graph database cluster is configured to create data partitions based on business domains, wherein the communication business domain corresponds to a main partition container. The distributed graph database cluster is configured to use a geographic region partitioning strategy to divide the wireless network device nodes within the primary partition container into sub-partitions, grouping wireless network device nodes in the same geographic region into the same sub-partition; and / or, The distributed graph database cluster is configured to use a device type hashing strategy to divide the core network device nodes within the main partition container into sub-partitions, and to evenly distribute core network device nodes of the same device type across multiple sub-partitions.

[0014] In a feasible design, the method also includes: Based on the key-value pair structure, the association results of the entity pairs consisting of two entities after association degree analysis are cached in the association result cache container. The association results include the identifier of the entity pair, the semantic similarity between the entity pairs, the topological association degree, the temporal tightness or the comprehensive association degree. The key includes the identifier of the entity pair, and the value includes the semantic similarity, topological association degree, temporal tightness or comprehensive association degree between the entity pairs. The initial memory allocation strategy of the association result cache container adopts a dynamic expansion mechanism. When the number of stored entity pairs reaches a second preset threshold, the least recently used algorithm is used to evict the least frequently accessed cache entry in the association result cache container. Cache entries include key-value pairs; and / or, Set an expiration time for cache entries and trigger an asynchronous cleanup thread to delete expired cache entries the next time they are accessed; When the queried entity pair does not exist or the cache entry to which it belongs has expired, the overall relevance of the queried entity pair is automatically determined based on the registered callback function. The corresponding key-value pair is determined based on the overall relevance of the queried entity pair and written into the association result cache container, and the access timestamp is updated.

[0015] Secondly, a knowledge topic generation system is provided, including a data acquisition module, an entity recognition module, a multimodal association analysis module, and a knowledge topic generation module; The data acquisition module is configured to acquire heterogeneous data from multiple sources; The entity recognition module is configured to process multi-source heterogeneous data to obtain at least two structured entities; The multimodal association analysis module is configured to determine the comprehensive association degree between any two entities among at least two entities. The comprehensive association degree is obtained by weighted fusion of semantic similarity, topological association degree and temporal tightness between the two entities. Semantic similarity is the similarity between entities in terms of semantics, topological association degree is the association degree between entities in terms of topological structure, and temporal tightness is the tightness between entities in terms of time. The multimodal association analysis module is also configured to cluster at least two entities to obtain at least one entity group based on the comprehensive association degree between any two entities. The multimodal association analysis module is also configured to filter out target entity groups from at least one entity group based on a first preset condition. The knowledge topic generation module is configured to generate corresponding knowledge topics based on each target entity group.

[0016] This application's embodiments overcome the problems of delayed knowledge updates and incomplete coverage caused by traditional solutions relying on static knowledge bases by automatically acquiring multi-source heterogeneous data and transforming it into structured entities. Furthermore, by integrating a weighted comprehensive correlation degree based on three dimensions—semantic similarity, topological relevance, and temporal density—it automatically mines deep implicit relationships between entities (such as semantic context, network structure dependencies, and temporal co-occurrence patterns), avoiding the one-sidedness of manually preset rules. Based on this, target entity groups are automatically generated through clustering and conditional filtering, and finally, the knowledge topic generation module outputs the topic. This solution systematically replaces the reliance on traditional knowledge bases and manually preset rules, fundamentally solving the problems of low efficiency and insufficient accuracy in topic generation caused by the difficulty in capturing deep implicit relationships in traditional solutions through data-driven intelligent association mining and group discovery. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart illustrating a knowledge topic generation method provided in an exemplary embodiment of this application; Figure 2 This is a schematic flowchart illustrating an example of data acquisition provided in an exemplary embodiment of this application; Figure 3 This is a schematic diagram of a knowledge topic generation system structure provided in an exemplary embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] To effectively uncover deep implicit relationships between multi-source heterogeneous data and improve the efficiency and accuracy of knowledge topic generation, such as... Figure 1 As shown, a method for generating knowledge topics is provided, the method including: S110, acquire multi-source heterogeneous data.

[0021] The multi-source heterogeneous data includes device alarm data, performance indicator data, knowledge graph data, base station logs, maintenance work orders, and maintenance reports. Performance indicator data includes operational status metrics for various network devices, such as CPU utilization and memory usage. Knowledge graph data (from graph databases) includes device connectivity, physical topology features (e.g., fiber optic cable length, port availability), and knowledge associations (e.g., device-fault-solution).

[0022] For example, Elasticsearch can be used to store unstructured text such as base station logs, maintenance work orders, and maintenance reports, so that text data can be retrieved from Elasticsearch later.

[0023] In one feasible design, knowledge graph data is hierarchically stored in an in-memory graph database, a distributed graph database cluster, and a columnar distributed database based on the latest date of data access. The in-memory graph database is used to store hot data, the distributed graph database cluster is used to store warm data, and the columnar distributed database is used to store cold data.

[0024] Hot data can be set to data accessed within the last 30 days, warm data can be set to data accessed within the last 31 to 90 days, and cold data is data that has not been accessed for more than 90 days.

[0025] For example, hot data is stored in a high-performance RedisGraph database (i.e., an in-memory graph database) to ensure a response time of less than 10 milliseconds, providing an ultimate performance experience; warm data is placed in a Neo4j cluster (i.e., a distributed graph database cluster) accelerated by high-performance SSD solid-state drives to balance speed and cost; and cold data is stored in an HBase database (i.e., a columnar distributed database) with a compression ratio of up to 8:1, effectively saving storage space and reducing long-term storage costs.

[0026] The above example demonstrates how designing a multi-mode hybrid storage engine enables efficient hierarchical management of knowledge graph data, thereby improving query performance and storage resource utilization.

[0027] In one feasible design, the distributed graph database cluster is configured to create data partitions based on business domains, where the communication business domain corresponds to the main partition container; The distributed graph database cluster is configured to use a geographic region partitioning strategy to divide the wireless network device nodes within the primary partition container into sub-partitions, grouping wireless network device nodes in the same geographic region into the same sub-partition; and / or, The distributed graph database cluster is configured to use a device type hashing strategy to divide the core network device nodes within the main partition container into sub-partitions, and to evenly distribute core network device nodes of the same device type across multiple sub-partitions.

[0028] For example, in any graph database in a distributed graph database cluster, a primary partition container (root.communications) associated with the communication business domain can be created. The primary partition container can serve as the logical data boundary of this business domain to isolate data from other industry domains (such as data from other business domains like finance and healthcare, which are not mixed into this partition).

[0029] The graph database is configured to use a geographic region partitioning strategy for wireless network device nodes within the primary partition. This strategy includes: Extract the region attribute (such as the regional code for East China, North China, etc.) of the device node; Physical sub-partitions are divided according to the lexicographical range of the regional code (RANGE type) to achieve centralized storage of devices in the same geographical area.

[0030] Furthermore, the graph database is configured to employ a device type hashing strategy for core network device nodes (core_network) within the primary partition. This strategy includes: Extract the type attribute of the device node (such as the type identifier of router, switch, firewall, etc.); The hash value of the type identifier is calculated by a hash function, and data of the same device type is evenly distributed across multiple physical sub-partitions.

[0031] When storing graph data belonging to telecommunication equipment entities (telecom_equipment), the graph database employing the above strategy can automatically match the data to the pre-divided sub-partitions based on the physical location and type characteristics, so that the device relationships (such as connection topology) are maintained within the same sub-partition.

[0032] The above example, through a multi-level partitioning combination of geographical regions and hashes, simultaneously satisfies the requirements of region-related queries and device type load balancing (e.g., improving the efficiency of regional operation and maintenance report generation by 50%), reducing cross-partition JOIN operations by over 90% in entity association constraints (benchmark dataset: 1 billion device nodes). Furthermore, the wireless network partitioning in the above embodiment can achieve hot and cold data separation by splitting regional ranges, and the core network partition can automatically expand based on hash results (supporting online node expansion). It is applicable to scenarios such as 5G network device management and cloud computing resource scheduling, effectively solving the coordination problem of multi-dimensional data distribution strategies. For example, when querying equipment of a provincial operator, calculations are first completed in the corresponding geographical region sub-partition, avoiding performance losses caused by global scanning.

[0033] It should be noted that while graph databases employing the above strategy can automatically assign most strongly related devices (e.g., devices within the same data center, switches and servers in the same geographical region) to the same physical sub-partition, some unavoidable cross-partition relationships exist in complex application scenarios. For example, a backbone optical cable connecting core routers in "East China" and "North China" will inevitably have devices located in different geographical partitions at both ends. To handle queries of cross-partition relationships in such scenarios, graph databases can also be configured to manage the relationships between entities across partitions through global indexes.

[0034] For example, such as Figure 2 As shown, the method is based on structured data pipelines, unstructured log pipelines, and streaming data buffer layers to acquire multi-source heterogeneous data.

[0035] The structured data pipeline is used to receive and preprocess device alarm data and performance metrics data in real time. This preprocessing standardizes the formats of device alarm data and performance metrics data from different sources and in different formats, facilitating subsequent processing.

[0036] In addition to the initial entity extraction from base station logs, maintenance work orders, and maintenance reports retrieved from Elasticsearch, the unstructured log pipeline can also detect newly added base station logs, maintenance work orders, and maintenance reports in Elasticsearch. Upon detecting new text data, it retrieves and preprocesses it from Elasticsearch to trigger incremental updates in the knowledge topic system. This preprocessing is used to initially extract entities from the base station logs, maintenance work orders, and maintenance reports.

[0037] The streaming data buffer layer is implemented based on message middleware (such as Kafka and Pulsar) and is used to partition and cache the preprocessed data from structured data pipelines and unstructured log pipelines based on semantic features.

[0038] For example, after data cleaning, the cleaned data is input into a structured data pipeline or an unstructured log pipeline.

[0039] The following example uses equipment alarm data to illustrate how structured data pipelines process data: The structured data pipeline continuously receives raw device alarm data streams encoded in Tag-Length-Value (TLV) format. Based on predefined TLV format parsing rules, it extracts the tag, length, and value fields from the raw alarm data, transforming the unstructured raw data into intermediate structured data containing multiple layers of fields. Using a pre-defined alarm field mapping table, device vendor-related field names (such as Vendor Alarm Code) in the intermediate structured data are converted into standardized field names (such as Standard Alarm Code), achieving standardized processing of the structured alarm data. The standardized structured alarm data is then distributed to the corresponding topic channels of the message middleware via an asynchronous communication protocol, allowing downstream entity recognition modules to subscribe and consume it in real time.

[0040] It should be clarified that a theme channel covers multiple sections, and downstream consumers can subscribe to the corresponding theme channel.

[0041] The above embodiments, through TLV decoding rules and mapping tables, can resolve the format differences in multi-source alarms (such as incompatible alarm field naming across different vendors). Employing message middleware (such as Kafka or Pulsar) enables peak shaving and fault isolation, ensuring data processing reliability in high-concurrency scenarios. Compared to traditional regular expression parsing methods, actual testing shows that the parsing efficiency of the method in this application is improved by approximately 40%.

[0042] Similarly, after receiving performance indicator data in real time, the structured data pipeline parses the data from devices from different manufacturers according to predefined rules and converts it into a unified standardized format, thus achieving standardized processing of structured performance indicator data. The standardized structured performance indicator data is then distributed to the corresponding topic channels of the message middleware via an asynchronous communication protocol for downstream consumers to subscribe to and consume in real time. Specific rules and standardized formats can be set according to actual needs, and this application does not impose any limitations on them.

[0043] The following example illustrates the data processing method for unstructured log pipelines: The unstructured log pipeline uses the Bidirectional Encoder Representations from Transformers (BERT) model and the Bidirectional Long Short-Term Memory (BiLSTM) model to perform entity recognition on texts such as base station logs, operation and maintenance work orders, and operation and maintenance reports pulled from Elasticsearch. See the following formula (1) for details: , formula (1); in, This represents the entity extraction result vector. This is the weight matrix. This represents the input text. This indicates that the BERT model and the BiLSTM model are compared on the input text. The output vectors are concatenated. This is the activation function, used to transform the concatenated feature vector into a probability distribution of entity categories.

[0044] The unstructured log pipeline, through this fusion model, can process 120,000 texts per minute, effectively capturing the contextual semantics and long-distance dependencies in texts such as base station logs, maintenance work orders, and maintenance reports, thereby improving the accuracy and robustness of entity recognition.

[0045] Unstructured log pipelines can also extract entities from texts such as base station logs, maintenance work orders, and maintenance reports through keyword extraction (such as regular expression matching).

[0046] The following example illustrates the data processing method of a streaming data buffer layer, which uses Kafka to partition structured alarm data output from a structured data pipeline: Step 1: Extraction of message semantic features Extract the predefined semantic classification identifier (i.e., the alarm type field alarmType) from the alarm event message body to be distributed (i.e., structured alarm data). The semantic classification identifier is used to characterize the business logic attributes of the alarm event (such as network congestion, hardware failure, etc.).

[0047] Step 2: Hash identifier conversion The semantic classification identifier obtained in step 1 is mapped to a fixed-length unique numeric identifier using a hash function to generate a hash feature value. This process ensures that alarm events with the same semantic classification always generate consistent feature values.

[0048] Step 3: Dynamic Partition Calculation Obtain the total number of currently available partitions for the corresponding Kafka topic, and perform a modulo operation between the hash feature value obtained in step 2 and the total number of partitions to determine the target partition number corresponding to the message, as shown in the following formula (2): Target partition number = (hash feature value) mod (total number of partitions), formula (2); Step 4: Message Directed Distribution Based on the partition number calculated in step 3, the alarm event message is routed to the message queue of the specified partition for storage, thus completing the semantically driven partition distribution process.

[0049] The above embodiment consistently distributes events of the same alarm type to the same partition, ensuring the continuity of state during downstream consumer processing (such as the accuracy of cumulative fault statistics). Furthermore, this embodiment automatically adapts to changes in the number of partitions through hash modulo operations, achieving uniform data distribution during horizontal scaling (actually, the message volume difference between partitions after scaling is <5%). The calculation process in this embodiment does not rely on external metadata queries, and pure in-memory computation takes <1ms / message (the benchmark test report of this application shows a query per second (QPS) of 120,000 / second).

[0050] The method by which the streaming data buffer layer partitions the structured performance metrics data output from the structured data pipeline based on Kafka is described in the example above, and will not be repeated here.

[0051] Similarly, when the streaming data buffer layer uses Kafka to partition the entity identification results output from the unstructured log pipeline, it also follows the logic of semantic feature extraction and hash partitioning. That is, the type of the identified entity (such as Internet Protocol address, device number) is used as a semantic classification identifier. The semantic classification identifier is then hashed and moduloed with the current number of partitions to obtain the target partition.

[0052] The semantic-based partitioning strategy described in the above embodiments is applicable to scenarios where data locality needs to be guaranteed according to business dimensions, such as real-time stream processing systems in fields such as telecommunications network alarm analysis and financial transaction risk control.

[0053] For example, this application manages different categories of data with differentiated storage strategies through a message middleware (such as Kafka).

[0054] The first storage strategy is used to store structured alarm data. The first storage strategy specifies the number of first partitions, the number of first copies, and the first retention time for storing structured alarm data.

[0055] For example, the first storage policy specifies that the number of first partitions is 24, the number of first replicas is 3, and the first retention time is 7 days, in order to meet the requirements of high throughput, high reliability, and short-term backtracking of alarm data.

[0056] A second storage strategy is used to store the entity data output by the entity identification module. This second storage strategy specifies the number of second partitions, the number of second copies, and the second retention period for storing the entity data. Specifically, the number of second partitions is less than the number of first partitions, the number of second copies is less than the number of first copies, and the second retention period is shorter than the first retention period. This is to accommodate the moderate throughput requirements and lower persistence requirements of intermediate results, thereby reducing storage overhead. For example, the number of second partitions is 16, the number of second copies is 2, and the second retention period is 3 days.

[0057] A third storage strategy is employed to store knowledge update data. Knowledge update data refers to incremental knowledge generated by the system after correlation analysis. Incremental knowledge includes, for example, newly added entity pairs with a comprehensive correlation greater than a third preset threshold, and new knowledge topics. The third storage strategy specifies the number of third partitions, the number of third copies, and the third retention time for storing knowledge update data. Specifically, the number of third partitions is less than the number of second partitions, the number of third copies is the same as or greater than the number of first copies, and the third retention time is set to permanent. This example is designed based on the low-frequency nature of system knowledge updates and long-term traceability.

[0058] Unless otherwise specified, the entity pair in this application refers to a pair of entities, that is, two entities.

[0059] For example, the TimescaleDB storage engine is used to store the performance metric data of the Kafka partition cache (the performance metric data has timestamps and belongs to time series data), thereby providing long-term and efficient storage for the performance metric data to support historical queries and macro trend analysis.

[0060] The TimescaleDB storage engine is a time-series extension of PostgreSQL, enabling efficient storage and querying of time-series data.

[0061] Building upon the examples above, the method also includes: A hybrid sharding strategy is applied to the target data table (e.g., the network metrics monitoring table (network_metrics)) that records performance metrics data. The hybrid sharding strategy includes: The timestamp field is used as the main sharding dimension to establish the basic organizational structure for time-series data storage. The data is divided into continuous time blocks (Chunks) at fixed time intervals (e.g., 1 day). Each time block is stored independently to enable fast query and location based on time range (e.g., querying traffic for a certain hour only scans 1 block). Using the device type field (device_type) as the secondary sharding dimension, the device type encoding is mapped to a preset number (e.g., 4) of physical sharding intervals (e.g., numbered 0-3) through a hash function, ensuring that data of the same type of devices is stored centrally and load balancing is achieved.

[0062] The example above generates a hybrid sharding structure that includes both time and device type dimensions by applying a hybrid sharding strategy to the target data table, which can simultaneously satisfy the following two types of query scenarios: Time-driven scenarios: Automatically manage the lifecycle of data blocks (e.g., delete expired data entirely); Device-driven scenario: Queries on similar devices only access 1-2 shards (actually reduced scan data volume by 75%).

[0063] As further exemplarily, this application also performs columnar compression optimization in the following manner: The target data table to be compressed is used as a data container to be compressed. The columnar compression function of the time series database is activated to change the data storage mode from row to column to improve storage density. Set the sorting rule of the data within the compressed block to descending timestamp (timestamp DESC) to ensure that the most recently written monitoring metrics are prioritized, so that high-priority data (such as network anomaly detection in the last hour) can be quickly located during queries by pushing down the predicate. Logically segment the data according to the device identifier field (device_id), and store the historical indicators of the same device in a continuous column block (Segment). This can improve the compression rate to 70%-85% by taking advantage of the locality characteristics of device data (such as repeated zero values) (actual test of a telecommunications company's device log scenario).

[0064] The example above performs columnar compression optimization on the target data table, generating a compressed block structure that simultaneously satisfies time sorting and device grouping. This compressed block structure supports cross-device queries, namely, quickly filtering irrelevant devices through segmented metadata (e.g., scanning only the column block of device_id IN (A, B)), and directly accessing the compressed block of the latest time window based on descending order (without a full table scan), enabling time-range-based queries.

[0065] S120 processes multi-source heterogeneous data to obtain at least two structured entities.

[0066] For example, entity recognition technology is used to annotate semantic units (i.e. entities) such as devices, events, and resources implied in unstructured or semi-structured multi-source heterogeneous data (such as text, logs, and network topology), and generate a unified entity set to extract structured entities from multi-source heterogeneous data.

[0067] In a feasible design, the following are some ways to process multi-source heterogeneous data: (1) Determine the entity (the entity’s name and attributes constitute the entity) based on the content of each field of the structured device alarm data or structured performance index data in the partition cache of the streaming data buffer layer. The entity’s attributes include, but are not limited to, entity type (Type), entity identifier (Identifier, ID), and data source association information.

[0068] Entity types include, but are not limited to, devices, events, resources, times, and links. An entity identifier is a unique identifier assigned to an entity. Data source association information reflects the origin of the entity data and can be recorded by binding pointers to the original data source (such as alarm IDs or log line numbers).

[0069] For example, the device entity "5th Generation Mobile Communication Technology (5G) Base Station" (i.e., the entity name) is extracted from the tag field of a structured device alarm data, and its entity identifier is assigned as "5G Base Station_001" according to a predefined identifier allocation rule, along with a pointer to the ID of its respective alarm data. Similarly, the device entity "Optical Module_A Port" (i.e., the entity name) is extracted from the value field of a structured device alarm data, with its entity identifier being "Optical Module_A Port_001", and a pointer to the ID of its respective alarm data is also assigned. The identifier allocation rule can be set according to actual needs; for example, the entity identifier assigned to an entity of a device type can be "Entity Name + Number", "Entity Type + Number", or "Entity Type + Entity Name + Number".

[0070] (2) Improve information based on entity identification results output from the unstructured log pipeline of the partitioned cache of the streaming data buffer layer.

[0071] Unstructured log pipelines extract raw entity fragments (such as "CPU overload") from text, representing a preliminary entity extraction. Their output lacks entity type and data source association information, and cannot assign entity identifiers. Therefore, further information completion processing is needed on the raw entity fragments output by the unstructured log pipeline to refine the entity's attributes. For example, a globally unique identifier (such as "Event_Fault_CPU001") could be assigned to "CPU overload," or heterogeneous representations could be mapped to a unified ontology (such as mapping "Alarm / Fault / Exception" to "Event") to determine the entity type.

[0072] By separating the "preliminary entity extraction" and "entity attribute refinement" processes, we can ensure high throughput of unstructured data (120,000 records / minute) while ensuring that entity data with completely consistent structure can be used for subsequent correlation analysis, thereby improving the accuracy and efficiency of correlation analysis.

[0073] (3) Determine entities based on the knowledge graph data stored in the graph database.

[0074] For example, entities can be identified through nodes in knowledge graph data.

[0075] For example, in the process of determining entities based on nodes, the predefined labels of the nodes are extracted as the names of the entities, and the predefined labels are converted into standardized entity types through a type mapping table. For example, the node label "5G_BaseStation" is mapped to the entity type "device". The attributes of a node are concatenated to generate an entity identifier. For example, if the attributes of a node include {vendor: "ZTE", model: "AAU5612", serial: "023488A"}, the entity identifier generated after concatenation is "ZTE_AAU5612_023488A". Bind a pointer to the node's identifier in the graph database as the data source association information.

[0076] For example, the attributes of an entity can be determined through nodes and connections in knowledge graph data. This example can identify important, business-meaning connections themselves as a new entity (such as "fiber optic transmission link").

[0077] For example, in the knowledge graph data, there exists node A: {id: "Device_A", type: "Router"}, node B: {id: "Device_B", type: "Switch"}, and a connection relationship (i.e., an edge): (Device_A) -[port: "Gig1 / 0 / 1"]-> (Device_B). Based on the connection type (physical connection), the port used (optical port), and the device types of the two connected nodes, the system can infer that this entity type is a "link".

[0078] Entity identifiers are generated by concatenating the nodes at both ends of the connection and the attributes of the connection relationship, such as "Link_Device_A_Gig1 / 0 / 1_Device_B".

[0079] Bind pointers to the identifiers of edges in the knowledge graph data as data source association information.

[0080] Finally, after entity recognition, a set of structured entities is generated, such as set E={e1,e2,...,en}, where each entity ei (i=1,...,n) contains attributes including entity type, entity identifier, and data source association information. Each entity in this set can be uniquely identified by its entity identifier.

[0081] This application generates structured entities, which can eliminate the differences in formats of multi-source heterogeneous data, enabling entities across data sources to have a unified basis for association, providing consistent input for subsequent calculation of the comprehensive association degree between entities, and avoiding association omissions caused by chaotic entity representation.

[0082] S130, determine the overall correlation degree between any two of the at least two entities.

[0083] The comprehensive relevance is obtained by weighted fusion of semantic similarity, topological relevance and temporal tightness between two entities. Semantic similarity is the similarity between entities in terms of semantics, topological relevance is the relevance between entities in terms of topological structure, and temporal tightness is the tightness between entities in terms of time.

[0084] The following describes the data model used in this application to determine the overall correlation between any two entities: The multimodal correlation feature space defined in this application is shown in the following formula (3): , formula (3); in, It represents the multimodal correlation feature space (composed of the Cartesian product of semantic, topological, and temporal modalities). Representing semantic space, Representing topological space, Representing temporal space, Represents the Cartesian product. It is defined as follows.

[0085] Semantic space , The numerical value is the same as the embedding dimension of the natural language processing model used for semantic feature extraction, for example, for the distilled BERT model. The value is 512. In fact, the vector space is normalized to [-1, 1].

[0086] Topological space , Represent a A vector space of dimension , where each dimension takes the value of a non-negative real number.

[0087] Temporal Space This is used to describe the degree of temporal correlation between entities (i.e., temporal density). For example, the temporal density between entity i and entity j can be calculated using a Gaussian kernel function. .

[0088] Based on the definition of the multimodal association feature space above, in a feasible design, the comprehensive association degree between any two entities of at least two entities can be determined in the following way: Semantic feature vectors of textual description information for each entity are obtained based on natural language processing models; semantic similarity between any two entities is determined based on the semantic feature vectors of each entity. Based on the attributes and / or connections of the corresponding nodes of each entity in the knowledge graph data, determine the topological association degree between any two entities; Based on the temporal attributes of entities, determine the temporal density between any two entities; The semantic similarity of any two entities is weighted by a first weighting coefficient, the topological relevance of any two entities is weighted by a second weighting coefficient, and the temporal tightness of any two entities is weighted by a third weighting coefficient. The weighted semantic similarity, topological relevance, and temporal tightness are then summed to obtain the comprehensive relevance. The first, second, and third weighting coefficients are dynamically adjusted coefficients.

[0089] The entity's textual description information includes the entity's name, entity type, and entity identifier (obtained from the previously acquired entity data). Furthermore, the entity's textual description information may also include contextual information about the entity within its data source.

[0090] It should be noted that the nodes corresponding to entities can be determined by interacting with graph databases, thereby obtaining information such as node attributes and connection relationships.

[0091] The following describes how semantic similarity is determined: For example, the natural language processing model is a distilled BERT model, whose output embedding vector dimension d=512. Based on the semantic space, entities... semantic feature vector As shown in the following formula (4): , formula (4); in, Representing entities The corresponding input text, This indicates that the text is encoded using the BERT model through distillation. Represents a vector L2 normalization (vector magnitude is 1) is performed to ensure that the semantic vectors of different entities are under the same "metric" to facilitate subsequent calculation of semantic similarity.

[0092] For example, the accuracy of semantic feature extraction can be improved by injecting a domain dictionary (such as the operation and maintenance term "AAU5636" or "X2 interface interruption") into the distilled BERT.

[0093] For example, two entities and Semantic similarity S1 between ij As shown in the following formula (5): , formula (5); in, Representing entities semantic feature vectors (see) (The method of determining i and j are the entity numbers, and cos() represents the cosine similarity function.) The range of its value is [-1, 1].

[0094] By calculating the cosine similarity between the semantic feature vectors of two entities, we can efficiently measure their proximity in the semantic space. The closer the value is to 1, the more similar the semantics are, and the closer it is to -1, the greater the semantic difference.

[0095] For example, two entities and Semantic similarity S2 between ij It can also be calculated based on the contrastive learning loss of distilled BERT, as shown in the following formula (6): , formula (6); in, This represents the normalized similarity probability value (range [0,1]). Indicates transpose. The temperature coefficient represents a scaling factor that controls the sharpness of the similarity distribution. Q is the number of negative samples (i.e., the number of negative samples in contrastive learning), which is the cardinality for normalizing the denominator to avoid gradient vanishing. q represents the number of negative samples (i.e., the number of negative samples compared to the previous one). Numbering of dissimilar entities Represents the semantic feature vector of the q-th negative sample (see...) The method for determining the similarity of samples is as follows: exp represents the exponential function, which is used to amplify the similarity of positive samples and suppress the similarity of negative samples.

[0096] In formula (6), the temperature coefficient The value is set to 0.05 to sharpen the distribution, thereby amplifying differences between similar samples (e.g., ...). =0.8 and =0.5 has a stronger distinguishing power. The smaller the value, the stronger the model's ability to distinguish similarities. (Denominator) The effect is equivalent to the InfoNCE loss function in contrastive learning, forcing the model to distinguish between positive and negative samples. This denominator can be approximated using random sampling to reduce computational complexity. Furthermore, due to the symmetry of the inner product, therefore... .

[0097] The above example optimizes through contrastive learning, allowing semantically similar entity vectors to form dense clusters in the semantic space, providing a high-quality feature foundation for the intelligent generation of knowledge topics.

[0098] The following describes how to determine topological affinity: For example, the improved PageRank algorithm of this application is used to determine the relationship between two entities based on the corresponding nodes of each entity in the knowledge graph data. and The topological correlation between them is shown in the following formula (7) of the improved PageRank algorithm: , formula (7); in, Represents a node The weight value (i.e., PageRank value) measures the overall importance of a node in the global network (the higher the value, the more important it is). is the damping coefficient (dynamically adjustable from 0.65 to 0.9, typically set to 0.85), representing the probability of a user randomly jumping to another node (to prevent "isolated nodes" from becoming inaccessible). N is the total number of nodes in the knowledge graph. This represents the set of all incoming edge nodes pointing to node u (i.e., upstream nodes that have voting rights on u). This represents the PageRank value of node v. This represents the edge weight from node v to node u. This represents the edge weight from node v to node m, where m is the node index variable. This represents the sum of the weights of all outgoing edges from node v.

[0099] Based on physical topology parameters (such as fiber optic cable length and port availability) and combined with business-related prior knowledge, the PageRank algorithm can be more closely aligned with actual business scenarios, avoiding purely mathematical calculations that ignore business logic. For example: Physical topology parameters, such as fiber optic cable length (affecting signal attenuation or delay) and port availability (reflecting bandwidth or reliability), can be used as... The basic input.

[0100] Prior business knowledge, such as node type weighting (e.g., higher weight for core nodes and lower weight for edge nodes) and business importance (e.g., assigning greater weight to nodes with high security levels), are weighting factors artificially set based on business experience or domain rules. The weighting coefficients.

[0101] The improved PageRank algorithm can also be represented by the following formula (8): , formula (8); By comparing formulas (7) and (8), it can be found that the difference between formulas (7) and (8) is that the first addend in formula (7) is "(1-d) / N", while the first addend in formula (8) is "(1-d)". Formula (7) means that when randomly jumping, the probability of jumping to any node in the graph is equal, which is suitable for scenarios such as webpage ranking that require fairness. Formula (8) means that when randomly jumping, the probability of jumping to each node in the graph is not equal, and it can favor certain important nodes, which is suitable for scenarios such as recommendation systems and knowledge graph association analysis.

[0102] The improvements of the above-mentioned improved PageRank algorithm compared to the traditional PageRank algorithm include: First, it has been increased. Its value is derived from network topology characteristics (such as fiber optic cable length, port availability, and device connection strength). For example, the shorter the fiber optic cable and the higher the port availability, the better. The larger the value, the stronger the contribution of the connection to the node's importance. This addresses the shortcoming of traditional algorithms that "ignore edge heterogeneity," making the calculation of node importance more closely reflect actual network topology.

[0103] Second, d is set to a dynamically adjustable value to adapt to changes in network state during fault propagation, avoiding the convergence delay problem caused by fixing the d value in traditional algorithms.

[0104] In entity Corresponding node v, entity Taking node u as an example, the degree of topological association As shown in the following formula (9): , formula (9); Formula (9) is used to take the absolute value of the difference between the RR values ​​of the corresponding nodes of two entities as the topological correlation degree of the two entities in the topological space, thus realizing the efficient determination of the correlation degree of the two entities in the topological space.

[0105] Topological correlation Alternatively, it can be shown in the following formula (10): , formula (10); Where N(v) represents the set of neighboring nodes of node v (including all nodes directly connected to node v), used to limit the calculation scope so that the topological association degree only considers the local topological structure. k represents the node number in N(v).

[0106] In formula (10), It can represent the relative importance of node u's PageRank value in its neighbor set (i.e., the proportion of neighbor importance). The weight of edge (v, u) is the proportion of the sum of all edge weights of node v (i.e., edge weight percentage), used to suppress the excessive influence of height-numbered nodes on the correlation calculation (such as the dominance effect of super nodes in social networks). This example, by combining the proportion of neighbor importance and the proportion of edge weight, can simultaneously measure the global importance (PageRank) and local connection strength (edge ​​weight) of a node, and can capture weak connections between key hub nodes (e.g., the relationship between low-weight but high-PageRank nodes).

[0107] The following example illustrates formula (10): Suppose a simple network: the adjacency matrix of node v (corresponding to entity i) include and , , The sum of the edge weights is 1 (0.8 + 0.2). The PageRank value of node (corresponding to entity j1) is 0.6. The PageRank value of the node (corresponding to entity j2) is 0.4; Neighbor set N(v) = { , The sum of the PageRank values ​​of the neighboring nodes is 1 (0.6 + 0.4).

[0108] Then, node v and node Topological correlation Node v and node Topological correlation .

[0109] In practical applications that determine topological affinity based on formula (10), the adjacency matrix can be preprocessed as follows: If the graph is undirected, it must be ensured that the adjacency matrix of node v is symmetric. ; If it is a weighted graph, the edge weights need to be standardized (e.g., minimum-maximum scaling).

[0110] It should be understood that in the embodiments of determining topological association based on the PageRank method, the dimension of the topological space is 1, i.e., the PageRank value.

[0111] For example, using a sparse matrix to store the adjacency matrix can effectively reduce the storage space requirement, which is especially suitable for processing large-scale knowledge graph data.

[0112] For example, using a distributed computing framework (such as Spark GraphX) to process topological association calculations can improve computational efficiency, especially when the knowledge graph is large and has a large number of nodes and edges. Distributed computing frameworks can distribute the topological association calculation task across multiple computing nodes for parallel execution, reducing overall computation time by fully utilizing the cluster's computing resources.

[0113] This application's embodiments are based on an improved PageRank algorithm, capable of accurately determining the association strength between two entities in the topological space. Specifically, the improved PageRank algorithm introduces a damping factor that can be dynamically adjusted based on the entity's network location and attributes. When an entity is located in the core of the network and connects to important nodes, the damping factor value is reduced to highlight its global influence; conversely, if the entity is located at the network edge, the damping factor value is appropriately increased to focus on local association. Furthermore, the improved PageRank algorithm incorporates edge weights, unlike traditional methods that only consider the presence or absence of node connections. This application's solution assigns different weights to different edges based on their connection attributes and strength. Therefore, by integrating the dynamically adjusted damping factor and edge weights into a product model of "neighbor importance ratio" and "edge weight ratio" for calculating topological association, the improved PageRank algorithm achieves a precise measurement that considers both the global influence of nodes and the strength of local connections. This scheme enables correlation assessment to move beyond static network structures and adapt to changes in business status. It also effectively suppresses the dominance effect of high-number nodes, thus more accurately identifying important correlations in real business scenarios (such as telecommunications network fault propagation analysis) that may not have the highest connection strength but occupy key hub positions in the topology. This significantly improves the reliability and interpretability of topological evidence chains in knowledge topics.

[0114] In a feasible design, the topological association degree between any two entities is determined based on the attributes and / or connections of the corresponding nodes in the knowledge graph data, including: Obtain the embedding representation matrix, adjacency matrix, and topology weight matrix of each entity's corresponding node in the knowledge graph data. The topology weight matrix is ​​a matrix composed of network topology feature parameters. Based on the topology-aware graph convolutional network model, the embedding representation matrix, adjacency matrix and topology weight matrix of each entity's corresponding node are fused to update the embedding representation matrix of each entity's corresponding node. The updated embedding representation matrix contains semantic information and topology information. The topological association degree between any two entities is determined based on the updated embedding representation matrix of the corresponding nodes of any two entities.

[0115] Network topology characteristic parameters include, but are not limited to, parameters such as optical cable length and port availability.

[0116] For example, the embedding representation matrix of a node is generated by encoding the textual description information of the node using a distilled BERT model.

[0117] The following is a brief introduction to the data processing process of the Topology Aware Graph Convolutional Network (T-GCN) model: Step 1: Receive the node's embedding representation matrix (node_emb), the original adjacency matrix (adj_matrix), and the topology weight matrix (topology_weight); where the topology weight matrix includes parameters such as fiber optic cable length and port availability. Step 2: The original adjacency matrix and the topological weight matrix are weighted and fused to generate an enhanced adjacency matrix (enhanced_adj). The fusion of topological features and connectivity relationships is achieved through element-wise multiplication. Step 3: Perform matrix multiplication on the enhanced adjacency matrix and the node embedding representation matrix to achieve topology-aware propagation of node features; Step 4: Output the updated node embedding representation matrix after topology-aware graph convolution processing, as the feature representation result of the network state.

[0118] Since the updated node's embedding representation matrix is ​​obtained by fusing the adjacency matrix and topological weight matrix that include topological information, as well as the embedding representation matrix that includes semantic information, the updated node's embedding representation matrix includes both semantic and topological information.

[0119] For example, the topological association degree between two entities is determined based on the updated embedding representation matrix of the corresponding nodes of the two entities in the following way: The cosine similarity of the updated embedding representation matrices of the corresponding nodes of two entities is used to determine the topological association between the two entities.

[0120] It should be understood that the dimension of the topological space in the above embodiments for determining topological correlation is the same as the output dimension of the T-GCN model.

[0121] In this embodiment, the initial semantic embedding of entity nodes, basic connection relationships, and topology weight matrix reflecting the actual physical link quality are input into a topology-aware graph convolutional network for fusion processing to generate an enhanced node representation that simultaneously contains semantic features and topology context. This allows the final calculated topology correlation degree to not only capture the direct connection relationships between entities, but also to deeply characterize their relative position in the network structure, connection quality, and business roles, among other complex topology semantics. This overcomes the limitations of traditional methods that calculate correlation degree based solely on static connections or single attributes, and improves the accuracy of topology correlation degree.

[0122] For example, the steps for determining the topological association degree between any two entities based on the connection relationships of corresponding nodes in the knowledge graph data include: Based on the degree centrality and betweenness centrality of the corresponding nodes of entities, the topological association degree between any two entities is determined as shown in the following formula (11): , formula (11); in, JSD represents the topological correlation between entity i and entity j. This represents the topological feature distribution vector of node v corresponding to entity i. This represents the topological feature distribution vector of node u corresponding to entity j. This represents the divergence calculation between two distribution vectors.

[0123] The following formula (12) is used for calculation: , formula (12); Wherein, D( ) is a node Degree centrality, ( ) represents a node betweenness centrality, This represents the maximum degree centrality of all nodes in the network. T represents the maximum betweenness centrality of all nodes in the network, and T represents the transpose.

[0124] For example, D( is determined using the following formula (13)). ): , formula (13); In other words, nodes Degree centrality is achieved by using nodes The weighted number of adjacent edges (i.e., the sum of the weights of all outgoing edges of node v) is used to represent the number of adjacent edges.

[0125] For example, O( ) is determined using the following formula (14). ): , formula (14); Where h1 represents the source node and h2 represents the target node. This represents the total number of shortest paths from source node h1 to target node h2 (without specifying whether the path passes through node v). The number of paths that pass through node v in the shortest path from source node h1 to target node h2. node v 1, h2 v

[0126] In other words, nodes The betweenness centrality of node v is represented by the proportion of node v in the shortest path of the entire network.

[0127] See the method for determination. This will not be elaborated upon here.

[0128] It should be understood that the topological space in this embodiment for determining topological affinity has a dimension of 2, with the first dimension being degree centrality and the second dimension being betweenness centrality. ,in, It is the maximum value related to the total number of network nodes (such as the maximum degree centrality or maximum betweenness centrality of nodes in the network), used to normalize the topological features to a uniform interval, ensuring that the topological features of networks of different sizes are comparable.

[0129] The above example determines the topological association between any two entities based on the degree centrality and betweenness centrality of the corresponding nodes of the entities. This can effectively characterize the structural similarity and connection tightness between entities in the network, and thus reflect their relative importance in the topological space.

[0130] The following describes how to determine the temporal tightness: In a feasible design, the temporal density of any two entities can be determined based on their temporal attributes in the following way: Based on the adaptive sliding window mechanism, the size of the current processing window is dynamically determined according to the event inflow density; The tightness of the time difference between the occurrence times of two entities is determined using a Gaussian kernel function based on the size of the current processing window; this is known as the time tightness.

[0131] For example, the following formula (15) is used to implement an adaptive sliding window mechanism to determine the size of the current processing window based on the event inflow density: , formula (15); Where n represents the number of steps in the current processing window, Indicates the size of the current processing window (range 30s-15min). This indicates the size of the processing window in the previous step. This indicates a high event density threshold (e.g., a value of 10 events / second). This indicates a low event density threshold (e.g., a value of 2 events / second).

[0132] It should be noted that it can also be used Indicates the current processing window.

[0133] For example, the following formulas (16) and (17) are used to determine the tightness of the difference in the occurrence time of two entities based on the size of the current processing window using a Gaussian kernel function: , formula (16); , formula (17); in, This represents the temporal tightness between entity i and entity j. Indicates the time when the event occurred for entity i. This indicates the time when the event occurred for entity j. The width parameter of the Gaussian kernel function is used to control time sensitivity.

[0134] In formula (16), when the time difference between two entities increases, the density index decreases, which effectively reflects the decay characteristics of the dynamic temporal relationship between events. The larger the value, the flatter the curve (allowing for a larger time span correlation). The smaller the value, the sharper the curve (significant correlation only between immediately adjacent events), thus achieving control over time sensitivity. This is the dynamic span of the sliding window, its value adaptively adjusted by the event density to ensure focus on recent correlations under high load and moderately relax time constraints under low density. By... Substitute into formula (17) to calculate The width of the Gaussian kernel can be dynamically adjusted. That is, when the event density is high, Shrink This decreases accordingly, recognizing only closely related temporal sequences; when event density is low, As the normalization coefficient increases, σ also expands, allowing for more relaxed time correlations. It enables the Gaussian kernel function to integrate to 1 in the time domain (a property of the probability density function), thus avoiding numerical scaling bias.

[0135] The above example uses an adaptive mechanism to achieve accurate modeling of temporal density, enabling the system to maintain sensitivity and robustness to temporal relationships under different event density scenarios.

[0136] For example, the temporal tightness of two entities is calculated by the event frequency within the sliding window, as shown in the following formula (18): , formula (18); in, This represents the temporal tightness between entity i and entity j. This represents the current timestamp (i.e., the end of the calculated time window). Indicates the size of the time window. This represents the time index, used to iterate through each time unit (e.g., per second) within the window. As an indicator function, when i and j are in time The value is 1 if both conditions are met, otherwise it is 0. express . Indicates the most recent statistics The total number of times i and j appear together within a given time period.

[0137] The aforementioned example uses a sliding window to limit the temporal characteristics between entities, enabling the correlation analysis to focus on the dynamic analysis of recent events and ensuring that knowledge topics are updated in real time as business changes occur (such as fault chain prediction).

[0138] For example, the reciprocal of the time difference between the occurrence of events of two entities is determined as the temporal closeness of the two entities, as shown in the following formula (19): , formula (19); in, This represents the temporal tightness between entity i and entity j. This represents a very small positive number and is used to avoid having a denominator of 0.

[0139] The above example uses a simple calculation method, but its sensitivity to time differences increases non-linearly. It is suitable for scenarios where high accuracy of time correlation is required and event distribution is relatively uneven.

[0140] Based on the foregoing explanations regarding semantic similarity, topological relevance, and temporal tightness, it can be seen that for any entity pair (i,j), its multimodal association feature vector... It can be shown in the following formula (20): , formula (20); in, This represents the semantic similarity between entity i and entity j. This represents the topological association degree between entity i and entity j. This represents the temporal tightness between entity i and entity j.

[0141] Based on the above embodiments, the following formula (21) can be used to achieve this: the semantic similarity of any two entities is weighted by the first weight coefficient, the topological relevance of any two entities is weighted by the second weight coefficient, and the temporal density of any two entities is weighted by the third weight coefficient. The weighted semantic similarity, topological relevance, and temporal density are then summed to obtain the comprehensive relevance: , formula (21); in, This represents the first weighting coefficient that is dynamically adjusted over time t. This represents the second weighting coefficient that is dynamically adjusted over time t. This represents the third weighting coefficient that is dynamically adjusted over time t. This represents the overall correlation between entity i and entity j. The first, second, and third weighting coefficients satisfy the constraint that their sum is 1, which ensures that the contributions of each modal feature are balanced and avoids the dominance of any one type of feature.

[0142] For example, the following formula (22) is used to determine : , formula (22); Where e is the natural index and t represents time. It is a sensitivity coefficient used to control the influence of semantic modality (or semantic features) on the final weight coefficient. This is an index function used to reflect the importance of semantic modalities at time t. For example: It can be the signal-to-noise ratio of text information in a real-time data stream; the higher the signal-to-noise ratio, the more important the semantics.

[0143] It is a sensitivity coefficient used to control the influence of topological modes (or topological features) on the final weighting coefficients. This is an index function used to reflect the importance of topological modes at time t. For example: It could be the stability of the network topology; the more drastic the topology changes, the more important the topology information becomes.

[0144] It is a sensitivity coefficient used to control the influence of temporal modalities (or temporal features) on the final weighting coefficients. This is an index function used to reflect the importance of time series modes at time t. For example: It can be the frequency of the event; the higher the frequency, the more important the temporal correlation.

[0145] Let t be the weight of semantic similarity in the overall relevance at time t.

[0146] For example, the following formula (23) is used to determine : , formula (23); Let t be the weight of topological correlation in overall correlation at time t.

[0147] For example, the following formula (24) is used to determine : , formula (24); Let t be the weight of temporal sequence tightness in the overall correlation degree at time t. This represents the size of the processing window corresponding to the computational timing density. Increase (increase the frequency of events or the short time interval). The function result increases. Increasing the proportion can enhance the contribution of temporal features; when Reduce (event sparsity or long time intervals). The function result decreases. Reduce the weight of time-series features proportionally.

[0148] For quantifiable index functions , and The weight coefficients can be determined using formulas (21)-(23). However, for modal indicators that are difficult to quantify, the weights can be dynamically allocated through predefined rules.

[0149] For example, the values ​​of each weight coefficient can be dynamically allocated based on real-time monitoring indicators (such as traffic peaks and sudden events); For example, the values ​​of each weight coefficient can be dynamically allocated according to the application scenario of the knowledge topic. In the social recommendation scenario, during the cold start phase for new users, the weight coefficient can be increased. (Semantic feature weights) are used to match interests based on registration information. During the active user phase, enhancements are implemented. (Time-series feature weights) track real-time click behavior. For industrial equipment fault prediction scenarios, during steady-state operation, improve... (Topology feature weights) Analyze device topology dependencies. During periods of abnormal fluctuations, improve... , to capture signal mutations.

[0150] In practical applications, it can be set The default value is 0.5. The default value is 0.3. The default value is 0.2.

[0151] For example, the weight coefficients can also be dynamically adjusted based on the Hidden Markov Model, including the following steps: Step 1, Obtain Observation Status: Monitor the characteristics of the data stream output from the streaming data buffer layer in real time, for example: Event stream entropy: the degree of data disorder (a higher entropy value indicates that the timing is more important); Semantic similarity variance: topic concentration (smaller variance indicates greater semantic importance); Topology change rate: Network stability (the faster the change, the more important the topology). Step 2, Obtain Hidden States: There are three preset states representing different weight preference modes, for example: State 1 (Semantic Dominant): high, Low, medium; State 2 (Topological Key): Low, high, medium; State 3 (Time-Sensitive): medium, wait, high.

[0152] Step 3, State Decoding and Output: Based on the characteristics of the observed data stream, a Hidden Markov Model (HMM) is used to infer the most likely hidden state using the Viterbi algorithm. This allows for the selection of the corresponding weight coefficient combination based on this state. , , ).

[0153] The above example dynamically adjusts the weight coefficients based on a Hidden Markov Model (HMM). By sensing features such as entropy, semantic variance, and topological change rate of the data stream in real time, it transforms static weight allocation into an intelligent decision-making process that adapts to changes in business scenarios. For example, it automatically increases the weight coefficients of temporal features during a fault outbreak to quickly clarify causality; enhances the weight coefficients of semantic features during knowledge retrieval to accurately match topics; and emphasizes the weight coefficients of topological features during network reconstruction to ensure structural reasoning. This allows the application to maintain sensitivity and responsiveness to the most critical feature dimensions in complex multimodal data environments, effectively improving the adaptability and accuracy of the solution in dynamic scenarios.

[0154] Based on the definition of a multimodal association feature space, this application determines the comprehensive association degree between entities by integrating features from three dimensions: semantic, topological, and temporal, thereby improving the accuracy and comprehensiveness of association analysis. Specifically, it uses a natural language processing model to extract semantic feature vectors of entities and calculates semantic similarity to accurately grasp the semantic-level association between entities; it determines topological association degree based on the attributes and connections of nodes in the knowledge graph to reflect the structural relevance of entities; and it combines the temporal attributes of entities to assess temporal tightness, capturing the connections between entities in the time dimension. By dynamically adjusting the first, second, and third weight coefficients, it weights and sums these three similarities to obtain the comprehensive association degree, enabling the automatic emphasis or weakening of the influence of a certain dimension according to different scenarios, enhancing the adaptability and flexibility of association analysis, thereby more accurately identifying and quantifying complex relationships between entities, and providing a solid foundation for the generation of knowledge topics.

[0155] For example, this implementation uses the Flink framework to process multi-source heterogeneous data to obtain at least two structured entities; and to determine the comprehensive correlation degree between any two of the at least two entities. This embodiment utilizes the Flink framework for stream and batch processing of real-time data streams to meet the real-time analysis and computation needs of large-scale knowledge graph data.

[0156] For example, the Flink framework can use a fixed-length time window, such as a 5-second or 10-second rolling window, to enable periodic batch processing of the data stream.

[0157] For example, the method further includes: Obtain current network load metrics (such as CPU utilization, network bandwidth utilization, or request queue depth), and apply the following windowing optimization scheme based on network load metrics to the Flink framework's time window. The specific steps include: Step 1: Real-time load monitoring Get the current network load metrics.

[0158] Step 2: Calculate the dynamic window size Based on the network load metric value from step 1, dynamically calculate the window size: By performing linear interpolation calculations within a preset range of minimum and maximum window sizes (minSize and maxSize), the precise length of the current processing window is dynamically determined, thereby achieving adaptive adjustment of the window size to system load. Specifically, when the load exceeds a preset threshold (e.g., CPU utilization exceeds 80%), a window shrinking strategy is triggered. Step 3: Window start time alignment Based on the event timestamp, the starting boundary time point of the window is determined through an offset alignment mechanism to ensure the continuity of the time window as an integer multiple.

[0159] Step 4: Dynamic Window Instance Generation Based on the window size obtained in step 2 and the start time calculated in step 3, a corresponding sliding time window instance (TimeWindow object) is generated as a container for processing the current data element.

[0160] The above example, based on network load metrics, dynamically optimizes the Flink time window, effectively balancing overall throughput and processing latency. It automatically shrinks the window under high load (e.g., from 10 seconds to 3 seconds) to reduce the risk of data backlog in a single window. Furthermore, hard constraints on minSize / maxSize prevent window failure under extreme loads (e.g., minSize=1 second avoids excessive window fragmentation). Compared to traditional solutions that periodically perform full analysis based on the knowledge graph, this window optimization strategy allows the system to focus only on recent events. Entity recognition or correlation analysis based on recent events improves the response efficiency and accuracy of real-time analysis and shortens the update interval of knowledge topics. Flink's sliding window mechanism limits the system to processing only newly added events within the window, achieving efficient aggregation of recent data and low-latency response, supporting second-level updates of knowledge topics.

[0161] The above examples are applicable to scenarios with sudden surges in traffic (such as e-commerce flash sales or communication network congestion), balancing the real-time and accuracy requirements of computing resources.

[0162] In a feasible design, before determining the overall association degree between any two of the at least two entities, the method further includes: screening the at least two entities, the screening steps including: Real-time acquisition of data streams of entities identified from the message queue of the message middleware; The data stream is grouped according to the entity identifier of the entity, and data with the same entity identifier are divided into the same processing group; The length of the current processing window is dynamically determined based on network load metrics, and the starting boundary time point of the processing window is determined based on the event timestamp. A sliding time window instance is generated by combining the length of the current processing window. Within each sliding time window instance, dynamic association aggregation processing is performed on entities in each grouped processing group. The dynamic association aggregation processing is based on time decay factor, cross-domain adjustment factor and feature filtering threshold, and outputs an aggregation score that reflects the degree of dynamic association between entities. Delete entity pairs consisting of two entities whose aggregation score is less than the first preset threshold.

[0163] For example, dynamic association aggregation processing is implemented using the following formula (25): , formula (25) in, Indicates the aggregate score. This is a time decay factor used to adjust the rate at which the weight of new events is increased. The cross-domain adjustment factor is used to enhance the association strength between corresponding nodes of entities across physical / logical regions. Spatial weights represent the physical topological importance of entities. Euclidean distance is used to quantify the spatial distance between entities. Feature similarity refers to the degree of similarity between entities' inherent attributes (such as directly usable attributes like device model and OS version), which can be calculated using cosine similarity or Jaccard similarity. This is the feature filtering threshold, used to discard entities with a similarity value lower than this value.

[0164] “ "This is a conditional triggering mechanism. Only when the features between entities are sufficiently similar will they contribute a score; otherwise, this item is zero, ensuring that only truly relevant events will be associated."

[0165] The above example uses Flink's windowing mechanism to quickly identify highly correlated entities in the data stream based on the inherent attributes of the entities, achieving efficient filtering of massive streaming data and thus improving the efficiency of subsequent comprehensive correlation analysis.

[0166] For example, in order to reduce the computational overhead of low-value nodes, before determining the comprehensive correlation degree between any two entities among at least two entities, a preliminary screening of entities is performed based on topological characteristics such as the degree centrality and / or betweenness centrality of the corresponding nodes of the entities.

[0167] In a feasible design, the method also includes: Based on the key-value pair structure, the association results of the entity pairs consisting of two entities after association degree analysis are cached in the association result cache container. The association results include the identifier of the entity pair, the semantic similarity between the entity pairs, the topological association degree, the temporal tightness or the comprehensive association degree. The key includes the identifier of the entity pair, and the value includes the semantic similarity, topological association degree, temporal tightness or comprehensive association degree between the entity pairs. The initial memory allocation strategy of the association result cache container adopts a dynamic expansion mechanism. When the number of stored entity pairs reaches a second preset threshold, the Least Recently Used (LRU) algorithm is used to evict the least frequently accessed cache entry in the association result cache container. Cache entries include key-value pairs; and / or, Set an expiration date for cache entries and trigger an asynchronous cleanup thread to delete expired cache entries (cache entries also include an expiration date) the next time an expired cache entry is accessed. When the queried entity pair does not exist or the cache entry to which it belongs has expired, the overall relevance of the queried entity pair is automatically determined based on the registered callback function. The corresponding key-value pair is determined based on the overall relevance of the queried entity pair and written into the association result cache container, and the access timestamp is updated.

[0168] The callback function is used to dynamically trigger the comprehensive correlation calculation task when the cache is missed.

[0169] In the above embodiments, by using the Least Recently Used algorithm to evict the cached entries with the lowest access frequency, memory resources are effectively released, and performance degradation caused by cache bloat is avoided. The expiration mechanism and asynchronous cleanup strategy effectively balance memory resource consumption and query efficiency. The on-demand calculation mechanism based on callback functions ensures the timeliness and integrity of cached data and avoids redundant calculation overhead.

[0170] For example, the association result caching container introduces a multi-level caching structure to optimize performance in high-load scenarios, specifically configured as follows: Setting a memory cache (such as a Guava Cache instance) as a first-level cache can handle more than 90% of high-frequency requests; Configure a distributed cache (such as a Redis cluster) as a second-level cache to store historically high-frequency entries and reduce cross-node computation; Configure a persistent database (such as HBase) as a three-level cache for backing up the full data and supporting disaster recovery.

[0171] For example, key parameters of the associated result caching container are dynamically adjusted based on real-time monitoring data to achieve adaptive optimization: During peak traffic periods, the validity period of cache entries is automatically shortened to improve data freshness; when a memory pressure warning is triggered, the capacity threshold is reduced and the eviction frequency is increased to ensure operational stability; during periods when computing resources are relatively idle, hot data is proactively preloaded into the cache, thereby improving the overall cache hit rate and resource utilization efficiency.

[0172] S140, based on the comprehensive correlation between any two entities, cluster at least two entities to obtain at least one entity group.

[0173] For example, at least one entity group can be obtained by clustering at least two entities based on the comprehensive correlation between any two entities in the following way: Generate an entity association matrix based on the comprehensive association score of all entity pairs; A weighted knowledge graph is constructed based on the entity association matrix, where nodes correspond to entities and edge weights are the comprehensive association degree of corresponding entity pairs; Perform group association analysis on the weighted knowledge graph. Group association analysis includes: deleting edges with association degree lower than the preset edge weight threshold; and clustering at least two entities using a community detection algorithm to obtain highly cohesive entity groups.

[0174] S150, select each target entity group from at least one entity group according to the first preset condition.

[0175] For example, this is achieved by filtering target entity groups from at least one entity group based on a first preset condition: The average correlation of all entity pairs within each entity group is taken as the group correlation degree of that group. Based on a preset group correlation degree threshold, target entity groups with a group correlation degree greater than the preset group correlation degree threshold are selected from the candidate entity groups. That is, the first preset condition is "group correlation degree greater than the preset group correlation degree threshold".

[0176] The above example effectively identifies closely related target entity groups from a large number of entities through a clustering and filtering process based on comprehensive relevance, thereby improving the accuracy and relevance of knowledge topic generation.

[0177] For example, when a new entity is added, only its overall correlation with the current core entities of the community is calculated (by identifying the core entity path through the Floyd-Warshall algorithm), reducing computational complexity and enabling incremental calculation of the overall correlation.

[0178] For example, in order to optimize the storage and persistence of intermediate state data in the Flink real-time computing engine, this application uses RocksDB as the state backend, that is, the intermediate state data is stored on disk (instead of memory), and read and write are accelerated through memory caching (write-buffer), thereby supporting large-scale state data management.

[0179] For example, incremental checkpointing is also enabled, which means that only intermediate state data that has changed since the last checkpoint is persisted instead of the full data, thereby reducing the amount of IO operations, shortening the persistence time, and avoiding job lag caused by full checkpointing.

[0180] The intermediate state data mainly includes: The structured entity data output by the aforementioned steps; Data related to dynamically generated sliding time window instances, such as the window's initial boundary time point, the current processing window length, and the set of entity events to be processed within the window; Within each sliding time window instance, the aggregate score between entities is calculated based on the time decay factor, cross-domain adjustment factor, and feature filtering threshold; Temporary results generated during multimodal association analysis, such as vector representations of entities generated during semantic similarity calculation, PageRank values ​​or intermediate embedding vectors of the T-GCN model generated during topological association calculation, and results of Gaussian kernel function calculation during temporal density calculation.

[0181] For example, the difference propagation algorithm is used to perform comprehensive correlation analysis only on the changed entities to achieve incremental calculation optimization, thereby supporting second-level updates of knowledge topics.

[0182] In this context, "changed entities" refers to newly added structured entities that are output during system operation relative to the previous calculation state. For example, after collecting newly stored text data, newly added data in the graph database, or newly added entity events in the data stream, an incremental update mechanism is triggered to parse this newly added data and output the new entities.

[0183] S160 generates corresponding knowledge topics based on each target entity group.

[0184] In a feasible design, the generation of corresponding knowledge topics based on each target entity group can be achieved in the following way: Based on the textual description information of each entity in the target entity group, generate the title of the knowledge topic; Based on the comprehensive correlation and correlation path between entities within the target entity group, construct a chain of related evidence for the knowledge topic; If the group correlation degree and / or business risk indicators of the target entity group meet the second preset condition, a disposal suggestion is matched or generated from the preset disposal suggestion library. The title, related evidence chain and disposal suggestion are integrated to generate a structured knowledge topic report and display it. The group correlation degree is the average of the comprehensive correlation degree of each entity pair in the target entity group. Each entity pair is based on any two entities in the target entity group. Otherwise, the title and related evidence chain are integrated to generate a structured knowledge-based report and presented.

[0185] For example, the title of a knowledge topic can be generated based on the textual description information of each entity in the target entity group in the following way: The common words of each entity in the target entity group are identified by frequent itemset mining (Apriori algorithm) (such as "power supply failure"), and titles are generated by combining them with the Long Short Term Memory (LSTM) network model (such as "Special Analysis of Overload of Power Supply Link of Base Station A").

[0186] For example, this can be achieved by constructing a chain of evidence for the knowledge topic based on the comprehensive correlation and correlation path between entities within the target entity group: Entity pairs with a comprehensive correlation degree greater than a preset comprehensive correlation degree threshold are extracted, related events are arranged in chronological order, and topological paths (such as "base station A → optical module B → power supply equipment C") are extracted through a graph convolutional network model to form a chain of evidence.

[0187] As a further example, the entities in the target entity group are matched with the work order database (which can be based on fuzzy search in Elasticsearch), and similar cases are output as part of the evidence chain.

[0188] For example, business risk indicators are related to the application scenarios of knowledge topics. In a communication network operation and maintenance scenario, business risk indicators include the probability of interruption. The second preset condition can be set according to the needs of different business scenarios, such as the group correlation degree of the target entity group being greater than the threshold of group correlation degree, and / or the business risk indicator of the target entity group being greater than the threshold of business risk indicator.

[0189] For example, in a communication network operation and maintenance scenario, if the group correlation degree of the target entity group is greater than the threshold of the group correlation degree set by the second preset condition, and the interruption probability is greater than the threshold of the business risk indicator set by the second preset condition, then a handling suggestion will be matched or generated from the preset handling suggestion library.

[0190] For example, the interruption probability can be determined using the following formula (26): , formula (26); in, This represents the probability of interruption of the target entity group. This represents the index of an entity within the target entity group. Indicates the number of entities in the target entity group. This refers to the entity with index ct. Representing entities In the weight of the target entity group, Represents entities within a specified statistical time window. Number of failures This represents the total number of observations within the statistical time window. Represents entities within the statistical time window The failure rate.

[0191] The following section uses 5G network fault analysis as an example to introduce the knowledge topic generation process.

[0192] (1) A standardized template structure (including a data layer, a logic layer and a presentation layer) is preset and mapped to a data interface. The data layer maps the alarm title, topology root node and handling suggestion set in the network alarm database; the logic layer has built-in relational graph rendering and list traversal instructions; the presentation layer binds HyperText Markup Language (HTML) semantic tags and visual style classes to ensure that the template can adapt to multi-source data.

[0193] (2) Module dynamically renders alarm titles: Extract the title field of the alarm event, inject it into the HTML title node through the text interpolation engine, and verify the legality of the text (such as non-empty, no illegal characters). At the same time, add an identifier according to the risk level (such as adding a red badge for high-risk alarms).

[0194] (3) Generate a network topology graph as the core of the evidence chain: Based on the instruction parameter depth (depth=3), the comprehensive correlation between entities and the correlation path (by traversing the knowledge graph according to the names of entities in the target entity group, the correlation path between each entity in the target entity group is determined), starting from the root node of the topology (e.g., the fault source device), traverse the three layers of adjacent nodes (e.g., the associated base stations, transmission equipment, and core network elements adjacent to the fault source device), dynamically load node attributes (e.g., including device model, load status, geographical location, etc.), and generate Scalable Vector Graphics (SVG) vector graphics, while binding interactive events (e.g., when a node is clicked, the relevant solutions in the evidence chain are highlighted).

[0195] (4) Rendering the Disposal Recommendation List: If the group correlation degree and / or business risk indicators of the target entity group meet the second preset condition, then a disposal recommendation is matched or generated from the preset disposal recommendation library, and differentiated rendering and filtering are performed; otherwise, this step is skipped. Data filtering includes removing invalid disposal recommendations, such as those with a status of "deprecated" or those that do not match the compatible device model. Differentiated rendering is used to perform graded rendering based on the risk level of the solution (disposal recommendations are associated with risk levels), for example: For high-risk solutions (risk level ≥ 8), add a flashing border warning style; For medium-risk options (4 < risk level < 8), display an operation confirmation dialog box; For low-risk solutions (risk level ≤ 4), the solution is collapsed by default, but can be expanded as needed. (5) Output a report that integrates multimodal data: Integrate the title, related evidence chain and disposal recommendations (such as meeting the conditions) to generate a structured knowledge topic report, and display it as an interactive HTML5 document. It also supports conversion to PDF (which can use asynchronous pagination rendering technology), JSON or mixed text and image briefing.

[0196] The entire process described above uses dynamic rendering and conditional filtering of entity groups to ensure that the generated knowledge topics for 5G network fault analysis can reflect the network status in real time and highlight key fault associations and handling measures.

[0197] For example, a RESTful API service layer (Spring Boot) is used to implement conditional retrieval of knowledge topics.

[0198] This embodiment achieves a structured output of knowledge topics from content construction to decision support by dynamically generating titles based on entity text descriptions, constructing evidence chains based on the comprehensive correlation and association paths between entities, and selectively integrating disposal suggestions according to the conditions of group correlation and business risk indicators. The display of evidence chains and comprehensive correlation can enhance user trust. In addition, the above embodiment not only ensures the integrity and logical consistency of the internal elements (title, evidence, and suggestions) of the topic report, but also realizes the on-demand output of disposal suggestions through a conditional judgment mechanism. Thus, while ensuring the accuracy of the report information, it significantly improves its practicality and pertinence in fault location and operation and maintenance decision-making scenarios.

[0199] This application's embodiments overcome the problems of delayed knowledge updates and incomplete coverage caused by traditional solutions relying on static knowledge bases by automatically acquiring multi-source heterogeneous data and transforming it into structured entities. Furthermore, by integrating a weighted comprehensive correlation degree based on three dimensions—semantic similarity, topological relevance, and temporal density—it automatically mines deep implicit relationships between entities (such as semantic context, network structure dependencies, and temporal co-occurrence patterns), avoiding the one-sidedness of manually preset rules. Based on this, target entity groups are automatically generated through clustering and conditional filtering, and finally, the knowledge topic generation module outputs the topic. This solution systematically replaces the reliance on traditional knowledge bases and manually preset rules, fundamentally solving the problems of low efficiency and insufficient accuracy in topic generation caused by the difficulty in capturing deep implicit relationships in traditional solutions through data-driven intelligent association mining and group discovery.

[0200] The following is an example of using the knowledge topic generation method of this application for intelligent operation and maintenance fault root cause analysis: Input data includes: Real-time alarm: 10 5G base stations simultaneously reported "RF unit heartbeat loss".

[0201] Topology data: The faulty base station belongs to the same optical cable ring (ID: FIBER_098).

[0202] Association analysis includes: Calculate the overall correlation degree between each entity and determine the target entity group.

[0203] The probability of fiber optic cable interruption in identifying target entity groups is >85%.

[0204] Match historical work orders (keyword: "optical cable splicing error") by the text description information of entities in the target entity group.

[0205] Output topics include: Title: "Rapid Handling Plan for Base Station Outages Due to Fiber Optic Cable Interruption" Linked evidence chain: Fiber optic cable topology diagram + similar historical work order summaries; Recommendations for handling: Step 1: Check the OTDR test results of FIBER_098 (priority: high).

[0206] Step 2: Initiate a cross-departmental collaborative work order (Template ID: TX_2023).

[0207] The following is an example of using the knowledge topic generation method of this application for intelligent aggregation of business logs: Input data: Log keywords: "Payment timeout", "Database connection pool exhausted".

[0208] Business monitoring: QPS suddenly increased by 300%, database response latency > 2s.

[0209] Association analysis includes: The correlation between "Payment Peak" and "Insufficient Connection Pool" was calculated (0.89).

[0210] Output topics include: Title: Database Connection Optimization Guide for High-Concurrency Scenarios

[0211] Optimization suggestions: Short-term: Expand the connection pool (code snippet available on GitHub).

[0212] Long-term: Introduce read-write separation (see appendix for cost analysis).

[0213] like Figure 3 As shown, this application also provides a knowledge topic generation system, including a data acquisition module, an entity recognition module, a multimodal association analysis module, and a knowledge topic generation module; The data acquisition module is configured to acquire heterogeneous data from multiple sources; The entity recognition module is configured to process multi-source heterogeneous data to obtain at least two structured entities; The multimodal association analysis module is configured to determine the comprehensive association degree between any two entities among at least two entities. The comprehensive association degree is obtained by weighted fusion of semantic similarity, topological association degree and temporal tightness between the two entities. Semantic similarity is the similarity between entities in terms of semantics, topological association degree is the association degree between entities in terms of topological structure, and temporal tightness is the tightness between entities in terms of time. The multimodal association analysis module is also configured to cluster at least two entities to obtain at least one entity group based on the comprehensive association degree between any two entities. The multimodal association analysis module is also configured to filter out target entity groups from at least one entity group based on a first preset condition. The knowledge topic generation module is configured to generate corresponding knowledge topics based on each target entity group.

[0214] In a feasible design, the steps of the multimodal association analysis module to determine the comprehensive association degree between any two entities of at least two entities include: Semantic feature vectors of textual description information for each entity are obtained based on natural language processing models; semantic similarity between any two entities is determined based on the semantic feature vectors of each entity. Based on the attributes and / or connections of the corresponding nodes of each entity in the knowledge graph data, determine the topological association degree between any two entities; Based on the temporal attributes of entities, determine the temporal density between any two entities; The semantic similarity of any two entities is weighted by a first weighting coefficient, the topological relevance of any two entities is weighted by a second weighting coefficient, and the temporal tightness of any two entities is weighted by a third weighting coefficient. The weighted semantic similarity, topological relevance, and temporal tightness are then summed to obtain the comprehensive relevance. The first, second, and third weighting coefficients are dynamically adjusted coefficients.

[0215] In a feasible design, the multimodal association analysis module is implemented by determining the topological association degree between any two entities based on the attributes and / or connections of the corresponding nodes in the knowledge graph data: Obtain the embedding representation matrix, adjacency matrix, and topology weight matrix of each entity's corresponding node in the knowledge graph data. The topology weight matrix is ​​a matrix composed of network topology feature parameters. Based on the topology-aware graph convolutional network model, the embedding representation matrix, adjacency matrix and topology weight matrix of each entity's corresponding node are fused to update the embedding representation matrix of each entity's corresponding node. The updated embedding representation matrix contains semantic information and topology information. The topological association degree between any two entities is determined based on the updated embedding representation matrix of the corresponding nodes of any two entities.

[0216] In a feasible design, the knowledge topic generation module generates corresponding knowledge topics based on the target entity group in the following way: Based on the textual description information of each entity in the target entity group, generate the title of the knowledge topic; Based on the comprehensive correlation and correlation path between entities within the target entity group, construct a chain of related evidence for the knowledge topic; If the group correlation degree and / or business risk indicators of the target entity group meet the second preset condition, a disposal suggestion is matched or generated from the preset disposal suggestion library. The title, related evidence chain and disposal suggestion are integrated to generate a structured knowledge topic report and display it. The group correlation degree is the average of the comprehensive correlation degree of each entity pair in the target entity group. Each entity pair is based on any two entities in the target entity group. Otherwise, the title and related evidence chain are integrated to generate a structured knowledge-based report and presented.

[0217] In a feasible design, the multimodal association analysis module determines the temporal closeness between any two entities based on their temporal attributes in the following way: Based on the adaptive sliding window mechanism, the size of the current processing window is dynamically determined according to the event inflow density; The tightness of the time difference between the occurrence times of two entities is determined using a Gaussian kernel function based on the size of the current processing window; this is known as the time tightness.

[0218] In a feasible design, the multimodal association analysis module is configured to screen at least two entities before determining the overall association degree between any two entities. The screening steps include: Real-time acquisition of data streams of entities identified from the message queue of the message middleware; The data stream is grouped according to the entity identifier of the entity, and data with the same entity identifier are divided into the same processing group; The length of the current processing window is dynamically determined based on network load metrics, and the starting boundary time point of the processing window is determined based on the event timestamp. A sliding time window instance is generated by combining the length of the current processing window. Within each sliding time window instance, dynamic association aggregation processing is performed on entities in each grouped processing group. The dynamic association aggregation processing is based on time decay factor, cross-domain adjustment factor and feature filtering threshold, and outputs an aggregation score that reflects the degree of dynamic association between entities. Delete entity pairs consisting of two entities whose aggregation score is less than the first preset threshold.

[0219] In one feasible design, a knowledge graph data storage module is also included. This module stores knowledge graph data and is configured as follows: Knowledge graph data is hierarchically stored in an in-memory graph database, a distributed graph database cluster, and a columnar distributed database based on the latest date of data access. The in-memory graph database is used to store hot data, the distributed graph database cluster is used to store warm data, and the columnar distributed database is used to store cold data.

[0220] In one feasible design, the distributed graph database cluster is configured to create data partitions based on business domains, where the communication business domain corresponds to the main partition container; The distributed graph database cluster is configured to use a geographic region partitioning strategy to divide the wireless network device nodes within the primary partition container into sub-partitions, grouping wireless network device nodes in the same geographic region into the same sub-partition; and / or, The distributed graph database cluster is configured to use a device type hashing strategy to divide the core network device nodes within the main partition container into sub-partitions, and to evenly distribute core network device nodes of the same device type across multiple sub-partitions.

[0221] In one feasible design, an association result caching container is also included. The association result caching container is configured to cache the association results of entity pairs consisting of two entities after association degree analysis based on a key-value pair structure. The association results include the identifier of the entity pair, the semantic similarity between the entity pairs, the topological association degree, the temporal tightness or the comprehensive association degree, where the key includes the identifier of the entity pair, and the value includes the semantic similarity, topological association degree, temporal tightness or the comprehensive association degree between the entity pairs. The initial memory allocation strategy of the association result caching container adopts a dynamic expansion mechanism. The associated result cache container is configured such that when the number of stored entity pairs reaches a second preset threshold, the least recently used cache entry is evicted using a Least Recently Used algorithm; and / or, The associated result cache container is configured to set an expiration date for cache entries and trigger an asynchronous cleanup thread to delete expired cache entries the next time they are accessed. The association result cache container is configured to automatically determine the overall association degree of the queried entity pair based on the registered callback function when the queried entity pair does not exist or the cache entry to which it belongs has expired. Based on the overall association degree of the queried entity pair, the corresponding key-value pair is determined and written into the association result cache container, and the access timestamp is updated.

[0222] Other implementation methods and effects of the above system can be found in the description of the embodiments of the knowledge topic generation method, and will not be repeated here.

[0223] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0224] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0225] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0226] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0227] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0228] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for generating knowledge topics, characterized in that, The method includes: Acquire multi-source heterogeneous data; The multi-source heterogeneous data is processed to obtain at least two structured entities; Determine the comprehensive correlation degree between any two entities among at least two entities. The comprehensive correlation degree is obtained by weighted fusion of semantic similarity, topological correlation degree and temporal tightness between the two entities. The semantic similarity is the similarity between entities in terms of semantics, the topological correlation degree is the correlation between entities in terms of topological structure, and the temporal tightness is the tightness between entities in terms of time. Based on the comprehensive correlation between any two entities, the at least two entities are clustered to obtain at least one entity group. Each target entity group is selected from the at least one entity group according to the first preset condition; Generate corresponding knowledge topics based on each of the target entity groups.

2. The method according to claim 1, characterized in that, Determining the comprehensive association degree between any two entities among at least two entities includes: Semantic feature vectors of textual description information for each entity are obtained based on natural language processing models; semantic similarity between any two entities is determined based on the semantic feature vectors of each entity. Based on the attributes and / or connections of the corresponding nodes of each entity in the knowledge graph data, determine the topological association degree between any two entities; Based on the temporal attributes of entities, determine the temporal density between any two entities; The semantic similarity of any two entities is weighted by a first weighting coefficient, the topological relevance of any two entities is weighted by a second weighting coefficient, and the temporal tightness of any two entities is weighted by a third weighting coefficient. The weighted semantic similarity, topological relevance, and temporal tightness are then summed to obtain the comprehensive relevance. The first weighting coefficient, the second weighting coefficient, and the third weighting coefficient are dynamically adjusted coefficients.

3. The method according to claim 2, characterized in that, The determination of the topological association degree between any two entities based on the attributes and / or connection relationships of the corresponding nodes in the knowledge graph data includes: Obtain the embedding representation matrix, adjacency matrix, and topology weight matrix of each entity's corresponding node in the knowledge graph data. The topology weight matrix is ​​a matrix composed of network topology feature parameters. Based on the topology-aware graph convolutional network model, the embedding representation matrix, adjacency matrix and topology weight matrix of each entity's corresponding node are fused to update the embedding representation matrix of each entity's corresponding node. The updated embedding representation matrix contains semantic information and topology information. The topological association degree between any two entities is determined based on the updated embedding representation matrix of the corresponding nodes of any two entities.

4. The method according to any one of claims 1-3, characterized in that, The generation of corresponding knowledge topics based on each of the target entity groups includes: Based on the textual description information of each entity in the target entity group, generate the title of the knowledge topic; Based on the comprehensive correlation and correlation path between entities within the target entity group, construct the correlation evidence chain for the knowledge topic; If the group correlation degree and / or business risk indicators of the target entity group meet the second preset condition, a disposal suggestion is matched or generated from the preset disposal suggestion library. The title, the associated evidence chain and the disposal suggestion are integrated to generate a structured knowledge topic report and display it. The group correlation degree is the average of the comprehensive correlation degree of each entity pair in the target entity group. Each entity pair is based on any two entities in the target entity group. Otherwise, the title and the associated evidence chain are integrated to generate a structured knowledge-based report and display it.

5. The method according to claim 2, characterized in that, The determination of the temporal density between any two entities based on their temporal attributes includes: Based on the adaptive sliding window mechanism, the size of the current processing window is dynamically determined according to the event inflow density; The tightness of the time difference between the occurrence times of two entities is determined using a Gaussian kernel function based on the size of the current processing window; this is known as the time tightness.

6. The method according to any one of claims 1-3, characterized in that, Before determining the overall correlation between any two of the at least two entities, the method further includes: filtering the at least two entities, wherein the filtering step includes: Real-time acquisition of data streams of entities identified from the message queue of the message middleware; The data stream is grouped according to the entity identifier of the entity, and data with the same entity identifier are divided into the same processing group; The length of the current processing window is dynamically determined based on network load metrics, and the starting boundary time point of the processing window is determined based on the event timestamp. A sliding time window instance is generated by combining the length of the current processing window. Within each sliding time window instance, dynamic association aggregation processing is performed on entities in each grouped processing group. The dynamic association aggregation processing is based on a time decay factor, a cross-domain adjustment factor, and a feature filtering threshold, and outputs an aggregation score that reflects the degree of dynamic association between entities. Delete entity pairs consisting of two entities whose aggregation score is less than the first preset threshold.

7. The method according to any one of claims 1-3, characterized in that, Knowledge graph data is hierarchically stored in an in-memory graph database, a distributed graph database cluster, and a columnar distributed database based on the latest date of data access. The in-memory graph database is used to store hot data, the distributed graph database cluster is used to store warm data, and the columnar distributed database is used to store cold data.

8. The method according to claim 7, characterized in that, The distributed graph database cluster is configured to create data partitions based on business domains, wherein the communication business domain corresponds to the main partition container; The distributed graph database cluster is configured to use a geographic region partitioning strategy to divide the wireless network device nodes within the main partition container into sub-partitions, assigning wireless network device nodes in the same geographic region to the same sub-partition; and / or, The distributed graph database cluster is configured to use a device type hashing strategy to divide the core network device nodes in the main partition container into sub-partitions, and to evenly distribute core network device nodes of the same device type across multiple sub-partitions.

9. The method according to any one of claims 1-3, characterized in that, The method further includes: Based on the key-value pair structure, the association results of two entities forming an entity pair after association degree analysis are cached in the association result cache container. The association results include the identifier of the entity pair, the semantic similarity between the entity pairs, the topological association degree, the temporal tightness or the comprehensive association degree, wherein the key includes the identifier of the entity pair, and the value includes the semantic similarity, topological association degree, temporal tightness or the comprehensive association degree between the entity pairs. The initial memory allocation strategy of the association result cache container adopts a dynamic expansion mechanism. When the number of stored entity pairs reaches a second preset threshold, the least recently used algorithm is used to evict the cached entry with the lowest access frequency in the association result cache container, where the cached entry includes key-value pairs; and / or, Set an expiration time for cache entries and trigger an asynchronous cleanup thread to delete expired cache entries the next time they are accessed; When the queried entity pair does not exist or the cache entry to which it belongs has expired, the overall relevance of the queried entity pair is automatically determined based on the registered callback function. The corresponding key-value pair is determined based on the overall relevance of the queried entity pair and written into the association result cache container, and the access timestamp is updated.

10. A knowledge topic generation system, characterized in that, It includes a data acquisition module, an entity recognition module, a multimodal association analysis module, and a knowledge topic generation module; The data acquisition module is configured to acquire multi-source heterogeneous data; The entity recognition module is configured to process the multi-source heterogeneous data to obtain at least two structured entities; The multimodal association analysis module is configured to determine the comprehensive association degree between any two entities among at least two entities. The comprehensive association degree is obtained by weighted fusion of semantic similarity, topological association degree and temporal tightness between the two entities. The semantic similarity is the similarity between entities in terms of semantics, the topological association degree is the association degree between entities in terms of topological structure, and the temporal tightness is the tightness between entities in terms of time. The multimodal association analysis module is further configured to cluster the at least two entities based on the comprehensive association degree between any two entities to obtain at least one entity group. The multimodal association analysis module is further configured to filter out each target entity group from the at least one entity group according to a first preset condition. The knowledge topic generation module is configured to generate corresponding knowledge topics based on each target entity group.

Images