A system and method for intelligent generation and dynamic adjustment of UGC tags based on multi-source hotspot linkage

Abstract

The application discloses a kind of UGC label intelligent generation and dynamic adjustment system and method based on multi-source hot spot linkage, it is related to internet information processing technical field, including first acquisition module, obtains the hot spot original data from multiple content sources and has multiple media forms, using entity extraction model to process hot spot original data, obtain the first core element set containing person and place, construction module, extract the co-occurrence frequency of each element in different content sources and different media forms in first core element set, if co-occurrence frequency is greater than preset threshold, then establish the cross-source connection edge between each element, obtain initial global hot spot linkage network;The UGC label intelligent generation and dynamic adjustment system and method based on multi-source hot spot linkage, realize the dynamic adjustment of label network, reduce the interference of invalid vocabulary to subsequent content understanding and information distribution, improve the real-time performance, effectiveness and content recommendation accuracy of label system.

Description

Technical Field

[0001] This invention relates to the field of Internet information processing technology, specifically to a system and method for intelligent generation and dynamic adjustment of UGC tags based on multi-source hotspot linkage. Background Technology

[0002] In internet content distribution and content understanding scenarios, tagging management is typically used to structurally identify user-generated content, enabling platforms to perform content retrieval, interest matching, trending topic aggregation, and precise recommendations based on tags. With the rapid growth of multimedia content such as text, images, short videos, live stream clips, and comment text, the same trending event often spreads simultaneously across multiple content sources and media formats, forming a complex dissemination relationship interwoven with elements such as people, locations, event subjects, and keywords. Therefore, accurately identifying the correlation between user-generated content and trending events has become a crucial issue in the field of UGC tagging.

[0003] Existing tag generation methods typically rely on keyword matching, manual classification trees, or semantic recognition results of single-media content. For example, text and image content mainly relies on titles, body text, and topic keywords to generate tags, while video content mainly relies on titles, subtitles, cover information, or manual annotation results. These methods are applicable to handling content from a single source or a single medium, but they fail to adequately utilize the relationships between different content sources and struggle to identify the interconnectedness of the same trending event across different platforms and media formats. In practice, the core elements of the same trending event may be scattered across different content sources and media formats. For example, a trending film or television drama may first gain traction on a video platform, while related interviews, character discussions, filming location analyses, and user comments may be distributed across text and image communities, social media platforms, or comment sections. Due to the different data structures and expression methods of different media, existing systems often struggle to uniformly extract core elements such as people and locations, and also find it difficult to establish cross-source associations based on the co-occurrence relationships between these elements, resulting in a fragmented overall context of the trending event. Furthermore, user-generated content (UGC) text often features concise expression, vague references, and a lack of context. For example, a user might only mention a character's nickname, a short name of a location, or a fragment of an event, without directly specifying the complete name of the trending topic. Existing tag generation methods, if based solely on the target text itself, are prone to incomplete entity recognition, unclear referents, and insufficient semantic completion, making it difficult to accurately determine the distance or strength of association between the UGC text and existing trending events. Due to the lack of a global network reflecting the interconnectedness of multiple trending topics, existing systems typically only provide broad category tags when generating tags. For instance, when a user's content is highly relevant to a specific person, location, or trending topic, the system might only generate generalized tags like "entertainment," "movies," or "travel," failing to generate trending topic-related tags specific to the particular trending topic. Conversely, when a user's content has a weak connection to existing trending topics, the system might incorrectly categorize it under a popular tag, affecting the accuracy of tag generation.

[0004] Furthermore, trending events exhibit significant timeliness and dynamic evolution. As user engagement changes, some trending nodes gradually become obsolete, and some topics also change due to new content dissemination. Existing methods relying on static tag libraries or fixed classification systems often struggle to promptly identify obsolete keywords with declining engagement and to update tag networks in a timely manner. This results in expired tags being retained in the tag library for extended periods, impacting the understanding and distribution of subsequent content. Summary of the Invention

[0005] The purpose of this invention is to provide a system and method for intelligent generation and dynamic adjustment of UGC tags based on multi-source hotspot linkage, thereby solving the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a UGC tag intelligent generation and dynamic adjustment system based on multi-source hotspot linkage, comprising: The first acquisition module acquires raw data of hot topics from multiple content sources and in multiple media formats, processes the raw data of hot topics, and obtains a set of first core elements including people and places. The module extracts the co-occurrence frequency of each element in the first core element set in different content sources and media formats. If the co-occurrence frequency is greater than the preset threshold, cross-source connection edges are established between each element to obtain the initial global hotspot linkage network. The second acquisition module acquires the target UGC text data published by the user, performs hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network, and uses an entity extraction model to resolve cross-source referencing to obtain the second core element set. The distance determination module processes the second core element set and the initial global hotspot linkage network to obtain the spatial distance of the second core element set in the initial global hotspot linkage network. If the spatial distance is less than a preset distance threshold, the first generation module extracts the target node corresponding to the second core element set in the initial global hotspot linkage network to obtain the hotspot association tag. The second generation module processes the second core element set to obtain generalized tags if the spatial distance is greater than or equal to a preset distance threshold. The judgment module obtains the access popularity value of the target node in the initial global hotspot linkage network. If the access popularity value is lower than the preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word. The update module removes the target nodes corresponding to invalid words from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network.

[0007] Preferably, the first acquisition module acquires raw hotspot data from multiple content sources and in multiple media formats, processes the raw hotspot data, and obtains a first core element set including people and locations, including: Obtain raw data on trending topics across multiple media formats; Multimodal feature vectors are obtained by extracting features from the original hotspot data; The fused feature representation is obtained by aligning features based on multimodal feature vectors; An entity extraction model is used to process the fusion feature representation to determine the initial feature sequence containing candidate entities; If the candidate entities in the initial element sequence meet the threshold, the initial element sequence is disambiguated to obtain the first core element set containing people and places.

[0008] Preferably, the construction module extracts the co-occurrence frequency of each element in the first core element set across different content sources and media formats. If the co-occurrence frequency is greater than a preset threshold, cross-source connection edges are established between the elements to obtain an initial global hotspot linkage network, including: Obtain the distribution characteristics of the first core element set and extract text features to obtain the semantic vector of the element; Calculate similarity values ​​based on element semantic vectors to obtain element co-occurrence frequencies; If the co-occurrence frequency of elements is greater than a preset threshold, then the association weights of each element are extracted to establish cross-source connection edges. By mapping network nodes through cross-source connection edges and constructing the topology, an initial global hotspot interconnected network is obtained.

[0009] Preferably, the second acquisition module acquires the target UGC text data published by the user, performs hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network, and uses an entity extraction model to resolve cross-source referencing, resulting in a second core element set including: Obtain the target UGC text data, and perform semantic completion on the target UGC text data through the initial global hotspot linkage network to obtain the completed text data; An entity extraction model is used to annotate the completed text data to obtain initial entities; Determine if the initial entity has a reference conflict. If the initial entity has a reference conflict, perform cross-source reference resolution on the initial entity to obtain the resolved entity. The second core element set is obtained by decomposing the entity clusters.

[0010] Preferably, the distance determination module processes the second core element set and the initial global hotspot linkage network to obtain the spatial distance of the second core element set in the initial global hotspot linkage network, including: Extract the node feature vectors and adjacency matrices of the initial global hotspot linkage network, and process the node feature vectors and adjacency matrices through a graph convolutional network to obtain the hidden layer representation; The hidden layer representation is dimensionality reduced to obtain a feature aggregation matrix. The weights of the second core element set are extracted and mapped based on the feature aggregation matrix to obtain the spatial distribution coordinates. Calculate the spatial distribution coordinates to obtain the spatial distance of the second core element set in the initial global hotspot linkage network.

[0011] Preferably, if the spatial distance is less than a preset distance threshold, the first generation module extracts the target node corresponding to the second core element set in the initial global hotspot linkage network to obtain hotspot association tags, including: Determine if the spatial distance is less than a preset distance threshold; If the spatial distance is less than the preset distance threshold, then extract the corresponding target node in the initial global hotspot linkage network; Determine the hotspot radiation range based on the target node; Obtain the associated feature vectors within the hotspot's radiation range, perform classification processing based on the associated feature vectors, and obtain hotspot association labels.

[0012] Preferably, if the spatial distance is greater than or equal to a preset distance threshold, the second generation module processes the second core element set to obtain generalized tags including: If the spatial distance is greater than or equal to the preset distance threshold, the similarity matrix is ​​calculated based on the semantic vector of the second core element set. Hierarchical clustering algorithm is used to process the similarity matrix to obtain clusters; Based on the clusters, a node hierarchy is constructed. The second core element set is then processed through the node hierarchy to obtain generalized labels.

[0013] Preferably, the determination module obtains the access popularity value of the target node in the initial global hotspot linkage network. If the access popularity value is lower than a preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word, including: Obtain the historical access trajectory of the target node in the initial global hotspot linkage network, and extract the access frequency sequence based on the historical access trajectory; The access frequency sequence is subjected to heat decay calculation to obtain the access heat value of the target node; If the access popularity value is lower than the preset popularity threshold, then extract the hotspot association tags corresponding to the target node; The hotspot association tags are parsed to obtain contextual distribution features, and the hotspot association tags corresponding to the target node are determined to be invalid words based on the contextual distribution features.

[0014] Preferably, the updating module removes the target nodes corresponding to invalid words from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network, including: Based on the invalid words, node mapping is performed in the initial global hotspot linkage network to determine the target node corresponding to the invalid word and the linkage edge weight associated with the target node; Determine whether the weight of the linked edge is less than the preset edge weight threshold; If the weight of the linked edge is less than the preset edge weight threshold, the linked relationship corresponding to the target node is cut off, and a transition network containing an isolated subgraph is obtained. Connectivity branch detection is performed on the transition network, and the target nodes corresponding to the failed words are removed from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network.

[0015] As can be seen from the above technical solution, the present invention has the following beneficial effects: This system and method for intelligent generation and dynamic adjustment of UGC tags based on multi-source hotspot linkage acquires raw hotspot data from multiple content sources and in various media formats. It extracts and analyzes the co-occurrence relationships of core elements such as people and locations, unifying and associating hotspot elements scattered across different platforms and media formats to form an initial global hotspot linkage network. This improves upon existing tag generation methods that rely solely on a single medium or surface keywords, making it difficult to identify cross-source hotspot relationships. By performing hotspot semantic completion and cross-source referencing resolution on user-published target UGC text data based on the initial global hotspot linkage network, it alleviates the problem of incomplete entity recognition caused by brief UGC text expression, ambiguous referencing, and lack of context, improving the accuracy of the second core element set. Furthermore, it utilizes graph convolutional networks... The system determines the spatial distance of the second core element set within the initial global hotspot linkage network and generates hotspot-related tags or generalized tags based on this spatial distance. This allows for more accurate hotspot-related tags when user content is strongly associated with specific hotspot nodes, and appropriate generalized tags when the association is weak, thereby improving the accuracy and adaptability of UGC tag generation. Simultaneously, by acquiring the access popularity value of target nodes and determining that the corresponding hotspot-related tags are invalid when the access popularity value falls below a preset popularity threshold, the system removes the target nodes corresponding to invalid tags from the initial global hotspot linkage network. This timely cleanup of expired tags with declining popularity enables dynamic adjustment of the tag network, reducing the interference of invalid tags on subsequent content understanding and information distribution, and improving the real-time performance, effectiveness, and accuracy of content recommendation within the tag system. Attached Figure Description

[0016] Figure 1 This is a flowchart of the present invention; Figure 2 This is a timing diagram of the present invention on the terminal and server; Figure 3 This is a system connection diagram of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1: like Figure 1 , Figure 2 and Figure 3As shown, this invention provides a technical solution: a UGC tag intelligent generation and dynamic adjustment system based on multi-source hotspot linkage, comprising: The first acquisition module acquires raw data of hot topics from multiple content sources and in multiple media formats. It then processes the raw data of hot topics using an entity extraction model to obtain a first set of core elements containing people and locations. The module extracts the co-occurrence frequency of each element in the first core element set in different content sources and media formats. If the co-occurrence frequency is greater than the preset threshold, cross-source connection edges are established between each element to obtain the initial global hotspot linkage network. The second acquisition module acquires the target UGC text data published by the user, performs hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network, and uses an entity extraction model to resolve cross-source referencing to obtain the second core element set. The distance determination module processes the second core element set and the initial global hotspot linkage network through a graph convolutional network to obtain the spatial distance of the second core element set in the initial global hotspot linkage network. If the spatial distance is less than a preset distance threshold, the first generation module extracts the target node corresponding to the second core element set in the initial global hotspot linkage network to obtain the hotspot association tag. The second generation module, if the spatial distance is greater than or equal to the preset distance threshold, uses a hierarchical clustering algorithm to process the second core element set to obtain a generalized label. The determination module obtains the access popularity value of the target node in the initial global hotspot linkage network. If the access popularity value is lower than the preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word. The update module removes the target nodes corresponding to invalid words from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network.

[0019] In this embodiment, the system is deployed in the tag generation service of a server, cloud computing platform, or content platform, and the modules transmit data through data interfaces, message queues, or internal calls.

[0020] Specifically, the first acquisition module is used to acquire raw data of trending topics from at least two content sources, including at least two of the following: news platforms, short video platforms, social media platforms, search hot lists, forum posts, comment sections, and text and image information platforms; the raw data of trending topics includes at least two media formats, including text, images, audio, video, and mixed text and image data; the first acquisition module performs unified processing on data of different media formats, including performing speech recognition on audio data, extracting titles, subtitles, and keyframe text from video data, performing text recognition on image data, and performing cleaning, word segmentation, deduplication, and timestamp annotation on plain text data, thereby forming standardized raw data of trending topics.

[0021] Furthermore, entity extraction models are used to identify core elements such as people and locations from standardized hotspot raw data.

[0022] Specifically, the entity extraction model employs a sequence labeling model based on a pre-trained language model, a named entity recognition model, a BiLSTM-CRF model, or an entity recognition model combining a rule dictionary and a deep learning model. The entity extraction model identifies personal names, place names, organizational locations, event locations, and aliases, abbreviations, and online expressions related to people or places in the original hot data, and normalizes different expressions of the same entity, thereby obtaining a first core element set containing people and places.

[0023] Each element in the first core element set carries attributes such as entity name, entity type, source identifier, media form identifier, time of appearance, frequency of appearance, and entity identification confidence level.

[0024] In this embodiment, the construction module constructs an initial global hotspot linkage network based on the co-occurrence of each element in the first core element set in different content sources and different media formats.

[0025] Specifically, the construction module uses core elements such as people and locations as network nodes and cross-source co-occurrence relationships between elements as connecting edges. When two elements co-occur in the same hot event, the same text paragraph, the same video title, the same video subtitle, the same trending list item, the same comment aggregation window, or a preset time window, the construction module records the number of times they co-occur. The construction module calculates the co-occurrence frequency based on the number of sources, the number of media formats, the number of co-occurrences, and the time decay coefficient.

[0026] When the co-occurrence frequency is greater than a preset threshold, the construction module determines that the corresponding elements meet the cross-source connection conditions, establishes cross-source connection edges between the corresponding nodes, and writes the co-occurrence frequency, source coverage, media coverage, and time freshness into the weight parameters of the connection edge to obtain the initial global hotspot linkage network.

[0027] Furthermore, the second acquisition module is used to acquire target UGC text data published by users, including user posts, comments, short video titles, image and text descriptions, bullet comments, Q&A content, or social dynamics; the second acquisition module performs hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network.

[0028] Specifically, when the target UGC text data contains abbreviations of people, places, or pronouns, the second acquisition module completes the associated people, places, or hotspot backgrounds based on the nodes, edge weights, and access popularity values ​​corresponding to the expression in the initial global hotspot linkage network.

[0029] Meanwhile, the second acquisition module uses an entity extraction model to resolve cross-source pronouns, matching pronouns, abbreviations, aliases, misspellings, or networked names in the target UGC text data with the standardized nodes in the initial global hotspot linkage network to obtain the second core element set.

[0030] Furthermore, the distance determination module processes the second core element set and the initial global hotspot linkage network through a graph convolutional network.

[0031] Specifically, the graph convolutional network takes the node attributes, edge weights, entity types, source coverage, media coverage, time decay value, and access popularity value in the initial global hotspot linkage network as input features, and generates node embedding vectors by aggregating neighbor node information.

[0032] Specifically, for each element in the second core element set, the distance determination module calculates the spatial distance between its embedding vector and the embedding vector of the candidate node in the initial global hotspot linkage network. The spatial distance is represented by Euclidean distance, cosine distance, Manhattan distance, or the shortest path distance in the graph structure. When the spatial distance is less than a preset distance threshold, it indicates that the core element in the target UGC text data and the target node in the initial global hotspot linkage network meet the hotspot association condition. When the spatial distance is greater than or equal to the preset distance threshold, it indicates that the core element in the target UGC text data and the target node in the initial global hotspot linkage network do not meet the hotspot association condition.

[0033] As a preferred embodiment, the first generation module generates hotspot association tags based on spatial distance.

[0034] Specifically, when the spatial distance between the second core element set and the initial global hotspot linkage network is less than a preset distance threshold, the first generation module extracts the target node corresponding to the second core element set from the initial global hotspot linkage network, and uses the standardized name, event-related words, person-location combination words or hot topic words of the target node as hotspot association tags, so that the UGC content published by the user can establish a corresponding relationship with the current hot events.

[0035] Furthermore, when the spatial distance is greater than or equal to a preset distance threshold, the second generation module uses a hierarchical clustering algorithm to process the second core element set.

[0036] Specifically, the hierarchical clustering algorithm aggregates the second core element set layer by layer based on entity semantic vectors, text context vectors, entity types, and co-occurrence relationships between entities, and selects upper-level categories that meet the clustering distance conditions in the clustering tree to generate generalized tags; the generalized tags include location tags, people tags, event category tags, topic domain tags, or content theme tags.

[0037] Furthermore, the determination module is used to dynamically determine the validity of hotspot association tags; the determination module obtains the access popularity value of the target node in the initial global hotspot linkage network, and the access popularity value is calculated based on at least one indicator of the target node in search, click, exposure, comment, forward, favorite, citation, tag hit, cross-source occurrence frequency and recent time decay.

[0038] Specifically, the determination module performs a weighted sum of the number of visits, interactions, and references from at least two content sources, and combines this with a time decay coefficient to obtain the access popularity value of the target node; when the access popularity value is less than a preset popularity threshold, the determination module determines that the hot topic association tag corresponding to the target node is an invalid word.

[0039] Furthermore, the update module maintains the initial global hotspot linkage network based on the judgment result of the judgment module.

[0040] Specifically, when a hot topic-related tag is determined to be an invalid word, the update module removes the target node corresponding to the invalid word from the initial global hot topic linkage network, and simultaneously deletes or adjusts the cross-source connection edges connected to the target node to obtain the updated target global hot topic linkage network. For other nodes that have a connection relationship with the removed target node, the update module recalculates their edge weights, access popularity values ​​and node embedding vectors to ensure that the target global hot topic linkage network is consistent with the current hot topic status.

[0041] Through the above processing, the system completes the acquisition of original hotspot data, the construction of an initial global hotspot linkage network, the semantic completion of target UGC text data, the generation of hotspot association tags, the determination of invalid words, and the updating of the target global hotspot linkage network.

[0042] Example 2: The first acquisition module acquires original hotspot data from multiple content sources and in various media formats. It then processes this original data using an entity extraction model to obtain a first core element set containing people and locations. This process includes: acquiring original hotspot data in various media formats; extracting features from the original data to obtain multimodal feature vectors; performing feature alignment based on the multimodal feature vectors to obtain a fused feature representation; processing the fused feature representation using an entity extraction model to determine an initial element sequence containing candidate entities; and if the candidate entities in the initial element sequence meet a threshold, performing disambiguation processing on the initial element sequence to obtain the first core element set containing people and locations.

[0043] In the above implementation, the first acquisition module receives hotspot raw data from more than 2 content sources. When each piece of hotspot raw data enters the first acquisition module, it is written with source identifier, collection time, media format identifier and content number.

[0044] Specifically, the content sources include two or more of the following: news platforms, short video platforms, social media platforms, search hot lists, forum pages, and comment pages; the original hot data includes two or more of the following: text data, image data, audio data, video data, and mixed text and image data; after receiving the text data, the first acquisition module sequentially performs encoding unification, invalid character deletion, duplicate text merging, word segmentation processing, and word position annotation, and retains the start and end positions of each word segmentation unit in the original text.

[0045] Furthermore, after receiving the image data, the first acquisition module identifies the text region in the image and extracts the text content, text region coordinates, and image source information; after receiving the audio data, the first acquisition module converts the audio content into text segments arranged in chronological order and records the audio time point corresponding to each text segment; after receiving the video data, the first acquisition module extracts the video title, subtitles, cover text, and keyframe text, and binds the keyframe text to the video playback time point.

[0046] After the above processing, the original data of the hotspot is formed into standardized data records, which include content text, source identifier, media form identifier, collection time, location index and time index.

[0047] In the above embodiments, the first acquisition module extracts features from the standardized data records to generate multimodal feature vectors.

[0048] Specifically, for text content, the first acquisition module inputs the word segmentation results into a text encoding model to obtain text feature vectors representing the semantics of the text; for image content, the first acquisition module inputs the image into an image encoding model to obtain image feature vectors representing the image content, and inputs the image text recognition results into the text encoding model to obtain image text feature vectors representing the semantics of the text in the image; for audio content, the first acquisition module inputs the audio into a speech encoding model to obtain audio feature vectors representing the acoustic information of the audio, and inputs the text segments obtained from speech recognition into the text encoding model to obtain audio text feature vectors representing the semantics of the audio text; for video content, the first acquisition module extracts features from the video title, subtitles, cover text, keyframe images, and keyframe text respectively, and then merges the features within the same time period according to the video playback time to obtain the video feature vector.

[0049] Furthermore, after the various feature vectors are generated, the first acquisition module performs normalization processing on each feature vector. Specifically, it first calculates the square root of the sum of squares of the values ​​of each dimension in the feature vector, and then divides the value of each dimension in the feature vector by the square root, so that the feature vectors corresponding to different media forms enter the same numerical scale.

[0050] In this embodiment, the first acquisition module performs feature alignment based on the multimodal feature vector; the feature alignment is based on the acquisition time, source identifier, entity context in the content text, media form identifier, and content theme identifier.

[0051] For any two feature vectors, the first acquisition module first calculates the semantic similarity between the two feature vectors. The semantic similarity is calculated as follows: the values ​​of the same dimension in the two feature vectors are multiplied and summed to obtain the vector multiplication result; then the length of each of the two feature vectors is calculated; finally, the vector multiplication result is divided by the product of the lengths of the two vectors to obtain the semantic similarity.

[0052] The first acquisition module then determines whether the time difference between the acquisition of the data corresponding to the two feature vectors is less than or equal to the preset time window, and determines whether the semantic similarity is greater than or equal to the alignment similarity threshold. When both of the above conditions are met, the first acquisition module writes the two feature vectors into the same alignment group.

[0053] In this embodiment, the preset time window is determined according to the collection cycle of the content source; the first acquisition module first counts the time interval between two consecutive collections of each content source, then calculates the arithmetic mean of the collection intervals of each content source, and takes twice the arithmetic mean as the preset time window.

[0054] Specifically, the alignment similarity threshold is determined through labeled hotspot samples. The first acquisition module first obtains feature vector pairs belonging to the same hotspot content from the labeled hotspot samples and calculates the semantic similarity of these feature vector pairs. Then, it calculates the arithmetic mean of the semantic similarity and the standard deviation of the deviation from the arithmetic mean. Subsequently, the initial threshold is obtained by subtracting the standard deviation from the arithmetic mean. If the initial threshold is less than 0.60, the alignment similarity threshold is set to 0.60. If the initial threshold is greater than 0.90, the alignment similarity threshold is set to 0.90. If the initial threshold is greater than or equal to 0.60 and less than or equal to 0.90, the initial threshold is set as the alignment similarity threshold.

[0055] Furthermore, the first acquisition module fuses the feature vectors within the same alignment group to obtain a fused feature representation.

[0056] Specifically, during fusion, the first acquisition module first reads the entity recognition accuracy of text, images, audio, and video in the labeled samples; the weight of each media form is obtained by dividing the entity recognition accuracy of that media form by the sum of the entity recognition accuracy of all media forms currently participating in the fusion; the first acquisition module then multiplies the feature vector of each media form by the corresponding weight, and adds the weighted feature vectors dimension by dimension to obtain the fused feature representation; if a certain media form does not exist in the current alignment group, then that media form does not participate in the weight allocation; the first acquisition module only recalculates the weights based on the currently existing media forms, so that the sum of the weights participating in the fusion is equal to 1.

[0057] The resulting fusion feature representation simultaneously includes content semantics, visual content, audio content, video temporal information, source information, and entity context information.

[0058] Furthermore, the first acquisition module inputs the fused feature representation into the entity extraction model, and the entity extraction model performs sequence labeling processing on the fused feature representation, outputting an initial feature sequence containing candidate entities.

[0059] Specifically, the entity extraction model outputs the entity category and corresponding label probability for each character unit or word segmentation unit, and determines the candidate entity boundary based on the consistency of entity categories of consecutive units. For a candidate entity composed of several consecutive character units or word segmentation units, the entity extraction model adds up the label probabilities corresponding to these units and then divides them by the number of units contained in the candidate entity to obtain the entity confidence of the candidate entity. The initial element sequence is arranged according to the order of appearance of the candidate entities in the standardized data records. Each candidate entity records the entity name, entity type, start position, end position, source identifier, media form identifier, and entity confidence.

[0060] In this embodiment, the first acquisition module performs threshold judgment on the candidate entities in the initial element sequence.

[0061] Specifically, the thresholds include entity confidence thresholds and entity occurrence thresholds. The entity confidence threshold is determined through the validation set of the entity extraction model. The first acquisition module sequentially uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, and 0.95 as candidate thresholds, and statistically analyzes the identification results of person and location entities under each candidate threshold. For each candidate threshold, the first acquisition module calculates precision, recall, and comprehensive evaluation value. Precision is the number of correctly identified person and location entities divided by the total number of entities identified as person or location entities; recall is the number of correctly identified person and location entities divided by the total number of person and location entities actually existing in the validation set; and comprehensive evaluation value is twice the product of precision and recall divided by the sum of precision and recall.

[0062] The first acquisition module determines the candidate threshold corresponding to the maximum comprehensive evaluation value as the entity confidence threshold. When the comprehensive evaluation values ​​corresponding to two or more candidate thresholds are the same, the candidate threshold with the largest value is taken as the entity confidence threshold. The entity occurrence threshold is determined according to the number of content sources. Specifically, when the value of half the number of content sources rounded up is less than 2, the entity occurrence threshold is 2. When the value of half the number of content sources rounded up is greater than or equal to 2, the entity occurrence threshold is the rounded value.

[0063] Furthermore, when the entity confidence of a candidate entity is greater than or equal to the entity confidence threshold, and the candidate entity appears in a content source that reaches the entity occurrence threshold, or appears in two or more media formats, and the entity type of the candidate entity is a person type or a location type, the first acquisition module retains the candidate entity.

[0064] Specifically, when the entity confidence of a candidate entity is less than the entity confidence threshold, or the candidate entity does not reach the entity appearance threshold and does not appear in more than two media forms, or the entity type of the candidate entity does not belong to the person type and the location type, the first acquisition module deletes the candidate entity from the initial element sequence.

[0065] In this embodiment, the first acquisition module performs disambiguation processing on the retained initial element sequence; for candidate entities with the same name, the first acquisition module extracts several word segmentation units before and after the candidate entity as a context window.

[0066] Specifically, the length of the context window is determined based on the training samples. Specifically, the number of effective descriptive words used to distinguish the identity of a person before and after the person entity in the training samples is counted, the arithmetic mean of the number of these effective descriptive words is calculated, and the arithmetic mean is rounded up. The rounded result is used as the length of the context window. The first acquisition module converts the occupation words, organization words, associated location words, and event words in the context window into context vectors, and calculates the similarity with the standard context vectors of candidate entities in the entity database. The similarity is calculated by multiplying the values ​​of the same dimension in the two context vectors respectively and summing them, and then dividing by the product of the lengths of the two context vectors.

[0067] When the similarity is greater than or equal to the disambiguation similarity threshold, the first acquisition module maps the candidate entity of the person to the corresponding normalized entity. The disambiguation similarity threshold is determined by training samples. The first acquisition module calculates the arithmetic mean of the context similarity of the same entity and the arithmetic mean of the context similarity of different entities, and then adds the two arithmetic means and divides by 2 to obtain the disambiguation similarity threshold.

[0068] As a preferred embodiment, for candidate entities with the same name, the first acquisition module performs disambiguation based on administrative division terms, geographical level terms, related event terms, and source region information in the context of the candidate entity; the first acquisition module calculates the administrative division matching result, geographical level matching result, and context event similarity respectively.

[0069] Specifically, when the administrative divisions are consistent, the administrative division matching result is 1; when the administrative divisions are inconsistent, the administrative division matching result is 0; when the geographical levels are consistent, the geographical level matching result is 1; when the geographical levels are inconsistent, the geographical level matching result is 0; the context event similarity is obtained in the manner of context vector similarity; the first acquisition module multiplies the administrative division matching result, the geographical level matching result, and the context event similarity by their respective weights and then adds them together to obtain the location matching score.

[0070] Specifically, the corresponding weights are determined through labeled location samples. Specifically, several weight combinations are set in the validation set, and the sum of the three weights in each weight combination is 1. The first acquisition module calculates the comprehensive evaluation value of location entity recognition corresponding to each weight combination, and uses the weight combination corresponding to the maximum comprehensive evaluation value as the weight of administrative division matching result, geographical level matching result and context event similarity. When the location matching score is greater than or equal to the location disambiguation threshold, the first acquisition module maps the candidate location entity to the corresponding normalized location entity. The location disambiguation threshold is determined using the validation set, and the determination method is consistent with the determination method of the entity confidence threshold.

[0071] Furthermore, after completing the disambiguation process, the first acquisition module merges the aliases, abbreviations, translations, and misspellings of the same person or place in different content sources and media formats into the same standardized entity name, and counts the number of occurrences, sources, and media formats of the standardized entity; the first acquisition module writes the standardized entity name, entity type, source identifier, media format identifier, occurrence time, occurrence frequency, entity confidence, and disambiguation result into the first core element set.

[0072] At this point, the first acquisition module has completed the acquisition of hotspot raw data, extraction of multimodal feature vectors, feature alignment, generation of fused feature representations, determination of initial element sequences, judgment of candidate entity thresholds, and disambiguation processing, and outputs the first core element set containing people and locations.

[0073] Example 3: The construction module extracts the co-occurrence frequency of each element in the first core element set across different content sources and media formats. If the co-occurrence frequency is greater than a preset threshold, cross-source connection edges are established between the elements to obtain an initial global hotspot linkage network. This includes obtaining the distribution characteristics of the first core element set and extracting text features to obtain the element semantic vectors; calculating similarity values ​​based on the element semantic vectors to obtain the element co-occurrence frequency; if the element co-occurrence frequency is greater than a preset threshold, extracting the association weights of each element to establish cross-source connection edges; mapping network nodes through cross-source connection edges and constructing a topology to obtain the initial global hotspot linkage network.

[0074] In this embodiment, after receiving the first set of core elements, the construction module first performs standardization verification on the character elements and location elements in the set.

[0075] Specifically, for abbreviations, translations, aliases, and misspellings of the same entity appearing in different content sources, the construction module uses the disambiguation results output by the first acquisition module to merge them into the same normalized entity name; for elements that do not carry source identifiers, media form identifiers, or appearance times, the construction module does not include them in the calculation scope of cross-source connection edges; each person element or location element after verification is used as a candidate node element for subsequent network construction.

[0076] Furthermore, when the construction module obtains the distribution characteristics of the first core element set, it reads the source identifier, media form identifier, occurrence time, occurrence location, context fragment, and occurrence frequency of each element.

[0077] Specifically, the source distribution feature is obtained by deduplicating the content source identifiers in which the element has appeared; the media distribution feature is obtained by deduplicating the text, image, audio, video, and mixed text and image media identifiers in which the element has appeared; the time distribution feature is obtained by recording the first appearance time, the most recent appearance time, and the time interval between two adjacent appearances of the element; and the location distribution feature is obtained by recording the position of the element in the title, body text, subtitles, comments, image text recognition results, or audio transcription segments.

[0078] The aforementioned distribution characteristics, along with the element identifiers, are written into the intermediate cache table of the building module for subsequent similarity and co-occurrence frequency calculations.

[0079] Furthermore, when extracting text features, the building module extracts contextual content centered on the location where each element appears.

[0080] For person elements, the construction module extracts adjacent identity descriptions, occupations, organizations, actions, related events, and related locations. For location elements, the construction module extracts adjacent administrative divisions, geographical levels, scenes, related events, and related people. The context truncation length is determined based on the training samples. Specifically, the number of effective descriptions required for person and location elements to be distinguished in the training samples is counted, the arithmetic mean of this number is calculated, and the arithmetic mean is rounded up. The rounded result is used as the context truncation length. The construction module inputs the truncated text into the text encoding model to obtain a text vector representing the semantics of the element.

[0081] In this embodiment, when the construction module generates the semantic vector of the elements, it merges the text vector with the distribution features.

[0082] Specifically, the merging process includes: arranging the numerical values ​​of each dimension, source distribution, media distribution, time distribution, and location distribution in the text vector into a numerical sequence of uniform length according to a preset field order; filling missing media form or location fields with 0; truncating the first part of the context encoding result that exceeds the uniform length; and then, the construction module performs normalization processing on the numerical sequence, specifically by first calculating the square root of the sum of squares of the numerical values ​​of each dimension in the numerical sequence, and then dividing the numerical value of each dimension by the square root, thereby obtaining the element semantic vector.

[0083] Through this process, the semantic vectors of elements generated from different sources and in different media formats have consistent dimensions and numerical scales.

[0084] In this embodiment, the construction module calculates the similarity value between any two elements based on the element semantic vector.

[0085] Specifically, during the calculation, the construction module multiplies the values ​​of the same dimension in the semantic vectors of the first and second elements one by one, and sums all the multiplication results to obtain the vector multiplication result; then, it calculates the vector length of the semantic vectors of the first and second elements respectively; finally, it divides the vector multiplication result by the product of the two vector lengths to obtain the similarity value between the two elements; the closer the similarity value is to 1, the closer the two elements are in terms of semantic content, source distribution, media distribution, and time distribution; the closer the similarity value is to 0, the less semantic basis there is between the two elements that can be used to establish a cross-source connection edge.

[0086] Furthermore, after calculating the similarity value, the module further counts the co-occurrence records between the two elements.

[0087] Specifically, the criteria for determining co-occurrence records include the same content record, the same title, the same text paragraph, the same video subtitle segment, the same image text recognition result, the same audio transcription segment, and the same time window; when two elements appear simultaneously in any of the above co-occurrence scenarios, the construction module records it as one co-occurrence; when the same two elements appear repeatedly in the same content record, only one co-occurrence is recorded; the number of cross-source co-occurrences is obtained by counting the number of content sources in which the two elements have appeared together, and the number of cross-media co-occurrences is obtained by counting the number of media forms in which the two elements have appeared together.

[0088] The time window is determined based on the 7-day consecutive collection interval of each content source. The construction module calculates the arithmetic mean of all collection intervals and determines 24 times the arithmetic mean as the time window.

[0089] In this embodiment, the co-occurrence frequency of elements is determined by the similarity value, cross-source co-occurrence results, cross-media co-occurrence results, and time decay results.

[0090] Specifically, the construction module first divides the number of content sources where the two elements co-occur by the total number of content sources participating in the statistics to obtain the cross-source co-occurrence ratio; then it divides the number of media forms where the two elements co-occur by the total number of media forms participating in the statistics to obtain the cross-media co-occurrence ratio; the time decay result is determined according to the most recent co-occurrence time: when the interval between the most recent co-occurrence time and the current calculation time is less than or equal to the time window, the time decay result is 1; when the interval is greater than the time window, the time window is divided by the interval, and the result is used as the time decay result.

[0091] The construction module multiplies the similarity value, cross-source co-occurrence ratio, cross-media co-occurrence ratio, and time decay result by their respective weights and then adds them together to obtain the element co-occurrence frequency.

[0092] The weights for the above items were determined using labeled hotspot samples.

[0093] The construction module pre-sets 10 weight combinations, each of which includes similarity value weight, cross-source co-occurrence ratio weight, cross-media co-occurrence ratio weight, and time decay result weight, and the sum of the four weights is equal to 1. The construction module uses the 10 weight combinations to calculate the co-occurrence frequency of elements in the labeled hotspot samples and compares it with the manually labeled element association relationship.

[0094] For each weight combination, the construction module counts the number of element combinations that are judged as related and are actually related, the number of element combinations that are judged as related but are not actually related, and the number of element combinations that are actually related but not judged as related. Then, it calculates precision, recall, and overall evaluation value. Precision is the number of element combinations judged as related and are actually related divided by the total number of element combinations judged as related. Recall is the number of element combinations judged as related and are actually related divided by the total number of element combinations manually labeled as actually related. Overall evaluation value is twice the product of precision and recall divided by the sum of precision and recall. The construction module determines the weight combination corresponding to the maximum overall evaluation value as the element co-occurrence frequency calculation weight. When two or more weight combinations correspond to the same overall evaluation value, the weight combination with the largest similarity value is selected.

[0095] Specifically, the preset thresholds are also determined based on the labeled hotspot samples. The construction module uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90 and 0.95 as candidate thresholds in turn to determine whether cross-source connection edges should be established for the combination of elements in the sample.

[0096] For each candidate threshold, the construction module calculates the precision, recall, and overall evaluation value as described above. The construction module determines the candidate threshold corresponding to the maximum overall evaluation value as the preset threshold. When two or more candidate thresholds correspond to the same overall evaluation value, the candidate threshold with the largest value is taken as the preset threshold.

[0097] Furthermore, when the co-occurrence frequency between any two elements is greater than a preset threshold, the construction module extracts the association weights of the two elements and establishes cross-source connection edges accordingly.

[0098] Specifically, the process of obtaining the association weight is as follows: the construction module reads the co-occurrence frequency, similarity value, number of co-occurring content sources, number of co-occurring media forms, and time decay results of the two elements; then, it weights the above values ​​according to the determined weights; finally, it adds up the weighted results to obtain the association weight of the cross-source connection edge; when the cross-source connection edge is established, the construction module writes the edge identifier, start node identifier, end node identifier, association weight, co-occurring content source, co-occurring media form, first co-occurrence time, most recent co-occurrence time, and number of co-occurrences.

[0099] If the co-occurrence frequency between any two elements is less than or equal to a preset threshold, the construction module will not establish a cross-source connection edge.

[0100] In this embodiment, when the construction module maps network nodes through cross-source connection edges, it maps each normalized person entity and normalized location entity in the first core element set to one network node.

[0101] Each network node is configured with a node identifier, normalized entity name, entity type, number of sources, number of media forms, number of occurrences, first occurrence time, and most recent occurrence time. For established cross-source connections, the construction module attaches the connection to the corresponding two network nodes based on the starting node identifier and the ending node identifier, and writes the association weight to the edge weight field of the connection.

[0102] Furthermore, when constructing the topology, the construction module generates an adjacency table based on network nodes and cross-source connection edges. The adjacency table uses network nodes as indexes and records the identifiers of adjacent nodes directly connected to that network node, the identifiers of connection edges, edge weights, co-occurrence sources, and co-occurrence media forms.

[0103] Subsequently, the construction module generates an initial global hotspot linkage network based on the adjacency table, so that the person nodes, location nodes, and cross-source connection edges form a graph structure that can be retrieved, traversed, and called for graph computation. The nodes in this graph structure are used to represent the person elements and location elements in the hotspot, and the cross-source connection edges are used to represent the association relationship between different elements after verification by content source, media form, and time window.

[0104] Furthermore, after the initial global hotspot linkage network is generated, the construction module performs a consistency check.

[0105] Specifically, the construction module checks whether both endpoint nodes corresponding to each cross-source connection edge exist in the network node set; if either endpoint node does not exist, the cross-source connection edge is deleted; for cases where there are more than two cross-source connection edges between the same two nodes, the construction module retains the connection edge with the largest association weight value and merges the co-occurrence count, co-occurrence source, and co-occurrence medium form of the remaining connection edges into the retained connection edge; after verification, the construction module outputs the initial global hotspot linkage network.

[0106] Example 4: The second acquisition module acquires the target UGC text data published by the user, performs hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network, and uses an entity extraction model to perform cross-source referencing resolution to obtain the second core element set. This includes acquiring the target UGC text data, performing semantic completion on the target UGC text data through the initial global hotspot linkage network to obtain completed text data; using an entity extraction model to annotate the completed text data to obtain initial entities; determining whether there are referencing conflicts in the initial entities; if there are referencing conflicts in the initial entities, performing cross-source referencing resolution on the initial entities to obtain resolved entities; and clustering and partitioning the resolved entities to obtain the second core element set.

[0107] In this embodiment, after the second acquisition module receives the target UGC text data published by the user, it writes the content number, publication time, publication entry point, user's business scenario, and text source identifier into the text data.

[0108] Specifically, the second acquisition module first preprocesses the target UGC text data, including unifying encoding, merging whitespace characters, deleting invalid links, escaping emojis, compressing repeated punctuation, and word segmentation, and records the start and end positions of each word segment in the original text; for abbreviations of people, places, pronouns, aliases, topic words, online names, and misspelled words in the text, the second acquisition module marks them as fragments to be completed, and saves the context window, publication time, and text position of the fragments to be completed.

[0109] Furthermore, when the second acquisition module performs semantic completion on the target UGC text data through the initial global hotspot linkage network, it first matches the fragment to be completed with the node name, node alias and node context words in the initial global hotspot linkage network.

[0110] When the fragment to be completed is completely identical to the node name or node alias, the second acquisition module will use the corresponding node as a candidate completion node; when the fragment to be completed is not completely identical, the second acquisition module will calculate the semantic similarity between the context of the fragment to be completed and the standard context of the network node.

[0111] Specifically, the semantic similarity calculation process is as follows: First, the context of the segment to be completed is converted into a context vector, and the standard context of the network node is converted into a node vector; then, the values ​​of the same dimension in the two vectors are multiplied item by item and summed to obtain the vector multiplication result; then, the square root of the sum of the squares of the values ​​of each dimension of the two vectors is calculated to obtain the length of the two vectors; finally, the semantic similarity is obtained by dividing the vector multiplication result by the product of the lengths of the two vectors.

[0112] Specifically, the semantic completion judgment value is determined based on the labeled user-generated content samples; the second acquisition module uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90 and 0.95 as candidate judgment values ​​in turn to determine whether there is a completion relationship between the fragment to be completed in the sample and the network node, and compares it with the manual annotation results.

[0113] For each candidate judgment value, the second acquisition module counts the number of correctly completed items, the number of incorrectly completed items, and the number of items that should have been completed but were not. The accuracy rate is the number of correctly completed items divided by the total number of completed items, the recall rate is the number of correctly completed items divided by the total number of items manually labeled as needing completion, and the comprehensive evaluation value is twice the product of accuracy and recall divided by the sum of accuracy and recall. The second acquisition module uses the candidate judgment value corresponding to the maximum comprehensive evaluation value as the semantic completion judgment value. When there are more than two candidate judgment values ​​corresponding to the same comprehensive evaluation value, the candidate judgment value with the largest value is taken.

[0114] Furthermore, when the fragment to be completed completes a perfect match, or when the semantic similarity between the fragment to be completed and the candidate completion node is greater than or equal to the semantic completion judgment value, the second acquisition module reads the normalized entity name, entity type, adjacent nodes, connection edge weights, most recent co-occurrence time, and co-occurrence source of the candidate completion node.

[0115] Specifically, the second acquisition module writes the normalized entity name and its associated nodes into the extended fields of the target UGC text data without changing the original text content input by the user. The completed text data consists of the original text, the completed entity, the completed source node, the completed connection edge, the completed time, and the completed confidence. The completed confidence is obtained by weighting the semantic similarity of the candidate completed nodes, the weight of the connection edge, and the time consistency result. The time consistency result is determined by the interval between the publication time of the target UGC text data and the most recent co-occurrence time of the candidate completed nodes. The interval is 1 when it is less than or equal to the effective time window, and 0 when it is greater than the effective time window.

[0116] The effective time window is determined based on the update cycle of the most recent co-occurrence time of nodes in the initial global hotspot linkage network, and is taken as 24 times the arithmetic mean of the node update interval within 7 consecutive days.

[0117] Furthermore, when the second acquisition module uses the entity extraction model to annotate the completed text data, it inputs both the original text field and the extended field into the entity extraction model. The entity extraction model outputs the entity category and corresponding label probability for each character unit or word segmentation unit, and determines the entity boundary based on the consistency of entity categories in continuous units.

[0118] For an entity composed of continuous units, the second acquisition module adds up the label probabilities corresponding to each unit and divides them by the number of units contained in the entity to obtain the entity confidence score. The entity confidence score is determined by the validation set of the entity extraction model, and the determination process is the same as that for the semantic completion score. If the entity confidence score is greater than or equal to the entity confidence score and the entity category belongs to the person category or the location category, the second acquisition module writes the entity into the initial entity table. Entities that do not meet the above conditions are not written into the initial entity table.

[0119] Furthermore, when the second acquisition module determines whether there is a referential conflict in the initial entity, it checks for name conflicts, alias conflicts, pronoun conflicts, and cross-source conflicts respectively.

[0120] Among them, name conflict refers to the same initial entity name corresponding to more than two normalized nodes in the initial global hotspot linkage network; alias conflict refers to the same alias corresponding to more than two normalized entities; pronoun conflict refers to the same pronoun having more than two entities that can be pointed to within the context window of the target UGC text data; cross-source conflict refers to the same initial entity having inconsistent normalized entity names in two or more content sources.

[0121] If any of the above situations exist, the second acquisition module determines that the initial entity has a referential conflict and generates a candidate referential list for the initial entity; the candidate referential list includes candidate node name, entity type, adjacent nodes, connection edge weight, source identifier, media form identifier and most recent co-occurrence time.

[0122] In this embodiment, for each candidate node in the candidate referential list, the second acquisition module calculates the referential matching score; the referential matching score is jointly determined by context similarity, connection edge weight, source consistency result, time consistency result, and entity type consistency result.

[0123] Specifically, the context similarity is obtained according to the aforementioned vector similarity calculation process; the edge weights are read from the edge weights between candidate nodes and identified entities in the initial global hotspot linkage network; when the sources are consistent, the result is 1, and when they are inconsistent, it is 0; when the interval between the publication time and the most recent co-occurrence time of the candidate node is less than or equal to the effective time window, the result is 1, otherwise it is 0; when the entity types are consistent, the result is 1, otherwise it is 0; the second acquisition module multiplies the above results by their corresponding weights and then adds them together to obtain the referencing matching score.

[0124] The weights of each item are determined by the labeled and represented samples. The second acquisition module sets 10 weight combinations in the verification set, with the sum of the weights in each group equal to 1, and selects the weight combination corresponding to the maximum comprehensive evaluation value.

[0125] Furthermore, in the case of a conflict of reference in the initial entity, the second acquisition module performs cross-source reference resolution. The second acquisition module first deletes candidate nodes whose entity type is inconsistent with the initial entity type, then sorts the remaining candidate nodes in descending order of reference matching score, and selects the first ranked candidate node as the entity to be resolved.

[0126] If two or more candidate nodes have the same referential matching score, the candidate node with the smallest time interval between the publication of the target UGC text data is selected; if the publication time interval is still the same, the candidate node with the largest connection edge weight value is selected; the entity to be resolved includes the normalized entity name, entity type, original expression, candidate node identifier, resolution basis and resolution confidence; for the initial entity without referential conflict, the second acquisition module directly uses the normalization result of the initial entity as the entity to be resolved.

[0127] In this embodiment, when the second acquisition module performs clustering on the resolved entities, it first generates clustering features for each resolved entity; the clustering features include normalized entity name vector, context vector, entity type, source identifier, adjacent hotspot nodes, connection edge weights, and publication time.

[0128] Specifically, the second acquisition module calculates the inter-entity distance between two resolved entities. The calculation process is as follows: first, the semantic similarity of the two normalized entity name vectors is calculated, and the semantic similarity is subtracted from 1 to obtain the name distance; then, the semantic similarity of the two context vectors is calculated, and the semantic similarity is subtracted from 1 to obtain the context distance; when the entity types are the same, the type distance is 0, and when the entity types are different, the type distance is 1; the difference in connection edge weights is the absolute value of the difference in the weight values ​​of the corresponding connection edges of the two resolved entities; the second acquisition module multiplies the name distance, context distance, type distance, and difference in connection edge weights by their respective weights and then adds them together to obtain the inter-entity distance.

[0129] In this embodiment, the clustering judgment value is determined by the labeled entity clustering samples. The second acquisition module sequentially uses 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45 and 0.50 as candidate judgment values ​​to cluster the labeled entity clustering samples respectively, and compares them with the manually labeled clustering results.

[0130] The second acquisition module determines the clustering judgment value based on the accuracy, recall and comprehensive evaluation value corresponding to each candidate judgment value. When two or more candidate judgment values ​​correspond to the same comprehensive evaluation value, the candidate judgment value with the smallest value is selected to reduce the situation where different entities are merged into the same cluster.

[0131] Furthermore, during clustering, the second acquisition module first treats each resolved entity as an initial cluster, and then calculates the distance between any two clusters.

[0132] Specifically, the distance between two clusters is the arithmetic mean of the pairwise distances of all entities within those two clusters; when the distance between two clusters is less than or equal to the clustering decision value, the second acquisition module merges the two clusters; when the distance between all clusters is greater than the clustering decision value, the clustering process ends; the second acquisition module selects the entity with the highest decryption confidence value from each cluster as the representative entity, and uses the remaining entities in the same cluster as synonyms, abbreviations, or cross-source representations of the representative entity.

[0133] Finally, the second acquisition module writes the representative entity, entity type, original expression set, source identifier, media form identifier, source completion, dissolution confidence, and cluster number into the second core element set.

[0134] Example 5: The distance determination module processes the second core element set and the initial global hotspot linkage network through a graph convolutional network to obtain the spatial distance of the second core element set in the initial global hotspot linkage network. This includes extracting the node feature vectors and adjacency matrices of the initial global hotspot linkage network, processing the node feature vectors and adjacency matrices through a graph convolutional network to obtain the hidden layer representation, reducing the dimensionality of the hidden layer representation to obtain the feature aggregation matrix, extracting the weights of the second core element set based on the feature aggregation matrix and mapping them to obtain the spatial distribution coordinates, and calculating the spatial distribution coordinates to obtain the spatial distance of the second core element set in the initial global hotspot linkage network.

[0135] In this embodiment, after receiving the initial global hotspot linkage network and the second core element set, the distance determination module first reads the node data and connection edge data in the initial global hotspot linkage network.

[0136] Specifically, node data includes node identifier, normalized entity name, entity type, number of sources, number of media forms, frequency of occurrence, most recent occurrence time, and access popularity value; connection edge data includes starting node identifier, ending node identifier, edge weight, co-occurrence source, co-occurrence media form, and most recent co-occurrence time; the distance determination module sorts all network nodes according to node identifier and assigns a unique row and column number to each network node, so that the position in the subsequent node feature vector and adjacency matrix can correspond one-to-one with the network node.

[0137] Furthermore, when the distance determination module extracts the node feature vectors of the initial global hotspot linkage network, it converts the entity type, number of sources, number of media forms, number of occurrences, access popularity value, recent occurrence time interval, number of adjacent nodes, and connected edge weight statistics of each network node into numerical fields; the entity type is represented by a fixed field, where a person entity is written as 1 in the person field and 0 in the location field, and a location entity is written as 0 in the person field and 1 in the location field.

[0138] Specifically, the number of sources, the number of media formats, the number of occurrences, and the access popularity value are all normalized. The processing is as follows: read the maximum value of the same type of field in the initial global hotspot linkage network, and divide the value of the corresponding field of the current node by the maximum value; when the maximum value of the same type of field is 0, the normalization result of the current field is 0; the most recent occurrence time interval is obtained by subtracting the most recent occurrence time of the node from the current calculation time, and then the effective time window is divided by the larger value between the most recent occurrence time interval and the effective time window, and the result is used as the time field value.

[0139] The effective time window here is determined based on the network node update records. Specifically, it is calculated by statistically analyzing the time interval between two adjacent updates of each node within a consecutive 7 days, calculating the arithmetic mean of all time intervals, and taking 24 times the arithmetic mean as the effective time window.

[0140] Specifically, for the connected edge weight statistics, the distance determination module reads all cross-source connected edges directly connected to the current node, adds up the edge weights of each connected edge, and then divides by the number of directly connected edges to obtain the connected edge weight statistics of the node; when the current node has no directly connected edges, the connected edge weight statistics are 0; the distance determination module arranges the above numerical fields according to the preset field order to form the node feature vector of the node.

[0141] In this process, the number and order of fields in the feature vector of each node are kept consistent to ensure that the graph convolutional network can perform the same dimensional calculations on different nodes.

[0142] Furthermore, when the distance determination module extracts the adjacency matrix, it uses the number of network nodes as the number of rows and columns of the matrix, with each row and each column corresponding to one network node in the node index table.

[0143] If there is a cross-source connection edge between two network nodes, the distance determination module writes the edge weight of the connection edge into the corresponding row and column positions; if there is no cross-source connection edge between two network nodes, 0 is written into the corresponding position; in order to enable the node's own information to participate in the graph convolution operation, the distance determination module writes 1 into the diagonal position corresponding to each node.

[0144] Subsequently, the distance determination module normalizes the adjacency matrix: first, it calculates the sum of the weights of each node's connections with other nodes and itself to obtain the degree value of that node; then, it divides the connection weight between any two nodes by the product of the square roots of the degree values ​​of those two nodes to obtain the normalized connection weights; the normalized adjacency matrix is ​​used to limit the contribution value of each neighboring node in the graph convolutional network to the representation of the current node.

[0145] Furthermore, when the distance determination module processes node feature vectors and adjacency matrices through the graph convolutional network, it first performs neighborhood feature aggregation on each node in the first layer of graph convolution.

[0146] The specific processing is as follows: Based on the normalized adjacency matrix, the current node itself and its directly connected neighboring nodes are read; the node feature vectors of these nodes are multiplied by the corresponding normalized connection weights; then the weighted feature vectors are added item by item according to each dimension to obtain the neighborhood aggregation result of the current node; subsequently, the graph convolutional network inputs the neighborhood aggregation result into the linear transformation unit, which multiplies the input value of each dimension with the trained parameters and sums them to obtain the linear transformation result; then, activation processing is performed on the linear transformation result, where values ​​less than 0 are set to 0, and values ​​greater than or equal to 0 are kept in their original values, forming the first layer node representation.

[0147] As a preferred embodiment, in the second layer of graph convolution, the distance determination module takes the previous layer node representation as input and repeatedly performs neighborhood aggregation, linear transformation and activation processing to obtain the hidden layer representation; the training parameters of the graph convolutional network are obtained from historical hotspot linked network samples.

[0148] Specifically, the training samples include historical node features, historical connection edges, and manually labeled associated and unassociated node pairs. During training, the graph convolutional network generates a representation for each node and calculates the distance between associated node pairs and the distance between unassociated node pairs. When the distance between associated node pairs is not less than the set training distance, or the distance between unassociated node pairs is not greater than the set training distance, the network parameters are updated. The set training distance is determined through the validation set. The distance determination module uses 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, and 0.50 as candidate distances in sequence, and calculates the node association judgment accuracy, recall, and comprehensive evaluation value in the validation set respectively. The candidate distance corresponding to the maximum comprehensive evaluation value is taken as the set training distance. When there are more than two candidate distances corresponding to the same comprehensive evaluation value, the candidate distance with the smallest value is taken.

[0149] Furthermore, after obtaining the hidden layer representation, the distance determination module performs dimensionality reduction on the hidden layer representation to obtain the feature aggregation matrix; the dimensionality reduction is performed using principal component projection. The distance determination module first calculates the arithmetic mean of each dimension in the hidden layer representation, and then subtracts the arithmetic mean of that dimension from the value of each node in that dimension to obtain the centered hidden layer representation.

[0150] Subsequently, the distance determination module calculates the covariance between each dimension by multiplying the centered values ​​of the corresponding nodes in the two dimensions one by one and summing them, then dividing by the number of nodes minus 1. The distance determination module performs feature decomposition on the covariance results to obtain the feature vectors representing the direction of data change and their corresponding feature values, and selects the first two feature vectors in descending order of feature values ​​for projection to obtain a two-dimensional feature aggregation matrix.

[0151] If the subsequent configuration is for three-dimensional distance calculation, the first three feature vectors are selected for projection to obtain a three-dimensional feature aggregation matrix.

[0152] In this embodiment, the distance determination module extracts the weights of the second core element set based on the feature aggregation matrix and performs mapping; for each element in the second core element set, the distance determination module first performs consistency matching between the normalized entity name and the node name in the initial global hotspot linkage network; when the consistency matching is successful, the network node is determined as the mapping node.

[0153] When consistency matching fails, the distance determination module converts the context content of the element into an element context vector, converts the standard context of the candidate network node into a node context vector, and calculates the semantic similarity between the two context vectors. The semantic similarity calculation process is as follows: multiply the values ​​of the same dimension in the two vectors one by one and sum them to obtain the vector multiplication result; calculate the square root of the sum of the squares of the values ​​of each dimension of the two vectors to obtain the length of the two vectors; and then divide the vector multiplication result by the product of the lengths of the two vectors to obtain the semantic similarity.

[0154] In this embodiment, the semantic similarity judgment value is determined by the labeled mapping samples; the distance determination module uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90 and 0.95 as candidate judgment values ​​in turn to determine the mapping relationship between the elements in the sample and the network nodes, and compares it with the manual annotation results.

[0155] For each candidate judgment value, the distance determination module counts the number of correctly mapped, the number of incorrectly mapped, and the number of elements that should have been mapped but were not. Accuracy is calculated as the number of correctly mapped elements divided by the total number of mapped elements; recall is calculated as the number of correctly mapped elements divided by the total number of manually labeled elements that should have been mapped; and the comprehensive evaluation value is twice the product of accuracy and recall divided by the sum of accuracy and recall. The candidate judgment value corresponding to the maximum comprehensive evaluation value is used as the semantic similarity judgment value. When two or more candidate judgment values ​​correspond to the same comprehensive evaluation value, the candidate judgment value with the largest value is taken. When the semantic similarity between an element and a candidate network node is greater than or equal to the semantic similarity judgment value, that candidate network node is used as the mapping node.

[0156] In this embodiment, the weight of each element in the second core element set is obtained based on the element confidence, occurrence frequency, completion confidence, and resolution confidence.

[0157] The distance determination module first divides the occurrence count of each element by the maximum occurrence count in the second core element set to obtain the occurrence count normalization result; when the maximum occurrence count is 0, the occurrence count normalization result is 0.

[0158] Subsequently, the distance determination module multiplies the element confidence, the normalized occurrence result, the completion confidence, and the resolution confidence by their respective weights, and then sums the multiplication results to obtain the mapping weight of the element. The corresponding weights are determined through the matching samples of the labeled samples. The distance determination module sets 10 weight combinations in the validation set, with the sum of the weights in each group being 1. It calculates the accuracy, recall, and comprehensive evaluation value of the label matching for each group, and uses the weight combination corresponding to the maximum comprehensive evaluation value as the weight combination used for calculating the mapping weight.

[0159] If a certain element does not have a complete confidence level or a dissipated confidence level, the corresponding item is set to 0, and the weights of the remaining participating items are redistributed so that the sum of the weights participating in the calculation is 1.

[0160] Furthermore, when the distance determination module obtains the spatial distribution coordinates, for elements directly mapped to network nodes, it reads the coordinates of the network node in the feature aggregation matrix as the spatial distribution coordinates of the element.

[0161] Specifically, for elements mapped to two or more candidate network nodes through semantic similarity, the distance determination module multiplies the coordinates of each candidate network node by the corresponding semantic similarity, then adds the weighted coordinates item by item according to the dimension, and divides the sum of the sums by the sum of the semantic similarities of each candidate network node to obtain the spatial distribution coordinates of the element; for the entire set of the second core elements, the distance determination module multiplies the spatial distribution coordinates of each element by the corresponding mapping weight, then adds them item by item according to the dimension, and divides the sum of the sums by the sum of the mapping weights of each element to obtain the set spatial distribution coordinates of the second core element set.

[0162] Furthermore, when the distance determination module calculates the spatial distance of the second core element set in the initial global hotspot linkage network, it first determines the target reference coordinates.

[0163] Specifically, the target reference coordinates are generated from the network node coordinates mapped to the second core element set. The generation method is as follows: the coordinates of each mapped node in the feature aggregation matrix are multiplied by the mapping weight of the corresponding element, and then added item by item according to the dimension. The sum of the sums is then divided by the sum of the mapping weights involved in the calculation. Subsequently, the distance determination module calculates the spatial distance between the spatial distribution coordinates of the set and the target reference coordinates. The calculation process is as follows: the difference between the two coordinates in each dimension is obtained, the difference in each dimension is squared and then added together. The square root of the sum is then taken. The resulting value is the spatial distance of the second core element set in the initial global hotspot linkage network. This spatial distance is output to the first generation module and the second generation module for subsequent judgment to generate hotspot association labels or generalized labels.

[0164] Example 6: The first generation module, if the spatial distance is less than a preset distance threshold, extracts the target node corresponding to the second core element set in the initial global hotspot linkage network to obtain hotspot association tags, including determining whether the spatial distance is less than the preset distance threshold; if the spatial distance is less than the preset distance threshold, extracts the corresponding target node in the initial global hotspot linkage network; determines the hotspot radiation range based on the target node; obtains the association feature vector within the hotspot radiation range; performs classification processing based on the association feature vector to obtain hotspot association tags.

[0165] In this embodiment, the first generation module receives the spatial distance output by the distance determination module, and simultaneously reads the second core element set, the network node identifier mapped to the second core element set, the spatial distribution coordinates of the set, and the target reference coordinates.

[0166] Specifically, the first generation module first determines whether the spatial distance is less than a preset distance threshold. The preset distance threshold is determined by the labeled user-generated content samples. Specifically, the first generation module reads the samples manually labeled as hotspot related and the samples manually labeled as non-hotspot related, and obtains the spatial distance of each sample in the initial global hotspot linkage network. Then, with an interval of 0.05, the module selects candidate distance thresholds from 0.10 to 1.00 in sequence, and determines whether each sample enters the hotspot association label generation process.

[0167] For each candidate distance threshold, the first generation module counts the number of hotspot-related samples that correctly entered the process, the number of non-hotspot-related samples that incorrectly entered the process, and the number of samples that were actually hotspot-related but did not enter the process. Based on this, it calculates the accuracy, recall, and overall evaluation value. The accuracy is the number of hotspot-related samples that correctly entered the process divided by the total number of samples that entered the process. The recall is the number of hotspot-related samples that correctly entered the process divided by the total number of samples manually labeled as hotspot-related. The overall evaluation value is twice the product of accuracy and recall divided by the sum of accuracy and recall.

[0168] The first generation module determines the candidate distance threshold corresponding to the maximum comprehensive evaluation value as the preset distance threshold; when there are more than two candidate distance thresholds corresponding to the same comprehensive evaluation value, the candidate distance threshold with the smallest value is selected.

[0169] Furthermore, when the spatial distance is less than a preset distance threshold, the first generation module enters the target node extraction process.

[0170] Specifically, the first generation module first reads the representative entities in the second core element set and performs node retrieval in the initial global hotspot linkage network using the normalized entity name of the representative entity. When the normalized entity name matches the network node name, the network node is written into the target node candidate table. When no matching node is found for the normalized entity name, the first generation module reads the context content of the representative entity and the standard context content of the network node, and converts them into context vectors respectively. The semantic similarity between context vectors is obtained as follows: the values ​​of the same dimension in the two vectors are multiplied item by item and then summed to obtain the vector multiplication result; the square root of the sum of the squares of the values ​​of each dimension in the two vectors is calculated to obtain the lengths of the two vectors; then the semantic similarity is obtained by dividing the vector multiplication result by the product of the lengths of the two vectors.

[0171] When the semantic similarity is greater than or equal to the node matching judgment value, the corresponding network node is written into the target node candidate table.

[0172] In this embodiment, the node matching judgment value is determined by the labeled node mapping sample.

[0173] Specifically, the first generation module sequentially uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, and 0.95 as candidate judgment values ​​to match representative entities and network nodes in the sample, and compares them with the manually labeled results. The first generation module calculates the precision, recall, and comprehensive evaluation value corresponding to each candidate judgment value, and determines the candidate judgment value corresponding to the maximum comprehensive evaluation value as the node matching judgment value. When there are more than two candidate judgment values ​​corresponding to the same comprehensive evaluation value, the candidate judgment value with the largest value is selected.

[0174] As a preferred embodiment, the first generation module filters the candidate nodes in the target node candidate table to determine the target node. During the filtering process, the first generation module reads the mapping weight between the candidate node and the representative entity in the second core element set, the edge weight of the corresponding connecting edge of the candidate node, the access popularity value of the candidate node, and the most recent co-occurrence time of the candidate node, and calculates the node association score.

[0175] Specifically, the node association score is calculated as follows: the mapping weight, edge weight, access popularity normalization result, and time consistency result are multiplied by their respective weights, and then the multiplication results are summed. The access popularity normalization result is obtained by dividing the access popularity value of the candidate node by the maximum value of the access popularity value in the candidate table of the target node. When the maximum value of the access popularity value is 0, the access popularity normalization result is 0. The time consistency result is determined according to the interval between the publication time of the target UGC text data and the most recent co-occurrence time of the candidate node. When this interval is less than or equal to the effective time window, the time consistency result is 1; when this interval is greater than the effective time window, the time consistency result is 0.

[0176] The effective time window is determined by the node update records of the initial global hotspot linkage network. Specifically, it involves statistically analyzing the time interval between two adjacent updates of each node within a consecutive 7 days, calculating the arithmetic mean of all time intervals, and taking 24 times the arithmetic mean as the effective time window.

[0177] In this embodiment, the weights of each item in the node association score are determined by the labeled hot spot node samples.

[0178] Specifically, the first generation module sets 10 weight combinations in the verification sample. Each weight combination includes the mapping weight corresponding coefficient, the edge weight corresponding coefficient, the access popularity normalization result corresponding coefficient, and the time consistency result corresponding coefficient, and makes the sum of all coefficients in each group equal to 1.

[0179] The first generation module calculates the node association score using 10 weight combinations and calculates the accuracy, recall, and comprehensive evaluation value based on the manually labeled target node results. The weight combination corresponding to the maximum comprehensive evaluation value is used as the weight for calculating the node association score. The target node judgment value is obtained using the same verification method as the node matching judgment value. When the node association score of a candidate node is greater than or equal to the target node judgment value, the first generation module determines the candidate node as the target node. When the node association score is less than the target node judgment value, the candidate node is not used as the target node.

[0180] Furthermore, after the target node is determined, the first generation module determines the hotspot radiation range based on the target node.

[0181] Specifically, the first generation module takes the target node as the starting point, reads the first-order adjacent nodes directly connected to the target node along the cross-source connection edges in the initial global hotspot linkage network, and calculates the radiation intensity from the target node to each first-order adjacent node. The radiation intensity is obtained by weighting the connection edge weight, the normalized result of the access heat of adjacent nodes, the consistent result of the most recent co-occurrence time, and the normalized result of the number of co-occurrence media forms.

[0182] Among them, the normalization result of the access popularity of adjacent nodes is obtained by dividing the access popularity value of adjacent nodes by the maximum value of the access popularity value in the current extension layer; the normalization result of the number of co-occurring media forms is obtained by dividing the number of co-occurring media forms corresponding to the connection edge by the total number of media forms participating in the statistics; the recent co-occurrence time consistency result is determined according to the interval between the publication time of the target UGC text data and the recent co-occurrence time of the connection edge. The interval is 1 when it is less than or equal to the effective time window, and 0 when it is greater than the effective time window.

[0183] In this embodiment, the radiation judgment value is determined by expanding the sample of the labeled hotspot. The first generation module uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90 and 0.95 as candidate judgment values ​​in turn to determine whether the adjacent nodes of the target node in the sample should be included in the radiation range of the hotspot, and compares them with the results of manual labeling.

[0184] Specifically, the first generation module determines the candidate judgment value corresponding to the maximum comprehensive evaluation value as the radiation judgment value; when there are more than two candidate judgment values ​​corresponding to the same comprehensive evaluation value, the candidate judgment value with the largest value is selected.

[0185] In this process, if the radiation intensity of a first-order adjacent node is greater than or equal to the radiation judgment value, that first-order adjacent node is included in the hotspot radiation range. For a first-order adjacent node already included in the hotspot radiation range, the first generation module continues to read its connected second-order adjacent nodes and calculates the radiation intensity of the second-order adjacent nodes in the same way. The number of expansion layers is limited to 2 layers; when the second layer expansion is completed, or when there are no adjacent nodes in the current expansion layer with a radiation intensity greater than or equal to the radiation judgment value, the hotspot radiation range is determined.

[0186] Furthermore, when the first generation module obtains the associated feature vectors within the hotspot radiation range, it extracts the node features and connecting edge features within the range respectively.

[0187] Specifically, node features include normalized entity name vector, entity type, access popularity value, number of occurrences, number of sources, number of media forms, recent occurrence time, and node association score; connection edge features include edge weight, number of co-occurrences, number of co-occurrence sources, number of co-occurrence media forms, first co-occurrence time, recent co-occurrence time, and radiation intensity.

[0188] Specifically, for access popularity value, occurrence frequency, number of sources, number of media forms, edge weight, co-occurrence frequency, and radiation intensity, the first generation module performs maximum value normalization processing, that is, divides the current field value by the maximum value of the same type of field within the hotspot radiation range; when the maximum value of the same type of field is 0, the normalization result is 0; for normalized entity name, the first generation module inputs the text encoding model to obtain the name vector.

[0189] Subsequently, the first generation module merges node features, connection edge features, and name vectors according to a preset field order to form an associated feature vector.

[0190] As a preferred embodiment, the first generation module performs classification processing based on the associated feature vector. The classification processing is completed by a pre-trained label classification model. The label classification model takes the associated feature vector as input and outputs the candidate label category and label confidence.

[0191] Specifically, the training samples for the label classification model include associated feature vectors within the historical hotspot radiation range and manually labeled hotspot association tags. During training, the label classification model uses the manually labeled hotspot association tags as the target output and adjusts the model parameters so that samples of the same category are output to the corresponding label category. During validation, the first generation module sequentially uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, and 0.95 as label confidence judgment values, and counts the number of correct labels, the number of incorrect labels, and the number of labels that should have been output but were not, under each judgment value, and calculates the accuracy, recall, and overall evaluation value.

[0192] Among them, the judgment value corresponding to the maximum comprehensive evaluation value is determined as the label confidence judgment value; when there are more than two judgment values ​​corresponding to the same comprehensive evaluation value, the judgment value with the largest value is taken.

[0193] Furthermore, when the tag confidence score of a candidate tag is greater than or equal to the tag confidence score judgment value, the first generation module determines the candidate tag as a hotspot related tag.

[0194] The content generated by the hotspot association tag is determined based on the target node, the neighboring node with the highest radiation intensity value within the hotspot's radiation range, and the tag category output by the tag classification model. When the target node is a person node and the neighboring node with the highest radiation intensity value is a location node, the first generation module generates a hotspot association tag containing the person's name and the location's name. When the target node is a location node and the neighboring node with the highest radiation intensity value is a person node, the first generation module generates a hotspot association tag containing the location's name and the person's name. When there are more than two nodes under the same event category within the hotspot's radiation range, the first generation module generates a hotspot association tag corresponding to the event category.

[0195] Example 7: The second generation module, if the spatial distance is greater than or equal to a preset distance threshold, uses a hierarchical clustering algorithm to process the second core element set to obtain generalized labels. This includes calculating a similarity matrix based on the semantic vectors of the second core element set; processing the similarity matrix using a hierarchical clustering algorithm to obtain clusters; constructing a node hierarchy based on the clusters; and processing the second core element set through the node hierarchy to obtain generalized labels.

[0196] In this embodiment, after receiving the spatial distance output by the distance determination module, the second generation module reads the normalized entity name, original expression, entity type, context content, source identifier, media form identifier, occurrence frequency, completion confidence, and resolution confidence of each element in the second core element set.

[0197] The second generation module first compares the spatial distance with a preset distance threshold. When the spatial distance is greater than or equal to the preset distance threshold, it determines that the second core element set and the existing hotspot nodes in the initial global hotspot linkage network do not meet the hotspot association tag generation conditions, and enters the generalized tag generation process.

[0198] In this embodiment, the preset distance threshold is determined by the user-generated content samples that have been annotated. The second generation module reads the samples that are manually annotated as hotspots and the samples that are manually annotated as non-hotspots, and obtains the spatial distance of each sample calculated by the distance determination module.

[0199] Subsequently, the second generation module selects candidate distance thresholds sequentially from 0.10 to 1.00 at intervals of 0.05, and determines whether a sample should enter the generalized label generation process. For each candidate distance threshold, the second generation module counts the number of non-hotspot related samples that correctly enter the generalized label generation process, the number of hotspot related samples that incorrectly enter the generalized label generation process, and the number of samples that should have entered the generalized label generation process but did not. The accuracy is the number of samples that correctly enter the generalized label generation process divided by the total number of samples that enter the generalized label generation process, the recall is the number of samples that correctly enter the generalized label generation process divided by the total number of samples manually labeled as non-hotspot related, and the comprehensive evaluation value is twice the product of accuracy and recall divided by the sum of accuracy and recall.

[0200] The second generation module determines the candidate distance threshold corresponding to the maximum comprehensive evaluation value as the preset distance threshold; when there are more than two candidate distance thresholds corresponding to the same comprehensive evaluation value, the candidate distance threshold with the largest value is selected.

[0201] Furthermore, when generating semantic vectors, the second generation module processes the elements in the second core element set according to the entity type.

[0202] For the character element, the second generation module extracts the standardized entity name, original expression, identity description words, associated location words, action words, and contextual event words, and inputs the above text into the text encoding model to obtain the character semantic vector.

[0203] For location elements, the second generation module extracts standardized entity names, original expressions, administrative division terms, geographical level terms, related person terms, and contextual event terms, and inputs the above text into the text encoding model to obtain location semantic vectors.

[0204] For elements that have completed or dissolved entities, the second generation module adds the completed entity text and dissolved entity text to the input content of the text encoding model, so that the generated semantic vector simultaneously reflects the original expression, the semantic completion result, and the reference resolution result.

[0205] For elements lacking contextual content, the second generation module generates semantic vectors using normalized entity names, entity types, and original expressions; for elements lacking normalized entity names but having decomposed entities, the second generation module uses the normalized entity names of the decomposed entities as the basis for vector generation.

[0206] Furthermore, the second generation module performs normalization processing on each semantic vector.

[0207] Specifically, during processing, the square root of the sum of squares of the numerical values ​​of each dimension in the semantic vector is first calculated, and then the numerical value of each dimension is divided by the square root to make the semantic vectors corresponding to each element have a consistent numerical scale. After normalization, the second generation module establishes a similarity matrix based on the number of elements in the second core element set. The number of rows and columns of the similarity matrix are the same as the number of elements, and each row and each column corresponds to one element.

[0208] Furthermore, when calculating the similarity matrix, the second generation module calculates the semantic similarity between any two elements one by one.

[0209] Specifically, the calculation process is as follows: multiply the values ​​of the same dimension in the two semantic vectors one by one, and add all the multiplication results to obtain the vector multiplication result; then calculate the square root of the sum of the squares of the values ​​of each dimension of the two semantic vectors to obtain the lengths of the two vectors; then divide the vector multiplication result by the product of the lengths of the two vectors to obtain the semantic similarity between the two elements; the second generation module writes the semantic similarity into the corresponding row and column positions in the similarity matrix; write 1 into the position corresponding to the same element itself; for the semantic similarity between two different elements that have been calculated, write the same value into the symmetrical position of the matrix, thereby obtaining the similarity matrix used for hierarchical clustering processing.

[0210] Furthermore, when the second generation module processes the similarity matrix using a hierarchical clustering algorithm, it first treats each element in the second core element set as an initial cluster.

[0211] For two clusters containing only one element, the second generation module subtracts the semantic similarity between the two elements from 1 to obtain the inter-cluster distance between the two clusters. For clusters containing two or more elements, the second generation module calculates the pairwise distances between all elements within the two clusters, sums all distances, and divides by the number of distances to obtain the inter-cluster distance between the two clusters. This inter-cluster distance is used to determine whether the two clusters should be merged.

[0212] In this embodiment, the cluster merging judgment value is determined by the labeled generalized sample.

[0213] Specifically, the second generation module sequentially uses 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, and 0.50 as candidate judgment values ​​to perform hierarchical clustering on the second core element set in the labeled generalized tag samples, and compares the clustering results with the manually labeled generalized categories. For each candidate judgment value, the second generation module counts the number of element pairs correctly classified into the same generalized category, the number of element pairs incorrectly classified into the same generalized category, and the number of element pairs that actually belong to the same generalized category but are not classified into the same category. The precision rate is the number of element pairs correctly classified into the same generalized category divided by the total number of element pairs classified into the same generalized category, the recall rate is the number of element pairs correctly classified into the same generalized category divided by the total number of element pairs manually labeled as belonging to the same generalized category, and the comprehensive evaluation value is twice the product of precision and recall divided by the sum of precision and recall.

[0214] The second generation module determines the candidate judgment value corresponding to the maximum comprehensive evaluation value as the cluster merging judgment value; when there are two or more candidate judgment values ​​corresponding to the same comprehensive evaluation value, the candidate judgment value with the smallest value is selected.

[0215] Furthermore, during the cluster merging process, the second generation module selects the two clusters with the smallest inter-cluster distance as the clusters to be merged each time.

[0216] Specifically, when the distance between clusters is less than or equal to the cluster merging judgment value, the second generation module merges the two clusters into a new cluster and recalculates the distance between the new cluster and the remaining clusters; when the distance between any two clusters is greater than the cluster merging judgment value, the hierarchical clustering process ends; after clustering, each cluster contains character elements, location elements, or combinations of character elements and location elements that satisfy the semantic merging conditions.

[0217] Furthermore, when constructing the node hierarchy based on the clusters, the second generation module first determines representative nodes from each cluster. The determination of representative nodes is based on entity confidence, occurrence count, completion confidence, and resolution confidence. The second generation module first divides the occurrence count of each element within a cluster by the maximum occurrence count within that cluster to obtain a normalized occurrence count result. When the maximum value is 0, the normalized occurrence count result is set to 0.

[0218] Subsequently, the second generation module multiplies the entity confidence, the normalized result of the occurrence count, the completion confidence, and the resolution confidence by their respective weights, and then adds the weighted results to obtain the representative score.

[0219] Specifically, the corresponding weights are determined by the labeled representative node samples. The second generation module sets 10 weight combinations in the validation set, with the sum of the weights in each group being 1. It calculates the accuracy, recall, and comprehensive evaluation value of the representative node selection results for each group, and uses the weight combination corresponding to the maximum comprehensive evaluation value as the weight for calculating the representative score. The element with the largest representative score value within a cluster is determined as the representative node of that cluster.

[0220] In this embodiment, the node hierarchy includes feature nodes, cluster nodes, and generalization category nodes.

[0221] The second generation module uses each original element in the second core element set as an element node and each cluster as a cluster node. It then determines the generalization category node based on the representative node, entity type, context keywords, and preset tag hierarchy table of the cluster. For clusters with people as the main element, the second generation module determines the people generalization category based on the identity description words, action words, and context event words of the representative node. For clusters with location as the main element, the second generation module determines the location generalization category based on administrative division words, geographical level words, and scene words. For clusters that contain both people and location elements, the second generation module counts the occurrence frequency of each context event word within the cluster and determines the category corresponding to the context event word with the highest occurrence frequency as the event generalization category.

[0222] Furthermore, when the second generation module generates generalized labels through the node hierarchy, it maps them in the order of feature nodes to cluster nodes and cluster nodes to generalized category nodes.

[0223] Specifically, when a representative node can match a superior category in the preset label hierarchy table, the second generation module uses that superior category as a candidate for generalized label; when a representative node does not match a superior category, the second generation module reads the context keyword with the highest frequency value within the cluster and uses the category word corresponding to that context keyword as a candidate for generalized label.

[0224] Subsequently, the second generation module calculates the label confidence of the generalized label candidates, specifically by multiplying the representative score of the representative node, the average similarity of elements within the cluster, the matching result of the higher category, and the normalized result of the frequency of occurrence of the context keyword by their respective weights and then summing them; the average similarity of elements within the cluster is obtained by summing the semantic similarities between all pairs of elements within the cluster and then dividing by the number of similarities; when the higher category matching is successful, the higher category matching result is set to 1, and when it is unsuccessful, it is set to 0; the normalized result of the frequency of occurrence of the context keyword is obtained by dividing the frequency of occurrence of the keyword by the maximum value of the frequency of occurrence of the keyword within the current cluster.

[0225] In this embodiment, the label confidence judgment value is determined by the labeled generalized label samples; the second generation module uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90 and 0.95 as candidate judgment values ​​in sequence to determine whether the generalized label candidates should be output, and compares them with the manually labeled generalized labels.

[0226] Specifically, for each candidate judgment value, the second generation module counts the number of correctly output generalized labels, the number of incorrectly output generalized labels, and the number of generalized labels that should have been output but were not, and calculates the accuracy, recall, and overall evaluation value. The second generation module determines the candidate judgment value corresponding to the one with the maximum overall evaluation value as the label confidence judgment value. When there are two or more candidate judgment values ​​corresponding to the same overall evaluation value, the candidate judgment value with the largest value is selected. When the label confidence of a candidate generalized label is greater than or equal to the label confidence judgment value, the second generation module determines the candidate generalized label as a generalized label.

[0227] Furthermore, when the second generation module outputs generalized labels, it writes the generalized label text, the corresponding cluster number, the representative node, the node hierarchical path, the label confidence, the generation time, and the second core element set number into the label result table.

[0228] Therefore, when the spatial distance is greater than or equal to the preset distance threshold, the second generation module obtains the generalized label corresponding to the second core element set through semantic vector calculation, similarity matrix generation, hierarchical clustering processing and node hierarchical mapping.

[0229] Example 8: The determination module obtains the access popularity value of the target node in the initial global hotspot linkage network. If the access popularity value is lower than a preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word. This includes obtaining the historical access trajectory of the target node in the initial global hotspot linkage network, extracting the access frequency sequence based on the historical access trajectory, performing a popularity decay calculation on the access frequency sequence to obtain the access popularity value of the target node, extracting the hotspot association tag corresponding to the target node if the access popularity value is lower than the preset popularity threshold, parsing the hotspot association tag to obtain context distribution features, and determining the hotspot association tag corresponding to the target node to be an invalid word based on the context distribution features.

[0230] In this embodiment, after receiving the hotspot association tag output by the first generation module, the determination module reads the target node identifier, tag generation time, tag source node, radiation range node, tag confidence level, and connection edge information of the target node in the initial global hotspot linkage network corresponding to the hotspot association tag.

[0231] Specifically, the determination module retrieves historical access records from the access log database based on the target node identifier. The historical access records include search records, click records, exposure records, comment records, forwarding records, collection records, tag hit records, content reference records, and cross-source occurrence records. Each access record is written with access type, access time, access source, media format, content number, target node identifier, and access count. Records lacking access time or target node identifier are not included in the access frequency sequence calculation.

[0232] Furthermore, when the determination module extracts the access frequency sequence based on the historical access trajectory, it first determines the length of the statistical window.

[0233] Specifically, the statistical window length is determined based on the update cycle of the content source containing the target node: the judgment module calculates the time interval between two consecutive data updates of the content source within 7 consecutive days, calculates the arithmetic mean of all time intervals, and uses this arithmetic mean as the statistical window length; when the target node exists in two or more content sources simultaneously, the statistical window length of each content source is calculated separately, and then the arithmetic mean of the statistical window lengths is calculated as the statistical window length of the target node. The judgment module divides the continuous statistical windows backward from the current calculation time and counts the number of visits to the target node within each statistical window, forming an access frequency sequence arranged chronologically.

[0234] In cases where the same user generates more than two records of the same access type to the same target node under the same content number and within the same statistics window, the judgment module only counts one access, so that the access frequency sequence reflects the actual triggering of different access behaviors.

[0235] Furthermore, after obtaining the access frequency sequence, the determination module performs weight conversion for different access types. The access type weights are determined using labeled popularity samples.

[0236] Specifically, the judgment module sets 10 weight combinations in the verification set. Each weight combination corresponds to search, click, exposure, comment, forward, favorite, tag hit, content reference and cross-source occurrence, and the sum of the weight values ​​of each group is equal to 1. The judgment module calculates the access popularity value of the sample node using each weight and compares it with the results of manually labeled valid and invalid tags.

[0237] For each weight combination, the judgment module counts the number of tags correctly judged as invalid words, the number of tags incorrectly judged as invalid words, and the number of tags that are actually invalid words but were not judged. The accuracy is the number of tags correctly judged as invalid words divided by the total number of tags judged as invalid words, the recall is the number of tags correctly judged as invalid words divided by the total number of tags manually labeled as invalid words, and the comprehensive evaluation value is twice the product of accuracy and recall divided by the sum of accuracy and recall.

[0238] The judgment module uses the weight combination corresponding to the maximum comprehensive evaluation value as the access type weight; when two or more weight combinations correspond to the same comprehensive evaluation value, the weight combination with the largest sum of the three weights of comment, forward and content reference is selected.

[0239] Furthermore, when the determination module performs heat decay calculation on the access frequency sequence, it first calculates the weighted access frequency for each statistical window.

[0240] The specific processing is as follows: within the same statistical window, the judgment module multiplies the number of visits for each access type by the corresponding access type weight, and then adds up the results of each multiplication to obtain the weighted access frequency for that statistical window.

[0241] Subsequently, the judgment module determines the time decay coefficient based on the distance between the statistical window and the current calculation time; the time decay coefficient of the current statistical window is set to 1; when backtracking one statistical window, the preset decay step size is subtracted from the previous window; when the calculated time decay coefficient is less than 0, the time decay coefficient is set to 0; the preset decay step size is determined through labeled popularity samples, and the judgment module uses 0.05, 0.10, 0.15, 0.20, 0.25 and 0.30 as candidate decay step sizes in sequence to calculate the accuracy, recall and comprehensive evaluation value of invalid word recognition, and determines the candidate decay step size corresponding to the maximum comprehensive evaluation value as the preset decay step size.

[0242] In this embodiment, the determination module multiplies the weighted access frequency of each statistical window by the corresponding time decay coefficient, and then adds the weighted results of all statistical windows to obtain the initial access popularity value of the target node.

[0243] In order to ensure that target nodes within the same calculation batch have a uniform comparison scale, the determination module normalizes the initial access popularity value. During normalization, the determination module reads the maximum value among all the initial access popularity values ​​of target nodes within the same calculation batch, and divides the initial access popularity value of the current target node by the maximum value to obtain the access popularity value of the target node. When the maximum value is 0, the access popularity value of the target node is 0.

[0244] As a preferred embodiment, the preset popularity threshold is determined by using labeled samples.

[0245] Specifically, the judgment module uses 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45 and 0.50 as candidate popularity thresholds in turn to determine whether the hot topic related tags in the sample belong to invalid words, and compares them with the manual annotation results.

[0246] For each candidate popularity threshold, the judgment module counts the number of tags correctly judged as invalid words, the number of tags incorrectly judged as invalid words, and the number of tags that are actually invalid words but were not judged, and calculates the accuracy, recall, and comprehensive evaluation value accordingly. The judgment module determines the candidate popularity threshold corresponding to the maximum comprehensive evaluation value as the preset popularity threshold. When two or more candidate popularity thresholds correspond to the same comprehensive evaluation value, the candidate popularity threshold with the largest value is taken.

[0247] Furthermore, when the access popularity value of the target node is less than the preset popularity threshold, the determination module extracts the hotspot association tags corresponding to the target node.

[0248] During extraction, the judgment module queries the tag result table based on the target node identifier, reads the hotspot related tags that have a direct source relationship with the target node, and simultaneously reads the tag generation time, tag confidence, tag source node, radiation range node, content number, and historical hit records. When a target node corresponds to more than two hotspot related tags, the judgment module sorts the tags from the most recent to the oldest according to their generation time, and performs context distribution feature parsing on each hotspot related tag.

[0249] Furthermore, when the judgment module parses the hotspot-related tags, it first performs word segmentation and entity recognition on the tag text, splitting the tag text into words related to people, places, events, topics, and modifiers. The judgment module then matches the splitting results with the normalized entity names, alias table, neighboring node names, and historical context word table of the target node to obtain the correspondence between the tag components and the target node.

[0250] Subsequently, the determination module reads the contextual occurrence record of the hot topic-related tag within the parsing window. The contextual occurrence record includes the content source, media form, contextual keywords, adjacent entities, publication time, and number of tag hits when the tag appears. The number of parsing windows is determined by the labeled invalid word samples. The determination module uses 3, 5, 7, and 10 as candidate window numbers in sequence, and calculates the accuracy, recall, and comprehensive evaluation value of invalid word determination respectively. The number of candidate windows corresponding to the maximum comprehensive evaluation value is determined as the number of parsing windows.

[0251] Specifically, contextual distribution features include source distribution features, medium distribution features, entity adjacency features, keyword continuation features, and time distribution features.

[0252] Among them, the source distribution feature is obtained by counting the number of content sources that appeared in the parsing window with the hotspot association tag; the media distribution feature is obtained by counting the number of media forms that appeared in the parsing window with the hotspot association tag; the entity adjacency feature is obtained by counting the number of times that the hotspot association tag and the adjacent entity of the target node co-occur; the keyword continuation feature is obtained by counting the number of overlaps between the context keywords in the parsing window and the context keywords when the tag was generated; the time distribution feature is obtained by calculating the interval between the most recent occurrence time of the hotspot association tag and the current calculation time. For the source distribution feature, media distribution feature, entity adjacency feature, and keyword continuation feature, the judgment module divides the current feature value by the maximum value of the corresponding feature in the same calculation batch to obtain the normalized result; when the maximum value of the corresponding feature is 0, the normalized result is 0; for the time distribution feature, the judgment module divides the effective time window by the larger value between the most recent occurrence time interval and the effective time window to obtain the time normalized result.

[0253] In this embodiment, the determination module calculates the label validity score based on the context distribution characteristics.

[0254] Specifically, during the calculation, the judgment module multiplies the source distribution normalization result, media distribution normalization result, entity adjacency normalization result, keyword continuation normalization result, and time normalization result by their respective weights, and then adds the results of each weighting to obtain the tag validity score. The corresponding weights are determined by the labeled invalid word samples. The judgment module sets 10 weight combinations in the validation set, with the sum of the weights in each group equal to 1. It calculates the precision, recall, and comprehensive evaluation value of the invalid word judgment results for each group, and uses the weight combination corresponding to the maximum comprehensive evaluation value as the weight for calculating the tag validity score.

[0255] In this embodiment, the label validity judgment value is determined by the sample of labeled invalid words.

[0256] The decision module uses 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, and 0.50 as candidate judgment values ​​to determine whether hotspot-related tags should be retained, and compares the results with those of manual annotation. The decision module counts the number of correctly retained tags, the number of incorrectly retained tags, and the number of tags that should have been retained but were not, under each candidate judgment value, and calculates the accuracy, recall, and comprehensive evaluation value. The decision module determines the candidate judgment value corresponding to the maximum comprehensive evaluation value as the tag validity judgment value. When two or more candidate judgment values ​​correspond to the same comprehensive evaluation value, the candidate judgment value with the largest value is taken.

[0257] Furthermore, when the access popularity value of the target node is less than the preset popularity threshold, and the tag validity score of the hotspot-related tag is less than the tag validity judgment value, the judgment module determines the hotspot-related tag corresponding to the target node as an invalid word.

[0258] Example 9: The update module removes the target nodes corresponding to invalid words from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network. This includes mapping nodes in the initial global hotspot linkage network based on the invalid words, determining the target nodes corresponding to the invalid words and the linkage edge weights associated with the target nodes; determining whether the linkage edge weights are less than a preset edge weight threshold; if the linkage edge weights are less than the preset edge weight threshold, severing the linkage relationship corresponding to the target nodes to obtain a transition network containing isolated subgraphs; performing connectivity branch detection on the transition network, removing the target nodes corresponding to invalid words from the initial global hotspot linkage network, and obtaining the updated target global hotspot linkage network.

[0259] In this embodiment, after receiving the invalid word judgment result output by the judgment module, the update module reads the invalid word text, target node identifier, hotspot association tag text, access popularity value, tag validity score, judgment time, and invalidation basis.

[0260] Specifically, the update module first performs word segmentation and entity recognition on the invalid word text, extracting words related to people, locations, events, and topics. The extracted results are then matched with the node names, aliases, historical tags, and context terms in the initial global hotspot linkage network. When an invalid word matches a network node name, the update module designates that network node as a candidate mapping node. When an invalid word matches a node alias, the update module reads the normalized node corresponding to that alias and writes it into the candidate mapping node table. When an invalid word does not match a node name or alias, the update module calculates the semantic similarity between the invalid word's context and the network node's standard context.

[0261] Specifically, the semantic similarity calculation process is as follows: the update module first converts the context of the invalid word into a context vector and converts the standard context of the network node into a node vector; then, the values ​​of the same dimension in the two vectors are multiplied item by item and summed to obtain the vector multiplication result; then, the square root of the sum of the squares of the values ​​of each dimension in the two vectors is calculated to obtain the length of the two vectors; finally, the semantic similarity is obtained by dividing the vector multiplication result by the product of the lengths of the two vectors.

[0262] The node mapping judgment value is determined through the labeled invalid word mapping samples. The update module sequentially uses 0.50, 0.55, 0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, and 0.95 as candidate judgment values ​​to determine whether there is a mapping relationship between invalid words in the samples and network nodes, and compares it with the manually labeled results. The candidate judgment value corresponding to the maximum comprehensive evaluation value is determined as the node mapping judgment value. When there are more than two candidate judgment values ​​corresponding to the same comprehensive evaluation value, the candidate judgment value with the largest value is taken. Network nodes with semantic similarity greater than or equal to the node mapping judgment value are written into the candidate mapping node table.

[0263] In this embodiment, the update module determines the target node corresponding to the invalid word based on the candidate mapping node table.

[0264] For each candidate mapping node, the update module reads the name matching result, alias matching result, semantic similarity, tag source consistency result, and target node identifier consistency result, and calculates the node mapping score. A successful name match results in a value of 1, while an unsuccessful match results in a value of 0. A successful alias match results in a value of 1, while an unsuccessful match results in a value of 0. When the source tag of the invalid word matches the historical tag of the candidate mapping node, the tag source consistency result is 1, and when they do not match, it is 0. When the target node identifier output by the judgment module matches the candidate mapping node identifier, the target node identifier consistency result is 1, and when they do not match, it is 0. The update module multiplies each of the above results by its corresponding weight and then sums them to obtain the node mapping score.

[0265] The corresponding weights are determined through labeled node mapping samples. The update module sets 10 weight combinations in the validation set, with the sum of the weights in each group equal to 1. It calculates the accuracy, recall, and comprehensive evaluation value of the node mapping results for each group, and uses the weight combination corresponding to the maximum comprehensive evaluation value as the weight for calculating the node mapping score. The update module determines the candidate mapping node with the largest node mapping score as the target node corresponding to the invalid word.

[0266] Furthermore, after the target node is determined, the update module reads all cross-source connection edges directly associated with the target node and extracts the linkage edge weight of each cross-source connection edge; the linkage edge weight is written by the construction module in the initial global hotspot linkage network, and the linkage edge weight corresponds to the association strength between the target node and its adjacent nodes.

[0267] For connection edges lacking linkage weights, the update module recalculates the linkage weights of the connection edge. The calculation process is as follows: read the co-occurrence count between the target node and its adjacent nodes, the number of co-occurring content sources, the number of co-occurring media formats, the most recent co-occurrence time, and the access popularity value of adjacent nodes; normalize the co-occurrence count, the number of content sources, the number of media formats, and the access popularity value respectively, that is, divide the current value by the maximum value of the corresponding field within the same calculation batch; convert the most recent co-occurrence time into a time consistency result according to the interval between the current calculation time and the most recent co-occurrence time, taking 1 when the interval is less than or equal to the effective time window, and taking 0 when the interval is greater than the effective time window; then multiply the above normalized result and the time consistency result by the corresponding weights and add them together to obtain the recalculated linkage weights.

[0268] The effective time window is determined based on the edge update time of the initial global hotspot linkage network. Specifically, it is calculated by statistically analyzing the time interval between two adjacent updates of each connected edge within 7 consecutive days, calculating the arithmetic mean of all time intervals, and taking 24 times the arithmetic mean as the effective time window.

[0269] In this embodiment, the preset edge weight threshold is determined by updating the labeled network samples. The update module sequentially uses 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45 and 0.50 as candidate edge weight thresholds to determine whether the connecting edges associated with the target node in the sample should be cut off, and compares them with the manually labeled network update results.

[0270] Specifically, for each candidate edge weight threshold, the update module counts the number of correctly cut edges, the number of incorrectly cut edges, and the number of edges that should have been cut but were not, and calculates the accuracy, recall, and overall evaluation value. The accuracy is the number of correctly cut edges divided by the total number of cut edges; the recall is the number of correctly cut edges divided by the total number of manually labeled edges that should have been cut; and the overall evaluation value is twice the product of accuracy and recall divided by the sum of accuracy and recall.

[0271] The update module determines the candidate edge weight threshold corresponding to the maximum comprehensive evaluation value as the preset edge weight threshold; when there are more than two candidate edge weight thresholds corresponding to the same comprehensive evaluation value, the candidate edge weight threshold with the largest value is selected.

[0272] As a preferred implementation, the update module checks one by one whether the weight of the linked edge associated with the target node is less than a preset edge weight threshold.

[0273] Specifically, when the weight of the linked edge is less than the preset edge weight threshold, the update module marks the linked edge as a linked edge to be cut off; when the weight of the linked edge is greater than or equal to the preset edge weight threshold, the update module retains the linked edge and writes it into the retained linked edge table; for the linked relationship marked as a linked edge to be cut off, the update module deletes the adjacent node record corresponding to the linked edge from the adjacency relationship table of the initial global hotspot linked network, and deletes the edge identifier, start node identifier, end node identifier and edge weight record of the linked edge from the linked edge table.

[0274] After deletion, the direct connection between the target node and some of its neighboring nodes is severed, and the initial global hotspot linkage network is transformed into a transitional network.

[0275] Furthermore, when forming the transition network, the update module synchronously checks the number of remaining connection edges of the target node.

[0276] If the number of remaining connecting edges of the target node is 0, the target node becomes an isolated node; if the target node is still connected to several nodes through retained connecting edges, but the node group is no longer connected to the main connecting region in the initial global hotspot linkage network, the node group becomes an isolated subgraph; the update module writes both the isolated node and the isolated subgraph into the branch table to be detected of the transition network, and records the corresponding number of nodes, number of connecting edges, sum of weights of remaining linkage edges, and most recent co-occurrence time.

[0277] In this embodiment, when the update module performs connectivity branch detection on the transition network, it traverses the network starting from each unvisited node.

[0278] Specifically, the traversal process is as follows: the update module first writes the starting node into the queue to be visited, and then reads the adjacent nodes of the node in the adjacency table; for adjacent nodes that have not yet been visited, they are written into the queue to be visited and marked as visited; when the queue to be visited is empty, the current traversal process ends, and a connected branch is formed by the visited nodes; the update module repeats the above process until all nodes in the transition network have been visited and marked, thereby obtaining all connected branches in the transition network.

[0279] Furthermore, after the connectivity branch detection is completed, the update module determines whether the target node corresponding to the failed word should be removed from the initial global hotspot linkage network.

[0280] The update module performs a removal process when the target node meets the following conditions: the hotspot association tag corresponding to the target node has been determined to be an invalid word, and the target node is an isolated node in the transition network; or the number of nodes in the connected branch where the target node is located is less than the branch node threshold; or the sum of the remaining linked edge weights in the connected branch where the target node is located is less than the branch edge weight threshold. The branch node threshold is determined based on historical network update samples. The update module uses 2, 3, 4 and 5 as candidate branch node thresholds in sequence, and determines the candidate value corresponding to the maximum comprehensive evaluation value as the branch node threshold. The branch edge weight threshold is determined using the same verification method as the preset edge weight threshold.

[0281] Furthermore, when removing a target node, the update module deletes the target node's node identifier, normalized entity name, entity type, access popularity value, number of sources, number of media forms, number of occurrences, and most recent occurrence time from the node table; deletes all connection edges with the target node as the starting or ending node from the connection edge table; deletes adjacent node records of other nodes pointing to the target node from the adjacency relationship table; and marks the hotspot association tag corresponding to the target node as invalid and writes the removal time into the tag result table.

[0282] Specifically, for nodes that lose their connection due to the removal of the target node, the update module recalculates the number of its neighboring nodes, the statistical value of the weight of the connected edges, the sum of the weights of the remaining linked edges, and the most recent co-occurrence time.

[0283] Furthermore, after the target node is removed, the update module regenerates the node index table, connection edge table, and adjacency table of the target global hotspot linkage network.

[0284] In cases where there are more than two retained connection edges between two nodes, the update module retains the connection edge with the largest weight value and merges the co-occurrence count, co-occurrence source, and co-occurrence medium form of the remaining connection edges into the retained connection edge; if any endpoint node of a connection edge has been removed, the update module deletes the connection edge; the update module then performs connectivity verification on the remaining nodes, retains the connected branches containing valid nodes and valid connection edges, and writes isolated nodes that do not meet the retention conditions into the historical node database.

[0285] Furthermore, after the updated target global hotspot linkage network is generated, the update module writes the update version number, update time, removed target nodes, severed connection edges, retained nodes, retained connection edges, and connected branch detection results; the target global hotspot linkage network is used for semantic completion, spatial distance calculation, hotspot association tag generation, and invalid word determination of subsequent target UGC text data.

[0286] Example 10: like Figure 1 and Figure 2 As shown, a method for intelligent generation and dynamic adjustment of UGC tags based on multi-source hotspot linkage is also provided. This method employs the aforementioned system for intelligent generation and dynamic adjustment of UGC tags based on multi-source hotspot linkage. The method includes: acquiring raw hotspot data from multiple content sources and in various media formats; processing the raw hotspot data using an entity extraction model to obtain a first core element set containing people and locations; extracting the co-occurrence frequency of each element in the first core element set across different content sources and media formats; if the co-occurrence frequency is greater than a preset threshold, establishing cross-source connections between the elements to obtain an initial global hotspot linkage network; acquiring target UGC text data published by users; performing hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network; and using an entity extraction model to resolve cross-source referencing, resulting in... The second core element set is processed using a graph convolutional network to obtain the spatial distance between the second core element set and the initial global hotspot linkage network. If the spatial distance is less than a preset distance threshold, the target node corresponding to the second core element set in the initial global hotspot linkage network is extracted to obtain hotspot association tags. If the spatial distance is greater than or equal to the preset distance threshold, a hierarchical clustering algorithm is used to process the second core element set to obtain generalized tags. The access popularity value of the target node in the initial global hotspot linkage network is obtained. If the access popularity value is lower than a preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word. The target node corresponding to the invalid word is removed from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network.

[0287] In this embodiment, the method is executed by the tag processing service of a server or content platform. First, the raw hotspot data of at least two content sources are obtained, and the text, image, audio, and video data are converted into standardized text records. Then, the entity extraction model identifies person entities and location entities. After confidence filtering, alias normalization, and homonym disambiguation, the first core element set is obtained.

[0288] Subsequently, semantic vectors for each element in the first core element set are generated based on its source distribution, media distribution, occurrence time, and contextual content. Semantic similarity and co-occurrence frequency between any two elements are then calculated. Co-occurrence frequency is obtained by weighting semantic similarity, cross-content source co-occurrence ratio, cross-media format co-occurrence ratio, and time decay results. When the co-occurrence frequency exceeds a preset threshold, cross-source connections are established between corresponding elements, forming an initial global hotspot linkage network. The preset threshold is determined through verification using labeled hotspot samples.

[0289] Furthermore, after acquiring the target UGC text data, semantic completion is performed on the abbreviations of people, places, pronouns, aliases, and misspelled words based on the initial global hotspot linkage network. Then, the entity extraction model is used to label and complete the text data, and cross-source referencing resolution and clustering are performed on the initial entities with more than two candidate referents to obtain the second core element set.

[0290] Next, the node features and adjacency relationships of the initial global hotspot linkage network are input into the graph convolutional network to obtain the hidden layer representation, which is then reduced in dimensionality to a feature aggregation matrix. After the features in the second core feature set are mapped to the network nodes in the feature aggregation matrix, the spatial distance is calculated based on the mapped node coordinates and feature weights.

[0291] Specifically, when the spatial distance is less than a preset distance threshold, the corresponding target node is extracted, and the hotspot radiation range is determined based on the adjacency relationship of the target node. Then, the node features and connection edge features within the range are classified to obtain hotspot association tags. When the spatial distance is greater than or equal to the distance threshold, a similarity matrix is ​​generated based on the semantic vector of the second core element set, and generalized tags are obtained through hierarchical clustering and node hierarchical mapping. The distance threshold is determined by verification of labeled user-generated content samples.

[0292] Specifically, for hotspot-related tags, the historical access trajectory of the corresponding target node is obtained, and the access popularity value is calculated based on access frequency, access type weight, and time decay results. When the access popularity value is less than the preset popularity threshold, the contextual distribution characteristics of the hotspot-related tags are analyzed. When the tag validity score is less than the corresponding judgment value, the hotspot-related tag is judged as an invalid word.

[0293] Finally, the target nodes are mapped in the initial global hotspot linkage network according to the invalid words, and it is determined whether the linkage edge weight associated with the target node is less than the preset edge weight threshold. If it is less, the corresponding linkage relationship is cut off to form a transition network. The transition network is then subjected to connectivity branch detection, and the target nodes and related connection edges corresponding to the invalid words are removed to obtain the updated target global hotspot linkage network.

[0294] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A UGC tag intelligent generation and dynamic adjustment system based on multi-source hotspot linkage, characterized in that, include: The first acquisition module acquires raw data of hot topics from multiple content sources and in multiple media formats, processes the raw data of hot topics, and obtains a set of first core elements including people and places. The module extracts the co-occurrence frequency of each element in the first core element set in different content sources and media formats. If the co-occurrence frequency is greater than the preset threshold, cross-source connection edges are established between each element to obtain the initial global hotspot linkage network. The second acquisition module acquires the target UGC text data published by the user, performs hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network, and uses an entity extraction model to resolve cross-source referencing to obtain the second core element set. The distance determination module processes the second core element set and the initial global hotspot linkage network to obtain the spatial distance of the second core element set in the initial global hotspot linkage network. If the spatial distance is less than a preset distance threshold, the first generation module extracts the target node corresponding to the second core element set in the initial global hotspot linkage network to obtain the hotspot association tag. The second generation module processes the second core element set to obtain generalized tags if the spatial distance is greater than or equal to a preset distance threshold. The judgment module obtains the access popularity value of the target node in the initial global hotspot linkage network. If the access popularity value is lower than the preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word. The update module removes the target nodes corresponding to invalid words from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network.

2. The UGC tag intelligent generation and dynamic adjustment system based on multi-source hotspot linkage according to claim 1, characterized in that: The first acquisition module acquires raw data of trending topics from multiple content sources and in various media formats, processes the raw data to obtain a first set of core elements including people and locations, including: Obtain raw data on trending topics across multiple media formats; Multimodal feature vectors are obtained by extracting features from the original hotspot data; The fused feature representation is obtained by aligning features based on multimodal feature vectors; An entity extraction model is used to process the fusion feature representation to determine the initial feature sequence containing candidate entities; If the candidate entities in the initial element sequence meet the threshold, the initial element sequence is disambiguated to obtain the first core element set containing people and places.

3. The UGC tag intelligent generation and dynamic adjustment system based on multi-source hotspot linkage according to claim 1, characterized in that: The construction module extracts the co-occurrence frequency of each element in the first core element set across different content sources and media formats. If the co-occurrence frequency is greater than a preset threshold, cross-source connection edges are established between the elements to obtain the initial global hotspot linkage network, including: Obtain the distribution characteristics of the first core element set and extract text features to obtain the semantic vector of the element; Calculate similarity values ​​based on element semantic vectors to obtain element co-occurrence frequencies; If the co-occurrence frequency of elements is greater than a preset threshold, then the association weights of each element are extracted to establish cross-source connection edges. By mapping network nodes through cross-source connection edges and constructing the topology, an initial global hotspot interconnected network is obtained.

4. The intelligent generation and dynamic adjustment system for UGC tags based on multi-source hotspot linkage according to claim 1, characterized in that: The second acquisition module acquires the target UGC text data published by the user, performs hotspot semantic completion on the target UGC text data based on the initial global hotspot linkage network, and uses an entity extraction model to resolve cross-source referencing, resulting in a second core element set including: Obtain the target UGC text data, and perform semantic completion on the target UGC text data through the initial global hotspot linkage network to obtain the completed text data; An entity extraction model is used to annotate the completed text data to obtain initial entities; Determine if the initial entity has a reference conflict. If the initial entity has a reference conflict, perform cross-source reference resolution on the initial entity to obtain the resolved entity. The second core element set is obtained by decomposing the entity clusters.

5. The UGC tag intelligent generation and dynamic adjustment system based on multi-source hotspot linkage according to claim 1, characterized in that: The distance determination module processes the second core element set and the initial global hotspot linkage network to obtain the spatial distance of the second core element set in the initial global hotspot linkage network, including: Extract the node feature vectors and adjacency matrices of the initial global hotspot linkage network, and process the node feature vectors and adjacency matrices through a graph convolutional network to obtain the hidden layer representation; The hidden layer representation is dimensionality reduced to obtain a feature aggregation matrix. The weights of the second core element set are extracted and mapped based on the feature aggregation matrix to obtain the spatial distribution coordinates. Calculate the spatial distribution coordinates to obtain the spatial distance of the second core element set in the initial global hotspot linkage network.

6. The UGC tag intelligent generation and dynamic adjustment system based on multi-source hotspot linkage according to claim 1, characterized in that: If the spatial distance is less than a preset distance threshold, the first generation module extracts the target node corresponding to the second core element set in the initial global hotspot linkage network to obtain hotspot association tags, including: Determine if the spatial distance is less than a preset distance threshold; If the spatial distance is less than the preset distance threshold, then extract the corresponding target node in the initial global hotspot linkage network; Determine the hotspot radiation range based on the target node; Obtain the associated feature vectors within the hotspot's radiation range, perform classification processing based on the associated feature vectors, and obtain hotspot association labels.

7. The intelligent generation and dynamic adjustment system for UGC tags based on multi-source hotspot linkage according to claim 1, characterized in that: The second generation module, if the spatial distance is greater than or equal to a preset distance threshold, processes the second core element set to obtain generalized tags, including: If the spatial distance is greater than or equal to the preset distance threshold, the similarity matrix is ​​calculated based on the semantic vector of the second core element set. Hierarchical clustering algorithm is used to process the similarity matrix to obtain clusters; Based on the clusters, a node hierarchy is constructed. The second core element set is then processed through the node hierarchy to obtain generalized labels.

8. The intelligent generation and dynamic adjustment system for UGC tags based on multi-source hotspot linkage according to claim 1, characterized in that: The determination module obtains the access popularity value of the target node in the initial global hotspot linkage network. If the access popularity value is lower than a preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word, including: Obtain the historical access trajectory of the target node in the initial global hotspot linkage network, and extract the access frequency sequence based on the historical access trajectory; The access frequency sequence is subjected to heat decay calculation to obtain the access heat value of the target node; If the access popularity value is lower than the preset popularity threshold, then extract the hotspot association tags corresponding to the target node; The hotspot association tags are parsed to obtain contextual distribution features, and the hotspot association tags corresponding to the target node are determined to be invalid words based on the contextual distribution features.

9. The intelligent generation and dynamic adjustment system for UGC tags based on multi-source hotspot linkage according to claim 1, characterized in that: The update module removes the target nodes corresponding to invalid words from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network, which includes: Based on the invalid words, node mapping is performed in the initial global hotspot linkage network to determine the target node corresponding to the invalid word and the linkage edge weight associated with the target node; Determine whether the weight of the linked edge is less than the preset edge weight threshold; If the weight of the linked edge is less than the preset edge weight threshold, the linked relationship corresponding to the target node is cut off, and a transition network containing an isolated subgraph is obtained. Connectivity branch detection is performed on the transition network, and the target nodes corresponding to the failed words are removed from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network.

10. A method for intelligent generation and dynamic adjustment of UGC tags based on multi-source hotspot linkage, employing the intelligent generation and dynamic adjustment system for UGC tags based on multi-source hotspot linkage as described in any one of claims 1-9, characterized in that, The method includes: We acquire raw data on trending topics from multiple content sources and in various media formats, and process the raw data using an entity extraction model to obtain a first set of core elements containing people and locations. Extract the co-occurrence frequency of each element in the first core element set in different content sources and media formats. If the co-occurrence frequency is greater than the preset threshold, establish cross-source connection edges between each element to obtain the initial global hotspot linkage network. The target UGC text data published by users is obtained. Based on the initial global hotspot linkage network, the target UGC text data is semantically completed by hotspots. The entity extraction model is used to resolve cross-source referencing to obtain the second core element set. The spatial distance between the second core element set and the initial global hotspot linkage network is obtained by processing the second core element set with a graph convolutional network. If the spatial distance is less than the preset distance threshold, the target node corresponding to the second core element set in the initial global hotspot linkage network is extracted to obtain the hotspot association tag; If the spatial distance is greater than or equal to the preset distance threshold, a hierarchical clustering algorithm is used to process the second core element set to obtain a generalized label. Obtain the access popularity value of the target node in the initial global hotspot linkage network. If the access popularity value is lower than the preset popularity threshold, the hotspot association tag corresponding to the target node is determined to be an invalid word. The target nodes corresponding to invalid words are removed from the initial global hotspot linkage network to obtain the updated target global hotspot linkage network.

Images