New media layout method facing semantic label matching

Abstract

The present application relates to computer data processing and layout technology field, specifically to a new media layout method for semantic label matching. The method divides the semantic units of new media content, extracts entity, theme, action, emotion and content form labels, and constructs a content semantic relationship graph according to the modification, causality, reference, time sequence and inclusion relationship; the visual level, spatial adjacency, reading order and content subordination of the candidate layout area are obtained to construct a layout role constraint graph; the initial layout result is generated by compatible mapping of semantic dependency type and layout role relationship, the label relationship is reconstructed in reverse according to the layout relationship, and the conflict residual is formed, and local rematching is performed on the conflict nodes and associated nodes. The method can keep the content master-slave hierarchy, causality order and graphic reference relationship consistent, reduce the layout fluctuation caused by overall semantic structure dislocation and irrelevant area repeated rearrangement.

Description

Technical Field

[0001] This invention relates to the field of computer data processing and typesetting technology, specifically to a new media typesetting method oriented towards semantic tag matching. Background Technology

[0002] New media content typically consists of titles, leads, body text, images, captions, quotations, and emphasis text. Layout systems need to allocate different content segments to corresponding areas within a template based on content type and semantic tags. Existing technologies often employ tag extraction and template matching, performing word segmentation, entity recognition, topic classification, or sentiment analysis on the content to be formatted to obtain title tags, topic tags, object tags, action tags, and content form tags. These tags are then converted into feature vectors, and their similarity is calculated against the pre-configured role features of the title area, main image area, body text area, quotation area, and emphasis area. The system selects the target layout area based on the similarity ranking results and fills the content by considering character count, image ratio, area size, and preset reading order. When a certain area is insufficient, adaptation is usually achieved by truncating text, scaling images, switching templates, or extending the remaining content to the next page. This approach can automatically allocate tags to layout areas, but its matching objects are mainly independent individual tags and individual areas.

[0003] To reduce content type misjudgments caused by single-tag matching, some existing technologies add contextual encoding, tag weights, template constraints, or graph structure representations. A common approach is to contextually encode words and sentences within the same paragraph, adjust the contribution of each tag to layout selection based on tag confidence, and then use preset regional order, area constraints, and content type constraints to eliminate candidate regions that do not meet the conditions. Another approach represents text units as nodes and adjacency or co-occurrence relationships as edges, then selects layout templates with high similarity through graph embedding results. After template selection, the system typically still fills the layout area sequentially according to the local scores of each content unit, and recalculates the local scores or replaces the entire template when conflicts occur. Although these approaches introduce contextual or structural information, the modification, causal, referential, temporal, and inclusion relationships between semantic tags, and the visual hierarchy, spatial adjacency, reading order, and content subordination relationships between layout areas, still do not form a bidirectionally verifiable correspondence.

[0004] Semantic tags and layout areas are typically matched based on local similarity or one-way constraints, making it impossible to determine whether the overall semantic relationship of the original content is preserved after combining multiple local matching results. For new media content containing hierarchical, causal, referential, or temporal relationships, a single tag may be suitable for multiple layout areas, but the combination mapping of different tags may cause secondary information to occupy a higher visual hierarchy, placing causal content after result content, or distributing images, captions, and the referred text in unrelated positions. Because the existing layout process lacks a step to reverse-engineer tag relationships based on layout relationships, and also lacks a mechanism to locate conflict nodes based on the difference between the reconstructed relationship and the original relationship, the system struggles to identify layout results with high local scores but incorrect overall semantic structure, relying solely on global re-layout or manual review. This leads to semantic relationships being reversed, hierarchically misplaced, or broken during layout conversion. Summary of the Invention

[0005] The purpose of this invention is to provide a new media layout method based on semantic tag matching, which can solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A new media layout method oriented towards semantic tag matching includes: dividing the new media content to be layout into semantic units, extracting entity tags, topic tags, action tags, sentiment tags and content form tags for each semantic unit, and constructing a content semantic relationship graph based on the modification, causal, referential, temporal and inclusion relationships between tags; Obtain the layout areas in the candidate layouts and the visual hierarchy, spatial adjacency, reading order and content hierarchy between the areas, and construct a layout role constraint diagram; Based on the compatible mapping between semantic dependency type and layout role relationship, the nodes and edges in the content semantic relationship graph are jointly mapped to the layout role constraint graph to obtain the initial layout matching result; Based on the initial layout matching results, the tag relationships are reconstructed in reverse, and the reconstructed tag relationships are compared with the original tag relationships to generate conflict residuals. Lock the mapping nodes that have not generated the conflict residuals, perform local rematching on the mapping nodes that have generated the conflict residuals and their associated nodes, and output a layout result that satisfies the bidirectional relationship consistency.

[0007] Preferably, constructing the content semantic relationship graph includes: dividing continuous text, titles, captions, quoted text, and accompanying image descriptions into semantic units with independent referential objects; For each semantic unit, a set of tag nodes is established, and modification relationship edges are determined according to the syntactic dominance direction within the same semantic unit. Temporal relationship edges and causal relationship edges are determined according to the sequence of events and conjunctions across semantic units. Referential relationship edges are determined according to entity coreference chains. Inclusion relationship edges are determined according to the degree of topic inclusion between the semantic units. Record the relationship direction, relationship type, evidence source, and evidence consistency for each relationship edge, and merge the evidence when multiple pieces of evidence point to the same labeled relationship to form a directed multi-relation graph for joint mapping, where the same labeled node can be connected to multiple associated labeled nodes through different relationship types.

[0008] Preferably, constructing the layout role constraint diagram includes: establishing layout role nodes for the title area, lead area, main image area, auxiliary image area, body text area, quotation area and emphasis area in the candidate layout; Spatial adjacency edges are generated based on the relative position of the layout area in the page coordinate system; visual hierarchy edges are generated based on font size level, area area ratio and salience order; reading order edges are generated based on the preset reading path and the sequential connection relationship of the areas; and content subordination edges are generated based on the parent and child container relationship between the areas. The content format that each layout role node can carry, the semantic relationship types that are allowed to be connected, and the acceptable relationship directions are written into the node constraints, and the node constraints and various layout relationship edges are jointly encoded into the role topology structure of the candidate layout.

[0009] Preferably, the compatibility mapping and the conflict residual generation include: generating a set of node candidates corresponding to each layout role node for each tag node in the content semantic relationship graph, and deleting incompatible candidates based on the matching relationship between the tag type and the content form that the layout role can carry; For the retained node candidates, the correlation degree between semantic relationship edges and visual hierarchy edges, spatial adjacency edges, reading order edges and content subordination edges is jointly calculated to form a node-edge joint matching matrix; Select a mapping combination from the node-edge joint matching matrix that satisfies the unique node occupancy constraint and the relationship direction constraint; Based on the layout relationship in the mapping combination, the predicted relationship between tags is generated in reverse, and the residual components are recorded according to the relationship type difference, relationship direction reversal, hierarchical crossing and reference interval, and the conflict residual is obtained by combination.

[0010] Preferably, multiple pieces of evidence are merged, including converting syntactic control evidence, conjunction evidence, entity coreference evidence, and topic inclusion evidence into relational evidence items with source identifiers respectively; Relation evidence items are merged according to the label node pairs and relation directions. When different relation types exist in the same label node pair, parallel relation edges are retained. When there are directional conflicts in the same relation type, a direction competition record is established. Based on the repetition of each relation evidence item in adjacent semantic units and the consistency across content forms, an evidence consistency sequence is generated, and the main relation edge and candidate relation edge are determined based on the evidence consistency sequence. The primary relation edge is written into the basic edge set of the content semantic relation graph, and the candidate relation edge is written into the replaceable edge set, so that subsequent local rematching can perform constrained replacement of relation edges without changing the label node.

[0011] Preferably, the encoding of the character topology includes: establishing a node description vector for each layout character node, consisting of a character identifier, a visual hierarchy number, a reading path number, a region parent-child identifier, a content form set, and a set of connectable relationships; For each layout relation edge, an edge description vector is created, consisting of the starting role, ending role, edge type, directional constraint, and cross-region span. The node description vectors are sorted according to the reading path sequence number, and the edge description vectors corresponding to adjacent nodes and non-adjacent reference nodes are embedded into the sorting result to form a sequential-topological joint encoding. When the candidate layout contains optional or merged regions, branch codes are generated for different region enable states, and the node retention relationship in each branch code is constrained according to the region parent-child identifier, resulting in multiple layout role code branches that can be called by the node-edge joint matching matrix.

[0012] Preferably, the reverse generation of predicted relationships between tags includes: reading the layout role nodes that carry each tag node in the mapping combination, and retrieving the direct relationship edges between the layout role nodes and the composite relationship paths within a limited number of steps along the layout role constraint graph; The visual hierarchy edges are converted into label master-slave prediction relationships, the reading order edges are converted into label temporal prediction relationships, the content subordinate edges are converted into label inclusion prediction relationships, and the spatial adjacency edges between the main image area or auxiliary image area and the corresponding caption area are converted into label reference prediction relationships. When the same label node corresponds to multiple composite relationship paths, a candidate set of predicted relationships is established based on the combination of edge types in the path. Candidate relationships with the same type and direction as the original relationship in the content semantic relationship graph are written into the consistent relationship set, and the remaining candidate relationships are written into the conflict relationship set.

[0013] Preferably, performing the local rematch includes: establishing a conflict subgraph based on the tag nodes, layout role nodes and relationship edges involved in the conflict residual, and adding the mapping nodes that have not entered the conflict subgraph and have no directional competition records with the conflict subgraph to the lock set; Within the conflict subgraph, cancel the original mapping combination while retaining the layout role occupancy status corresponding to the locked set; A new node candidate set is generated from the layout role nodes that have not been occupied by the locked set, and the relationship type difference, relationship direction reversal, hierarchy crossing and reference interval in the conflict residual are respectively converted into candidate deletion constraints. The node-edge joint matching matrix is ​​updated according to the deletion constraint, a new local mapping combination is selected, and the new local mapping combination is concatenated with the locked set before the reverse relation reconstruction is performed again.

[0014] Preferably, it also includes: when candidate relation edges in the replaceable edge set participate in reverse relation reconstruction, establishing an edge replacement pairing table corresponding to the predicted relation candidate set for each candidate relation edge; When the number of conflicting relationships between the main relationship edge and the layout relationship is greater than the number of conflicting relationships between the candidate relationship edge and the layout relationship, the candidate relationship edge is temporarily replaced with the corresponding main relationship edge, and the source identifier of the replaced main relationship edge is retained. Regenerate consistent and conflicting relation sets for the semantic relation graph after edge replacement; When a new label node pair conflicts or disrupts the original consistent relationship after replacement, the edge replacement is revoked. When no new conflict is added after replacement, the corresponding candidate relationship edge is transferred to the basic edge set, and the updated basic edge set is used for the next round of node-edge joint mapping and reverse relationship reconstruction.

[0015] Preferably, after performing the reverse relation reconstruction again, the method further includes: comparing the boundaries of the conflicting subgraphs before and after the rematch, and identifying the boundary relation edges that still contain conflicting residuals and are connected to the locked set; According to the direction of the boundary relationship edge, the corresponding locked node is transferred to the set to be unlocked, and the expansion order is determined based on the number of relationship types between the node to be unlocked and the conflict subgraph, the reading path span, and the hierarchical connection depth. Each time a mapping node or a group of mapping nodes with common relationship edges is released from the set to be unlocked, the released mapping node is merged into the conflict subgraph and the layout role occupancy status is updated. After each expansion, node candidate generation, candidate deletion, local mapping, and reverse relationship reconstruction are re-executed until the conflict residual is empty, or the conflict subgraph covers all mapped nodes, at which point the process transitions to full graph joint mapping.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By encoding the modification, causal, referential, temporal, and inclusion relationships among entity tags, topic tags, action tags, sentiment tags, and content form tags, a content semantic relationship graph is formed. Furthermore, a layout role constraint graph is created by mapping the visual hierarchy, spatial adjacency, reading order, and content subordination relationships among candidate layout areas. This transforms layout matching from an independent comparison of individual tags and areas into a joint mapping of tag nodes, semantic relationship edges, layout role nodes, and layout relationship edges. After initial mapping, predicted relationships between tags are generated in reverse based on the assigned layout relationships. These are then compared with the original tag relationships to form conflict residuals corresponding to relationship type, relationship direction, hierarchical span, and referential interval. This allows for the identification of mapping combinations where local matching is valid but the overall semantic relationship is inconsistent. Local re-matching is performed on conflicting and associated nodes, while retaining non-conflicting mapping nodes, ensuring that the information hierarchy, causal order, referential correspondence, and content inclusion relationships in the output layout remain consistent with the original semantic structure.

[0017] 2. The content semantic relationship graph adopts a directed multi-relation representation with relational direction, relational type, evidence source, and evidence consistency. It can retain parallel relationships when multiple semantic connections exist between the same tag node pairs, and establish replaceable relational edges when evidence directions conflict. The layout role constraint graph writes role identifiers, visual hierarchy numbers, reading path numbers, regional parent-child relationships, content form sets, and allowed connection relationships into the joint encoding of nodes and edges, allowing for different layout branches to be formed for selectable and merged regions. During local rematching, a conflict subgraph is established based on the conflict residual, freezing the layout occupancy state unrelated to the conflict, and converting relational direction reversal, hierarchy crossing, and referential intervals into candidate deletion conditions, reducing redundant calculations of irrelevant nodes. If the conflict is still not resolved, locked nodes are gradually released and the conflict subgraph is expanded according to boundary relationships and dependency directions. Simultaneously, acceptable mappings are retained through relational edge replacement and reverse reconstruction verification, increasing the scope of layout correction with the propagation range of semantic conflict, avoiding layout structure fluctuations caused by insufficient local adjustments or direct full-map re-layout. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the overall process of a new media layout method based on semantic tag matching; Figure 2 A flowchart for constructing a content semantic relationship graph; Figure 3 Flowchart for constructing layout role constraint graph and node-edge joint mapping; Figure 4 Flowchart for conflict residual handling, local rematching, and conflict subgraph expansion; Figure 5 A continuous tracking graph of the consistency of relational evidence; Figure 6For the joint mapping evaluation quantity continuous evaluation graph; Figure 7 The convergence plot of the conflict residual closed loop; Figure 8 Replace the net income tracking chart with the relationship edge; Figure 9 Release the priority sorting graph for nodes. Detailed Implementation

[0019] In one embodiment, please refer to Figure 1 This paper provides a new media layout method based on semantic tag matching, including: dividing the new media content to be layout into semantic units, extracting entity tags, topic tags, action tags, sentiment tags, and content form tags for each semantic unit, and constructing a content semantic relationship graph based on the modification, causal, referential, temporal, and inclusion relationships between tags; obtaining the layout areas in the candidate layouts and the visual hierarchy, spatial adjacency, reading order, and content subordination relationships between areas, and constructing a layout role constraint graph; based on the compatibility mapping between semantic dependency types and layout role relationships, jointly mapping the nodes and edges in the content semantic relationship graph to the layout role constraint graph to obtain the initial layout matching result; and reconstructing the tag relationships in reverse based on the initial layout matching result, and then connecting the reconstructed tag relationships with... The original tag relationships are compared to generate conflict residuals; mapping nodes that do not generate conflict residuals are locked, and local rematching is performed on the mapping nodes that generate conflict residuals and their associated nodes. The output is a layout result that satisfies the consistency of bidirectional relationships. In this embodiment, the new media content to be layout is a set of content objects formed after unified character encoding and media reference parsing. Each content object records at least the content identifier, content form, original position, context scope and media reference relationship. During the layout process, individual keywords are not directly bound to the layout area. Instead, the relationship direction between tag nodes and the topological constraints between layout role nodes are preserved at the same time, so that the semantic structure can participate in the layout area selection and the layout selection result can be reversed to accept the original semantic structure verification.

[0020] In this embodiment, semantic unit segmentation adopts a processing method that combines boundary candidate generation and boundary consistency verification. The system reads paragraph boundaries, syntactic pause positions, referential object change positions, topic center change positions, and media reference insertion positions in continuous text, and uses these positions as candidate boundaries. Independent semantic units are directly established for titles, captions, cited text, and illustration descriptions. For the main text content, the text fragments between adjacent candidate boundaries are used as initial semantic units. Subsequently, the entity overlap relationship, action succession relationship, causal connection relationship, and topic continuity relationship between adjacent initial semantic units are calculated. When two adjacent initial semantic units share the same central entity and the action in the latter semantic unit depends on the action result in the former semantic unit, the independent nodes of the two are retained and cross-unit relationship edges are established. When the candidate boundary is formed only by typesetting line breaks or punctuation habits and the central entity, action predicate, and topic center on both sides of the boundary are consistent, the text fragments on both sides of the candidate boundary are merged. Specifically, each semantic unit also saves its starting character position, ending character position, belonging content object, cited media identifier, and context index, supporting subsequent relationship evidence backtracking and content backfilling after local rematching.

[0021] Furthermore, entity tags are obtained by uniformly identifying named objects, ordinary objects, and pronoun referents in semantic units. For the full name, abbreviation, and pronouns of the same object, a coreference chain is established and merged into the same entity tag. Topic tags are obtained by extracting noun phrases and event objects with a central dominance relationship in semantic units. The same semantic unit can record main topic tags and subordinate topic tags. Action tags are composed of predicates and their objects to distinguish events formed when the same action word corresponds to different objects. Sentiment tags are used to record the evaluation direction, intensity level, and target of the semantic unit. Sentiment tags do not directly determine the layout area, but serve as an auxiliary constraint for the candidate relationship between the emphasis area and the quotation area. Content form tags are determined based on continuous text, titles, images, captions, quotation text, list content, and combined text and images. The system records the source semantic unit, tag category, standardized value, contextual evidence, and traceable location for each tag. When the same semantic unit contains multiple tags, they are stored as a set of tag nodes without being merged in advance to avoid topic tags, action tags, and entity tags losing their respective semantic roles before entering relation modeling.

[0022] In one embodiment, please refer to Figure 2The construction of the content semantic relationship graph includes: dividing continuous text, titles, captions, cited text, and accompanying image descriptions into semantic units with independent referents; establishing a set of tag nodes for each semantic unit, and determining modification relationship edges based on the syntactic dominance direction within the same semantic unit, determining temporal and causal relationship edges based on the event sequence and conjunctions across semantic units, determining referential relationship edges based on entity coreference chains, and determining inclusion relationship edges based on the topic inclusiveness between semantic units; recording the relationship direction, relationship type, evidence source, and evidence consistency for each relationship edge, and merging the evidence when multiple pieces of evidence point to the same tag relationship to form a directed multi-relation graph for joint mapping, wherein the same tag node is allowed to pass through different... Multiple related tag nodes are connected to the same relationship type. In specific implementation, the direction of the modifying relationship edge is from the modifying tag to the modified tag, the direction of the causal relationship edge is from the cause event tag to the result event tag, the direction of the temporal relationship edge is from the earlier event tag to the later event tag, the direction of the referential relationship edge is from the referent expression tag to the referent entity tag, and the direction of the containing relationship edge is from the superior subject tag to the subordinate subject tag. Each relationship edge consists of a record item composed of a start node identifier, an end node identifier, a relationship type, a relationship direction, a set of evidence items, and a current state. The current state includes basic edges, candidate edges, temporary replacement edges, and revocation edges, so that the same tag node pair can simultaneously retain causal and temporal relationships without being compressed into a single edge.

[0023] In this embodiment, the consistency of relational evidence is calculated by combining the reliability of the evidence source, the consistency of the relational direction, and the reproducibility of the context. A tag node is set. With tag nodes The relationship type is t, and the corresponding number of evidence items is . The source weight of the q-th piece of evidence is The quantity with the same direction is The context-consistent quantity is Then the amount of evidence related to the relationship Determined according to the following formula: ; in, Based on a pre-defined category of evidence source that remains unchanged within the same typesetting task, syntactic dominance evidence, explicit conjunction evidence, entity coreference evidence, and topic inclusion evidence each use their corresponding source weights. The value is 1 when the direction of the evidence aligns with the current relationship direction, -1 when the direction is opposite, and 0 when the direction cannot be determined. The value is between 0 and 1, determined by the recurrence of the same relation in adjacent semantic units or across content formats. For example, if the causal text contains a conjunction indicating cause and effect and subsequent captions again explain the same cause and effect, the two evidence items are considered. All values ​​are set to 1, and reproduction is achieved across text and captions. near The result For a positive value to enter the basic edge candidate, when one conjunction points to the cause first and another syntactic evidence points to the cause later, the two evidence items are... With opposite signs, the amount of evidence obtained for the relationship is close to the neutral range, and the corresponding relationship enters the set of replaceable edges instead of being directly used as the basic edge.

[0024] Please refer to Figure 5 , Figure 5 The horizontal axis represents the sequence number of the continuous sampling window for relational evidence, ranging from 1 to 120. The vertical axis represents the strength of relational evidence, ranging from approximately 0.817 to 0.924. The black solid line represents the dynamic strength of relational evidence, the gray dashed line represents the first warning threshold of 0.740, and the gray dotted line represents the second warning threshold of 0.900. This figure corresponds to the calculation process of the consistency of relational evidence. The sampling window is formed by the modification, causal, referential, temporal, and inclusion relationship evidence between semantic units such as titles, introductions, body text, images, and captions. The curve values ​​are jointly determined by the reliability of the evidence source, the consistency of the relationship direction, and the recurrence in the context.

[0025] from Figure 5 As can be seen, all 120 sampling windows were above the first warning threshold, with 30 windows exceeding the second warning threshold. The highest value of 0.924 appeared in the 102nd sampling window, while the final value of 0.907 remained within the high confidence interval. The first part of the curve fluctuated around 0.84 to 0.87, indicating that relatively stable evidence had been formed between the initial semantic units. The middle part briefly dropped to 0.817, usually corresponding to local windows with insufficient conjunctions, entity coreference, or figure caption reproduction. The latter part of the curve continued to rise and repeatedly crossed the second warning threshold, indicating that the evidence of the same direction across text and media was strengthened.

[0026] The overall trend indicates that this method, when constructing a content semantic relationship graph, does not rely solely on single label similarity but rather merges evidence from multiple sources into a traceable sequence of relationship strengths. When the curve remains high, the basic relationship edges between the cause paragraph, the result paragraph, the main graph, and the caption can be directly included in subsequent joint mappings; when the curve is at a low point but still above the first threshold, these should be retained as replaceable relationship edges and reviewed during local rematching. This graph illustrates the engineering judgment criteria between relationship evidence merging, basic edge selection, and candidate edge retention.

[0027] In a preferred embodiment, when constructing the content semantic relationship graph, the system also performs hierarchical verification of topic inclusion relationships. The system constructs a topic coverage set by the keyword set, entity set, and action set corresponding to each topic tag. When the coverage set of topic tag A contains the main entities and actions of topic tag B and topic tag B only describes local events in topic tag A, an inclusion relationship edge pointing from topic tag A to topic tag B is established. When the coverage sets of two topic tags partially overlap but do not have a stable superior-subordinate relationship, no inclusion relationship edge is established, but a parallel association record is established. The parallel association record only participates in the layout area adjacency constraint and does not participate in the master-slave hierarchical reverse reconstruction. For referential relationships, the system searches along the co-reference chain for the candidate entity that is closest to the current referent and meets the entity category constraint, and determines the referent object by combining the media reference relationship between the title, body text, image, and caption. When the referent in the caption may point to both the main image and the auxiliary image, the system retains multiple referential candidate edges and records their respective media reference evidence. After the layout role mapping is completed, the referential edge consistent with the layout topology is selected through spatial adjacency relationship and caption subordinate relationship.

[0028] In one embodiment, please refer to Figure 3 The construction of the layout role constraint diagram includes: establishing layout role nodes for the title area, lead area, main image area, auxiliary image area, body text area, quotation area, and emphasis area in the candidate layout; generating spatial adjacency edges based on the relative positions of the layout areas in the page coordinate system; generating visual hierarchy edges based on font size level, area proportion, and salience order; generating reading order edges based on the preset reading path and the sequential connection relationship between areas; and generating content subordination edges based on the parent and child container relationship between areas; writing the content form that each layout role node can carry, the allowed semantic relationship type, and the acceptable relationship direction into the node constraints, and connecting the node constraints with each... The relationship edges of the layout are jointly encoded as the role topology of the candidate layout. Specifically, the title area node allows the title form and high-level topic tags, the lead area node allows the continuous text that summarizes the topic, the main image area node allows the image that is directly related to the main topic entity or main action event, the auxiliary image area node allows the image that is related to the subordinate topic or explanatory event, the caption is a child node of the main image area or auxiliary image area, the body area allows the continuous text with a complete semantic chain, the quotation area allows the quotation text with source identification, and the emphasis area allows the short text that is clearly marked by the original content or is jointly limited by the sentiment tag and the topic center.

[0029] In this embodiment, the visual hierarchy edge is not determined solely by the layout area, but rather by the area salience number and the parent-child relationship. A visual hierarchy edge is established between the title area and the body text area, pointing from the title area to the body text area. Similarly, a visual hierarchy edge is established between the main image area and the auxiliary image area, pointing from the main image area to the auxiliary image area. No independent hierarchical edge is established between the caption area and the corresponding image area; instead, a content-dependent edge is established pointing from the image area to the caption area. Spatial adjacency edges store the adjacent direction between two layout role nodes, the existence of an intermediate area, and the acceptable reference span. Reading order edges are generated according to the preset reading path of the candidate layout. In a vertical flow layout, the edge points from the upper area to the lower area; in a column layout, a directed path is formed according to the order within and between columns. Content-dependent edges store the parent role, child role, and the allowed subordinate content form. When the caption area is subordinate to the main image area, the referencing relationship between the caption label and the main image label can only be mapped to the corresponding parent-child role combination and cannot cross over to auxiliary image areas without a subordinate relationship.

[0030] Furthermore, the encoding of the role topology structure includes: establishing a node description vector for each layout role node, consisting of a role identifier, visual hierarchy number, reading path number, region parent-child identifier, content form set, and connectable relationship set; establishing an edge description vector for each layout relationship edge, consisting of a starting role, ending role, edge type, direction constraint, and cross-region span; sorting the node description vectors according to the reading path number, and embedding the edge description vectors corresponding to adjacent nodes and non-adjacent reference nodes into the sorting result to form a sequence-topology joint encoding; when the candidate layout contains optional regions or merged regions... Branch codes are generated for different regions with different activation states. The node retention relationship in each branch code is constrained according to the region parent-child identifier, resulting in multiple layout role code branches that can be called by the node-edge joint matching matrix. In the specific data structure, the node description vector and the edge description vector are stored in the role node table and the role edge table, respectively. The sequential-topological joint coding uses the reading path sequence number as the primary index and the role identifier as the secondary index. Each role node is followed by the edge index directly connected to it and the composite path index within the limited span, so that the node content form constraint and the directional constraint between nodes can be read simultaneously during joint mapping.

[0031] In a preferred embodiment, the candidate layout includes a main image optional branch, a quotation area optional branch, and a text area merged branch. When there is no image in the content to be typed, the system generates a branch code to close the main image area and its caption area, and deletes all content subordinate edges pointing to the main image area and caption area. When there is an image but no independent quotation text, the system generates a branch code to close the quotation area, and allows the text area originally adjacent to the quotation area to expand into a merged area. When the text content consists of multiple semantic units with complete causal chains, the system retains multiple text sub-area branches. When the text content cannot be split at the semantic boundary, the system generates a text merged branch and prohibits mapping the same indivisible semantic unit to multiple text role nodes. Multiple branch codes share the same role type definition and relationship type definition, and only differ in the node activation state, parent-child relationship retention state, and reading path connection state, to avoid template branches changing the semantic meaning of the same role.

[0032] In one embodiment, the compatibility mapping and conflict residual generation includes: generating a candidate set of nodes corresponding to each layout role node for each tag node in the content semantic relationship graph, and deleting incompatible candidates based on the matching relationship between tag type and the content form that the layout role can carry; for the retained node candidates, jointly calculating the correspondence degree between semantic relationship edges and visual hierarchy edges, spatial adjacency edges, reading order edges, and content subordinate edges to form a node-edge joint matching matrix; selecting mapping combinations that satisfy the unique node occupancy constraint and relationship direction constraint from the node-edge joint matching matrix; and generating labels in reverse based on the layout relationships in the mapping combinations. The relationships between tags are predicted, and residual components are recorded according to differences in relationship type, reversal of relationship direction, hierarchical crossing, and referential interval. The conflict residuals are then combined to obtain the conflict residuals. The generation of the node candidate set is based on content form compatibility, with additional restrictions on topic role, relationship direction, and parent-child constraints. For example, title form tags only enter the title area candidate set, image form tags only enter the main image area or auxiliary image area candidate set, and caption form tags only enter the caption area candidate set belonging to the image area. Ordinary text tags are not allowed to enter the main image area. Caption tags with referential edges and the image tags they refer to must enter the role combination with content subordinate edges or allowed to reference adjacent edges.

[0033] In this embodiment, the joint mapping uses binary variables to represent the mapping state between tag nodes and layout role nodes. Let the tag node in the content semantic relationship graph be... The character nodes in the layout character constraint diagram are Mapping variables exist Mapped to The value is 1 if the condition is met, and 0 otherwise. The node compatibility is... tag node and Semantic relationships and role nodes and The correspondence between the layout relationships is The total number of violations of unique occupancy constraints, content form constraints, or relationship direction constraints is [number missing]. The joint evaluation quantity of the mapping combination X Determined according to the following formula: ; in, Used to balance node compatibility and relationship mapping This is used to restrict combinations of mappings that violate constraints, and their values ​​are greater than the compatibility of any single node. The node compatibility value is... The quantity of relationships is determined by the tag category, content format, and the set that the role can carry. Positive values ​​are assigned when causal relationships are mapped to reading paths with consistent order; positive values ​​are assigned when master-slave relationships are mapped to visual hierarchy edges with consistent order; positive values ​​are assigned when referential relationships are mapped to the subordinate relationship between an image and its corresponding caption; and negative values ​​are assigned when the relationship direction is opposite. For example, when the cause label is placed before the result label and the reading order between candidate roles is also from front to back, the corresponding value is positive. If the cause label is placed after the result label, the relationship corresponding value is negative and the joint evaluation value of the mapping combination is reduced.

[0034] Please refer to Figure 6 , Figure 6 The horizontal axis represents the candidate pattern mapping combination number, ranging from 1 to 100, and the vertical axis represents the mapping evaluation intensity, ranging from approximately 34.792 to 37.082. The solid black line represents the comprehensive evaluation result of each candidate mapping combination, while the dashed gray line and the dotted gray line represent the reference line for further arrangement and the stable reference line, respectively. Figure 6 The evaluation process of joint mapping between corresponding tag nodes and layout role nodes, each number on the horizontal axis represents a candidate combination generated by node compatibility, relationship correspondence and constraint deduction.

[0035] from Figure 6 As can be seen, all 100 candidate combinations are above the stable reference line, with an average evaluation strength of 35.997 and a maximum value of 37.082 appearing in the 100th candidate combination. The curve gradually rises from around 34.792 in the early stage, forming a stable plateau in the middle and later stages. This indicates that the node matching relationship between content objects such as titles, introductions, body text, main images, and captions and layout roles is gradually converging. At the same time, causal, hierarchical, referential, and temporal relationships are consistent in reading order, visual hierarchy, and content subordination structure.

[0036] Should Figure 6This reflects the key difference between this method and local label matching: similarity between a single semantic unit and a single layout area does not necessarily indicate a reasonable overall layout. Only when cross-node semantic relationships maintain directional consistency in layout role relationships will the mapping evaluation strength remain high. If a candidate combination shows a significant downward shift, priority should be given to checking whether the caption in that combination is detached from the main image, whether the cause paragraph is placed after the result paragraph, and whether the subordinate topic occupies an excessively high visual hierarchy. A stable platform in the figure indicates that the current candidate branch has met the conditions for entering reverse relationship reconstruction and conflict residual verification.

[0037] Specifically, the unique occupancy constraint of a node is set separately according to whether the content object can be reused. Title text, body semantic units, images and captions can only be mapped to one layout role node. Theme tags can be used as semantic attributes of the same content object to participate in multiple relation calculations but do not generate duplicate content copies. Whether a role node can carry multiple content objects is determined by the area type of the candidate layout. The title area and main image area adopt single object occupancy. The body area allows multiple semantic units connected in the reading order to be occupied sequentially. The emphasis area can only carry consecutive short texts from the same semantic unit. When the system selects mapping combinations from the candidate set, it uses the joint evaluation score as the sorting basis. Under the same evaluation score, it prioritizes the mapping combination with more basic relation edges, fewer candidate relation edges and no triggered direction competition records, so that the initial layout matching result retains more semantic relationships with clear evidence.

[0038] In one embodiment, the reverse generation of tag prediction relationships includes: reading the layout role nodes carrying each tag node in the mapping combination, and retrieving the direct relationship edges and composite relationship paths within a limited number of steps between the layout role nodes along the layout role constraint graph; converting visual hierarchy edges into tag master-slave prediction relationships, reading order edges into tag temporal prediction relationships, content subordinate edges into tag inclusion prediction relationships, and spatial adjacency edges between the main image area or auxiliary image area and the corresponding caption area into tag reference prediction relationships; when the same tag node corresponds to multiple composite relationship paths, establishing prediction relationship candidates based on the edge type combination in the path. The system sets up candidate relationships that are of the same type and direction as the original relationships in the content semantic relationship graph and writes them into the consistent relationship set. The remaining candidate relationships are written into the conflict relationship set. The number of steps is limited based on the maximum parent-child nesting level and the allowed reference span in the candidate layout. The system does not cross areas marked as reading path breakpoints. When a tag node is mapped to the title area and the body area, the system reads the visual hierarchy edge and reading order path between them and generates master-slave prediction relationship and temporal prediction relationship respectively. When an image tag is mapped to the main image area and the caption tag is mapped to the corresponding caption area, the system reads the content subordinate edge and spatial adjacency edge and generates inclusion prediction relationship and referential prediction relationship.

[0039] Furthermore, the reverse reconstruction does not perform a fixed one-to-one conversion between layout relationships and semantic relationships. Instead, it determines the candidate set of predicted relationships based on the combination of edge types. For example, when there are both visual hierarchy edges and reading order edges between the title area and the body area, the generated candidate set includes master-slave predicted relationships and temporal predicted relationships. When there are both content subordinate edges and spatial adjacency edges between the main image area and the caption area, the generated candidate set includes inclusion predicted relationships and referential predicted relationships. When there are only reading order edges between body area A and body area B, only temporal predicted relationships are generated. When the original tag node pair is recorded as a causal relationship and the layout role node pair can only obtain a temporal predicted relationship in reverse, the system checks whether the original causal relationship is also accompanied by a temporal relationship edge. If there is a temporal base edge with the same direction, the temporal predicted relationship is written into the consistent relationship set and the missing causal correspondence is recorded as a relationship type difference. If there is no corresponding temporal edge, the predicted relationship is written into the conflict relationship set to distinguish between two conflict sources: the layout cannot directly express the complete causal type and the layout relationship direction is wrong.

[0040] In this embodiment, the conflict residual consists of relation type difference, relation direction reversal, hierarchy span, and reference interval. The relation type difference is: The amount of the relationship direction reversal is The number of levels spanned is , refers to the interval quantity The corresponding combination coefficients are respectively , , and The total number of conflicting residuals Determined according to the following formula: ; in, Calculated based on the number of labeled node pairs whose original relation type is inconsistent with the predicted relation type and the amount of relation evidence. Calculated based on the tag node pairs whose original relationship direction is opposite to the predicted relationship direction. Calculated based on the reverse span of tag nodes with master-slave or containment relationships in the layout visual hierarchy. The distance between the corresponding layout areas of tag nodes with referential relationships, exceeding the allowed reference span, is calculated. For example, when the main topic tag is mapped to the body area while the subordinate topic tag is mapped to the heading area, a reverse hierarchical span occurs. Addition: When the caption label and the image label being referred to are distributed in an area without dependent edges and exceeding the allowed adjacent span, Increased when the original causal direction is opposite to the reading order. The combination coefficients are increased to remain consistent across all mapping rounds of the same candidate version.

[0041] Please refer to Figure 7 , Figure 7 The horizontal axis represents the typesetting correction check number, ranging from 1 to 100, and the vertical axis represents the remaining conflict amount, ranging from approximately 0.000 to 0.449. The solid black line represents the remaining conflict residuals, the dashed gray line represents the review reference line, and the dotted gray line represents the blocking reference line. This graph corresponds to the synthetic check process of four types of residuals: relation type difference, relation direction reversal, hierarchy crossing, and referential interval. It is used to observe whether the system has completed the closed loop from conflict identification to local rematching and then to stable output.

[0042] from Figure 7 As can be seen, the curve initially reaches a maximum of 0.449, with 21 check numbers exceeding the blocking reference line, indicating that there were issues such as direction reversal, text-image distance, or hierarchical misalignment that needed to be addressed in the initial mapping. As local candidate deletion, conflict subgraph rearrangement, and boundary relationship verification were executed round by round, the curve rapidly decreased, falling below the verification reference line in the latter part and reaching 0.000 at the end. This change indicates that the original conflicts were gradually eliminated, and the final mapping combination met the layout consistency requirements before output.

[0043] Figure 7 This graph can be used to determine whether the layout results can be exported as the final product. If the curve remains above the blocking reference line for an extended period, it indicates a hard conflict such as a reversed causal order, an inverted hierarchy, or misplaced captions. In this case, publishing should be paused and the conflicting sub-graph expanded. If the curve drops to near the review reference line, the manual review entry can be retained to confirm the continuity of local text and images. When the curve approaches zero, as shown in the latter part of this graph, it indicates that the label relationship and the layout relationship have achieved a closed loop consistency. This graph provides a visual basis for determining whether local rematching has ended.

[0044] In one embodiment, please refer to Figure 4The execution of local rematching includes: establishing a conflict subgraph based on the tag nodes, layout role nodes, and relationship edges involved in the conflict residuals, and adding mapping nodes that have not entered the conflict subgraph and have no directional competition records with the conflict subgraph to the locking set; canceling the original mapping combination within the conflict subgraph, while retaining the layout role occupancy status corresponding to the locking set; regenerating a node candidate set from the layout role nodes that have not been occupied by the locking set, and converting the relationship type difference, relationship direction reversal, hierarchical crossing, and referential interval in the conflict residuals into candidate deletion constraints; updating the node-edge joint matching matrix according to the deletion constraints, selecting a new local mapping combination, and concatenating the new local mapping combination with the locking set before performing reverse relationship reconstruction again. The initial node set of the conflict subgraph consists of the tag nodes that directly generate the residuals. The system extends outward along the content semantic relationship graph to the adjacent nodes that have basic relationship edges with the tag nodes, and simultaneously adds the layout role nodes that carry the above tag nodes and the role relationship edges that are directly connected to the role nodes. The node corresponding to the candidate relationship edge only enters the conflict subgraph when the candidate relationship edge participates in the current reverse reconstruction.

[0045] In this embodiment, candidate deletion constraints corresponding to relation type differences are used to delete role node pairs that cannot form the required layout relation combination. Candidate deletion constraints corresponding to relation direction reversal are used to delete role node pairs whose reading order direction or visual hierarchy direction is opposite to the original relation direction. Candidate deletion constraints corresponding to hierarchy span are used to delete role combinations that make the subordinate topic higher than the main topic. Candidate deletion constraints corresponding to referential interval are used to delete role combinations that do not have a content subordinate edge between the image label and the caption label and exceed the allowed adjacency span. The deletion constraints act on the node candidate set and relation correspondence, without modifying the mapping nodes in the locked set. After a new local mapping combination is generated, the system restores the boundary relationship between the conflict subgraph and the locked set, and performs reverse reconstruction on the spliced ​​global mapping. If the original conflict residual is eliminated and no new boundary conflict is generated, the new local mapping combination is saved. If a new boundary conflict occurs, the corresponding boundary node is added to the set to be unlocked.

[0046] In a preferred embodiment, merging multiple pieces of evidence includes: converting syntactic dominance evidence, conjunction evidence, entity coreference evidence, and topic inclusion evidence into relational evidence items with source identifiers; merging relational evidence items according to tag node pairs and relational directions, retaining parallel relational edges when different relational types exist in the same tag node pair, and establishing a direction competition record when there is a direction conflict in the same relational type; generating an evidence consistency sequence based on the recurrence of each relational evidence item in adjacent semantic units and the consistency across content forms, and determining the main relational edge and candidate relational edges based on the evidence consistency sequence; writing the main relational edge into the basic edge set of the content semantic relation graph, and writing the candidate relational edges into the replaceable edge set, so that subsequent local rematching can perform constrained replacement of relational edges without changing the tag nodes. The data fields of the relational evidence items include source type, source position, relation start point, relation end point, relation category, direction marker, context range, and evidence status. When the same entity relation appears repeatedly in adjacent semantic units, the evidence consistency sequence records the recurrence position according to the original content order. Consistency across content forms refers to the independent support for the same tag node pair and the same relational direction in the text, captions, or titles.

[0047] Furthermore, the determination of the consistent evidence sequence adopts continuous support and conflict segmentation. When multiple consecutive evidence items support the same relational direction, they are grouped into a continuous support segment. When the supportal direction changes, a conflict segment is formed at the changed position. The system compares the source types, the number of semantic units covered, and the cross-content form of each continuous support segment. Continuous support segments with more source types and consistent cross-content form correspond to the main relation edge. Other continuous support segments with independent sources but insufficient support correspond to candidate relation edges. If two continuous support segments support opposite directions and both contain explicit connector evidence, a direction competition record is established and two candidate directions are retained. When the layout is reversed, the results are verified according to the reading order and master-slave hierarchy. If the candidate directions all conflict with the layout relationship, the original direction competition state is retained and the relevant nodes are added to the conflict subgraph to avoid forcibly rewriting the original semantic relationship based solely on the layout relationship.

[0048] In a preferred embodiment, when candidate relation edges in the replaceable edge set participate in reverse relation reconstruction, an edge replacement pairing table corresponding to the predicted relation candidate set is established for each candidate relation edge. When the number of conflicting relations formed by the main relation edge and the layout relation is greater than the number of conflicting relations formed by the candidate relation edge and the layout relation, the candidate relation edge temporarily replaces the corresponding main relation edge, and the source identifier of the replaced main relation edge is retained. A consistent relation set and a conflict relation set are regenerated for the content semantic relation graph after the edge replacement is completed. When a new tag node pair conflicts or breaks the original consistent relation after the replacement, the edge replacement is revoked. When no new conflict is added after the replacement, the corresponding candidate relation edge is transferred to the basic edge set, and the updated basic edge set is used for the next round of node-edge joint mapping and reverse relation reconstruction. The edge replacement pairing table at least stores the main relation edge identifier, candidate relation edge identifier, predicted relation identifier, original number of conflicts, number of conflicts after replacement, newly added conflicting node pairs, and revocation status. Candidate relation edges can only replace main relation edges that connect the same tag node pairs, and it is not allowed to change tag nodes or content objects through edge replacement.

[0049] In this embodiment, whether to retain an edge replacement is determined by a replacement evaluation metric, and a label node is set. With tag nodes The primary relation is of type t, the candidate relation is of type s, and the conflict rate when the primary relation participates in the mapping is: The number of conflicts after the candidate relation participates in the mapping is The amount of consistency relationship disruption after replacement is The damage coefficient is Then replace the evaluation quantity. Determined according to the following formula: ; in, and All calculations are based on differences in relation type, direction reversal, hierarchical crossing, and referential interval records within the conflict relation set. The sum of the number of relations and the corresponding amount of evidence transferred from the original consistent relation set to the conflict relation set after replacement. This is used to restrict substitution behaviors that reduce local conflicts by disrupting existing consistency relationships. For example, when a candidate temporal relationship can eliminate a reading order conflict but causes the original containment relationship to lose its layout subordination, it is appropriate to use such a relationship. If the replacement evaluation value is not greater than 0, the system will cancel the replacement. If the candidate relationship can eliminate directional conflicts and does not destroy the original consistent relationship, the replacement evaluation value will be greater than 0. The candidate relationship edge will be transferred to the basic edge set and the source record of the main relationship edge will be retained for restoration when the content is modified later.

[0050] Please refer to Figure 8 , Figure 8The horizontal axis represents the candidate relation edge number, ranging from 1 to 90, and the vertical axis represents the net benefit of replacement, ranging from approximately 2.742 to 3.276. The solid black line represents the net benefit after replacing the original principal relation edge with a candidate relation edge, the dashed gray line represents the retained boundary, and the dotted gray line represents the high-confidence boundary. This figure corresponds to the relation edge replacement evaluation process, and the curve values ​​are jointly determined by the original conflict amount, the conflict amount after replacement, and the new damage introduced by the replacement.

[0051] from Figure 8 As can be seen, all 90 candidate edge numbers are higher than the retention boundary and the high confidence boundary, with an average net gain of 3.002. The highest value of 3.276 appears at the 12th candidate edge number. The curve is generally above 2.7, indicating that these candidate edges make a stable contribution to reducing conflict residuals without significantly disrupting the original consistent relationships. Small local fluctuations indicate that different candidate edges improve the degree of improvement in caption reference, text causality, and topic inclusion relationships to varying degrees, but no negative gain intervals requiring reversal or replacement have been observed.

[0052] This diagram illustrates that the handling of replaceable relation edges is not simply about deleting the original relation or blindly adopting a new one, but rather about whether it truly reduces layout conflicts. Candidate edges below the zero line should be retained as the original relation and not participate in this round of local rearrangement; candidate edges above the high confidence boundary can be transferred to the basic edge set, and the source record of the replaced main relation should be retained for easy restoration when content is modified. Figure 8 The fact that the entire segment maintains high returns indicates that the candidate temporal edges, referential edges, and content subordinate edges in this embodiment can effectively correct the reading order and text-image binding issues.

[0053] In one embodiment, the establishment of the lock set adopts relational isolation judgment. Mapping nodes that have not entered the conflict subgraph are added to the lock set only if there are no directional competition records, candidate replacement edges, and unresolved composite paths between them and the conflict subgraph. The label nodes, role nodes, and mapping relationships in the lock set remain unchanged in the current local rematching round. The occupancy status of the locked role nodes is deleted from the local candidate space, but the basic relationship edges between the locked nodes and the conflict subgraph are still retained as boundary constraints. Local mapping combinations must satisfy the boundary relationship direction. When there is a causal basic edge between the cause label in the conflict subgraph and the result label in the lock set, the optional role in the conflict subgraph must be located at the reading path position before the locked result role. When the caption label in the conflict subgraph points to the image label in the lock set, the local candidate only retains the caption roles that have a content subordinate relationship with the image role or allow reference adjacency.

[0054] In a preferred embodiment, after re-performing the reverse relationship reconstruction, the process further includes: comparing the boundaries of the conflict subgraphs before and after rematching, identifying boundary relationship edges that still contain conflict residuals and are connected to the locked set; transferring the corresponding locked nodes to the unlocked set according to the direction of the boundary relationship edges, and determining the expansion order based on the number of relationship types between the unlocked nodes and the conflict subgraphs, the reading path span, and the hierarchical connection depth; releasing one or a group of mapping nodes with common relationship edges from the unlocked set each time, merging the released mapping nodes into the conflict subgraphs, and updating the layout role occupancy status; re-performing node candidate generation, candidate deletion, local mapping, and reverse relationship reconstruction after each expansion until the conflict residuals are empty, or when the conflict subgraphs cover all mapping nodes, switching to full-graph joint mapping. Boundary relationship edges refer to relationship edges where the starting node and the ending node are located in the conflict subgraph and the locked set, respectively. "Still containing conflict residuals" means that the predicted relationships corresponding to the boundary relationship edges after local rematching still have type differences, direction reversals, hierarchical crossings, or referential intervals.

[0055] In this embodiment, the expansion order of the nodes to be unlocked is determined by the boundary conflict density, the number of relation types, the hierarchical connection depth, and the node stability. Let the node to be unlocked be z, and the number of boundary conflicts connected to node z is... The number of relationship types between the conflict subgraph and the conflict subgraph is The hierarchical connection depth of the node in the role topology is The number of nodes that maintain the same role in each mapping is a stable quantity. The corresponding coefficient is , , and Then the node expansion priority Determined according to the following formula: ; in, A larger value indicates that there are more unresolved boundary relationships between the node and the current conflicting subgraph. The larger the value, the more likely the node can handle multiple semantic relationships simultaneously. The larger the value, the deeper the node is located in the layout hierarchy. A larger value indicates that the node has remained stable in previous mapping rounds and should not be released prematurely. For example, when a caption node has boundary conflicts with both the main graph node and the text reference node. and If the larger value is chosen, and its stability is low, then the expansion priority is higher, and the system prioritizes releasing the caption node. If the title node has only one weakly associated boundary relationship and maintains its title role in all mappings, then... The value is large, so the system delays releasing the title node.

[0056] Please refer to Figure 9 , Figure 9 The horizontal axis represents the sequence number of the semantic node to be processed, ranging from 1 to 90, while the vertical axis represents the release priority strength, ranging from approximately -3.035 to 8.174. The solid black line represents the release priority of different nodes to be unlocked, the dashed gray line represents the low-priority reference line, and the dotted gray line represents the high-priority reference line. This diagram corresponds to the node release priority calculation process, with the curve formed by the number of boundary conflicts, the number of relation types, the hierarchical connection depth, and the node stability.

[0057] from Figure 9 As can be seen, the curve exhibits a clear stratification, with 60 nodes above the high-priority reference line, the highest value of 8.174 appearing at the 11th node; simultaneously, there is a group of nodes below the low-priority reference line, the lowest value being -3.035, appearing at the 36th node. High-priority nodes typically correspond to semantic units that simultaneously connect causal relationships within the text, image references, or thematic inclusion boundaries, and releasing them can affect multiple types of conflicts at once; low-priority nodes are mostly stable nodes such as titles and main images, and releasing them prematurely may disrupt the page's structure.

[0058] The overall trend indicates that this method employs an orderly release strategy when expanding conflict subgraphs, rather than revoking all locked nodes at once. When high-priority nodes appear consecutively, the system should first release the layout slots associated with them, and then re-examine the text-image references and paragraph connections; keeping low-priority nodes locked can maintain the stability of the title hierarchy, main image position, and basic reading path. Figure 9 This demonstrates why the local rematching process can eliminate conflicts without causing widespread jitter across the entire page, providing a basis for controllable corrections in actual new media layout.

[0059] In a specific application embodiment, the content to be formatted includes a topic title, an event cause paragraph, an event result paragraph, a main image, a main image caption, supplementary images, and supplementary explanatory text. The system divides the topic title into a title semantic unit, divides the cause paragraph and the result paragraph into body text semantic units, and establishes independent semantic units for the main image, caption, supplementary images, and supplementary explanatory text. It extracts the main topic tag and central entity tag from the topic title, the cause action tag and the affected entity tag from the cause paragraph, the result action tag and the result object tag from the result paragraph, the image object tag and the referential tag from the main image and its caption, and the subordinate topic tag from the supplementary explanatory text. In the content semantic relationship graph, it establishes an inclusion relationship edge from the main topic tag to the subordinate topic tag, a causal relationship edge and a temporal relationship edge from the cause action tag to the result action tag, and a referential relationship edge from the caption referential tag to the main image object tag. It also generates a corresponding evidence consistency sequence based on repeated evidence in the title, body text, and image / text description.

[0060] The candidate layout includes a title area, an introduction area, a main image area, a main image caption area, two body text areas, an auxiliary image area, an auxiliary image caption area, and an emphasis area. The role constraint diagram establishes visual hierarchy edges (title area above body text area), reading order edges (title area before body text area), reading order edges (cause body text area before result body text area), content subordination edges and spatial adjacency edges (main image area to main image caption area), and content subordination edges (auxiliary image area to auxiliary image caption area). The initial joint mapping maps the main topic tag and title content to the title area, the cause paragraph to the first body text area, the result paragraph to the second body text area, the main image and caption to the main image area and main image caption area, and the supplementary image and supplementary explanation to the auxiliary image area and auxiliary image caption area. Reverse reconstruction yields the master-slave prediction relationship between the title area and body text area, the temporal prediction relationship between the two body text areas, and the referential prediction relationship between the main image area and main image caption area. Comparison with the original relationship diagram shows no direction reversal or hierarchy jump. The system locks the above mappings and outputs the layout result.

[0061] In another specific scenario, the initial joint mapping assigns the result paragraph to the first text area and the cause paragraph to the second text area due to the high compatibility of local nodes. Both text areas can carry continuous text and the node content forms are compatible. However, the reading time sequence prediction relationship obtained by the reverse reconstruction, which points from the result paragraph to the cause paragraph, is opposite to the causal and time sequence relationship of the original cause action label pointing to the result action label. The system records the conflict residual in the relation direction reversal component and establishes the cause action label, the result action label, the two text role nodes and their reading order edges as a conflict subgraph. The corresponding mappings of the title area, main figure area, figure caption area and auxiliary figure area are added to the locking set. Local candidate deletion constraints exclude role combinations that place the cause paragraph after the result paragraph. The updated node-edge joint matching matrix assigns the cause paragraph to the first text area and the result paragraph to the second text area. After splicing the locking set, the reverse reconstruction is re-executed. The original time sequence direction is consistent with the predicted reading sequence direction, and the conflict residual is eliminated.

[0062] In an extended scenario, the reference in the main image caption may point to both the main image and the supplementary image. Entity co-reference evidence supports the main image object, while neighboring word evidence supports the supplementary image object. The content semantic relationship graph retains two candidate reference edges and establishes a directional competition record. The initial mapping incorrectly assigns the caption to the auxiliary caption area. Reverse layout reconstruction enables the caption reference label to form a reference prediction relationship with the supplementary image object label, while the original main relationship edge points to the main image object label. The system establishes an edge replacement pairing table with the two candidate reference edges and the prediction relationship candidate set respectively. When replacing with the supplementary image reference edge would disrupt the existing consistent relationship with the main image object in the text, the new consistency relationship disruption amount ensures that the edge replacement evaluation amount is not greater than 0. The system cancels the edge replacement and includes the caption node, main image node, supplementary image node, and corresponding layout role in the conflict subgraph. Local rematching restores the caption to the main caption area. The content subordinate edge and spatial adjacency edge between the main image area and the main caption area generate a reference prediction relationship consistent with the original main relationship edge.

[0063] In one embodiment, a mapping traceability record is saved before the typesetting result is output. The mapping traceability record includes the content semantic relationship graph version, the layout role constraint graph branch identifier, the node candidate set, the deleted candidate and the deletion basis, the initial mapping combination, the reverse prediction relationship, the consistent relationship set, the conflict relationship set, the conflict subgraph, the locking set, the edge replacement record, and the final mapping combination. When the content to be typed is partially modified, the system locates the affected semantic unit according to the content identifier and character position, re-extracts the corresponding label and updates the relationship edge connected to the label. For the unaffected label node and layout role node, the previous mapping traceability record is read as the initial locking candidate, and then the release is determined according to the new relationship evidence and the reverse reconstruction result. This allows the typesetting process after the content modification to continue to use the same bidirectional relationship consistency mechanism, instead of directly replacing the semantic relationship verification with the original layout position.

[0064] In a preferred embodiment, the system performs node candidate generation, node-edge joint mapping, reverse relationship reconstruction, and conflict residual calculation on multiple candidate layout branches. For layout branches that can eliminate conflict residuals through local rematching, the system saves their final mapping combination and the number of layout roles used. For layout branches that still have reversed relationship directions or irremovable referential intervals after the conflict subgraph is expanded to all mapping nodes, the system marks them as incompatible branches and stops content filling. Finally, from the layout branches without conflict residuals, the system selects branches with a large number of basic relationship edges retained, a small number of candidate relationship edge replacements, and a small number of empty layout roles. After selection, the system generates content writing order according to the reading path sequence number of the role nodes and writes the corresponding content objects into the title area, introduction area, main image area, auxiliary image area, body text area, quotation area, or emphasis area, while maintaining the parent-child identifiers and reference relationships between the captions and the corresponding image areas.

[0065] In this embodiment, after the content is written into the layout area, the tag nodes and relationship edges are not changed again. Text truncation is only allowed to occur at the boundaries of semantic units that are pre-marked as divisible. Images and captions must not be split between areas where there is no content subordination relationship. Text semantic units with causal and temporal relationships are written sequentially according to the reading order in the final mapping. Main topics and subordinate topics with inclusion relationships are written into the title area, introduction area, or body area according to the final visual hierarchy. Caption content with referential relationships is written into the caption area corresponding to the image being referred to. When the target layout area cannot accommodate complete and indivisible semantic units, the system returns to the corresponding layout branch to regenerate the role code. The system does not force the area to be filled by deleting relationship edges, changing relationship directions, or splitting references, ensuring that the final layout content, tag relationships, and layout role relationships are consistent with the data used for reverse reconstruction verification.

Claims

1. A new media layout method based on semantic tag matching, characterized in that, include: The new media content to be typed is divided into semantic units, and entity tags, topic tags, action tags, emotion tags and content form tags of each semantic unit are extracted. Based on the modification, causal, referential, temporal and inclusion relationships between tags, a content semantic relationship diagram is constructed. Obtain the layout areas in the candidate layouts and the visual hierarchy, spatial adjacency, reading order and content hierarchy between the areas, and construct a layout role constraint diagram; Based on the compatible mapping between semantic dependency type and layout role relationship, the nodes and edges in the content semantic relationship graph are jointly mapped to the layout role constraint graph to obtain the initial layout matching result; Based on the initial layout matching results, the tag relationships are reconstructed in reverse, and the reconstructed tag relationships are compared with the original tag relationships to generate conflict residuals. Lock the mapping nodes that have not generated the conflict residuals, perform local rematching on the mapping nodes that have generated the conflict residuals and their associated nodes, and output a layout result that satisfies the bidirectional relationship consistency.

2. The new media layout method based on semantic tag matching according to claim 1, characterized in that, Constructing the content semantic relationship graph includes: dividing continuous text, titles, captions, quoted text, and accompanying image descriptions into semantic units with independent referential objects; For each semantic unit, a set of tag nodes is established, and modification relationship edges are determined according to the syntactic dominance direction within the same semantic unit. Temporal relationship edges and causal relationship edges are determined according to the sequence of events and conjunctions across semantic units. Referential relationship edges are determined according to entity coreference chains. Inclusion relationship edges are determined according to the degree of topic inclusion between the semantic units. Record the relationship direction, relationship type, evidence source, and evidence consistency for each relationship edge, and merge the evidence when multiple pieces of evidence point to the same labeled relationship to form a directed multi-relation graph for joint mapping, where the same labeled node can be connected to multiple associated labeled nodes through different relationship types.

3. The new media layout method based on semantic tag matching according to claim 1, characterized in that, Constructing the layout role constraint diagram includes: establishing layout role nodes for the title area, lead area, main image area, auxiliary image area, body text area, quotation area and emphasis area in the candidate layout; Spatial adjacency edges are generated based on the relative position of the layout area in the page coordinate system; visual hierarchy edges are generated based on font size level, area area ratio and salience order; reading order edges are generated based on the preset reading path and the sequential connection relationship of the areas; and content subordination edges are generated based on the parent and child container relationship between the areas. The content format that each layout role node can carry, the semantic relationship types that are allowed to be connected, and the acceptable relationship directions are written into the node constraints, and the node constraints and various layout relationship edges are jointly encoded into the role topology structure of the candidate layout.

4. The new media layout method based on semantic tag matching according to claim 1, characterized in that, The compatibility mapping and the conflict residual generation include: generating a set of node candidates corresponding to each layout role node for each tag node in the content semantic relationship graph, and deleting incompatible candidates based on the matching relationship between the tag type and the content form that the layout role can carry; For the retained node candidates, the correlation degree between semantic relationship edges and visual hierarchy edges, spatial adjacency edges, reading order edges and content subordination edges is jointly calculated to form a node-edge joint matching matrix; Select a mapping combination from the node-edge joint matching matrix that satisfies the unique node occupancy constraint and the relationship direction constraint; Based on the layout relationship in the mapping combination, the predicted relationship between tags is generated in reverse, and the residual components are recorded according to the relationship type difference, relationship direction reversal, hierarchical crossing and reference interval, and the conflict residual is obtained by combination.

5. The new media layout method based on semantic tag matching according to claim 2, characterized in that, Multiple pieces of evidence were merged, including converting syntactic dominance evidence, conjunction evidence, entity coreference evidence, and topic inclusion evidence into relational evidence items with source identifiers. Relation evidence items are merged according to the label node pairs and relation directions. When different relation types exist in the same label node pair, parallel relation edges are retained. When there are directional conflicts in the same relation type, a direction competition record is established. Based on the repetition of each relation evidence item in adjacent semantic units and the consistency across content forms, an evidence consistency sequence is generated, and the main relation edge and candidate relation edge are determined based on the evidence consistency sequence. The primary relation edge is written into the basic edge set of the content semantic relation graph, and the candidate relation edge is written into the replaceable edge set, so that subsequent local rematching can perform constrained replacement of relation edges without changing the label node.

6. The new media layout method based on semantic tag matching according to claim 3, characterized in that, The encoding of the character topology includes: establishing a node description vector for each layout character node, consisting of a character identifier, a visual hierarchy number, a reading path number, a region parent-child identifier, a content form set, and a set of connectable relationships; For each layout relation edge, an edge description vector is created, consisting of the starting role, ending role, edge type, directional constraint, and cross-region span. The node description vectors are sorted according to the reading path sequence number, and the edge description vectors corresponding to adjacent nodes and non-adjacent reference nodes are embedded into the sorting result to form a sequential-topological joint encoding. When the candidate layout contains optional or merged regions, branch codes are generated for different region enable states, and the node retention relationship in each branch code is constrained according to the region parent-child identifier, resulting in multiple layout role code branches that can be called by the node-edge joint matching matrix.

7. The new media layout method based on semantic tag matching according to claim 4, characterized in that, The reverse generation of predicted relationships between tags includes: reading the layout role nodes that carry each tag node in the mapping combination, and retrieving the direct relationship edges and composite relationship paths between the layout role nodes along the layout role constraint graph; The visual hierarchy edges are converted into label master-slave prediction relationships, the reading order edges are converted into label temporal prediction relationships, the content subordinate edges are converted into label inclusion prediction relationships, and the spatial adjacency edges between the main image area or auxiliary image area and the corresponding caption area are converted into label reference prediction relationships. When the same label node corresponds to multiple composite relationship paths, a candidate set of predicted relationships is established based on the combination of edge types in the path. Candidate relationships with the same type and direction as the original relationship in the content semantic relationship graph are written into the consistent relationship set, and the remaining candidate relationships are written into the conflict relationship set.

8. The new media layout method based on semantic tag matching according to claim 1, characterized in that, Performing the local rematch includes: establishing a conflict subgraph based on the tag nodes, layout role nodes and relationship edges involved in the conflict residual, and adding the mapping nodes that have not entered the conflict subgraph and have no directional competition records with the conflict subgraph to the lock set; Within the conflict subgraph, cancel the original mapping combination while retaining the layout role occupancy status corresponding to the locked set; A new node candidate set is generated from the layout role nodes that have not been occupied by the locked set, and the relationship type difference, relationship direction reversal, hierarchy crossing and reference interval in the conflict residual are respectively converted into candidate deletion constraints. The node-edge joint matching matrix is ​​updated according to the deletion constraint, a new local mapping combination is selected, and the new local mapping combination is concatenated with the locked set before the reverse relation reconstruction is performed again.

9. The new media layout method based on semantic tag matching according to claim 1, characterized in that, Also includes: When candidate relation edges in the replaceable edge set participate in reverse relation reconstruction, an edge replacement pairing table corresponding to the predicted relation candidate set is established for each candidate relation edge. When the number of conflicting relationships between the main relationship edge and the layout relationship is greater than the number of conflicting relationships between the candidate relationship edge and the layout relationship, the candidate relationship edge is temporarily replaced with the corresponding main relationship edge, and the source identifier of the replaced main relationship edge is retained. Regenerate consistent and conflicting relation sets for the semantic relation graph after edge replacement; When a new label node pair conflicts or disrupts the original consistent relationship after replacement, the edge replacement is revoked. When no new conflict is added after replacement, the corresponding candidate relationship edge is transferred to the basic edge set, and the updated basic edge set is used for the next round of node-edge joint mapping and reverse relationship reconstruction.

10. The new media layout method based on semantic tag matching according to claim 1, characterized in that, After performing the reverse relation reconstruction again, it also includes: comparing the boundaries of the conflicting subgraphs before and after the rematch, and identifying the boundary relation edges that still contain conflicting residuals and are connected to the locked set; According to the direction of the boundary relationship edge, the corresponding locked node is transferred to the set to be unlocked, and the expansion order is determined based on the number of relationship types between the node to be unlocked and the conflict subgraph, the reading path span, and the hierarchical connection depth. Each time a mapping node or a group of mapping nodes with common relationship edges is released from the set to be unlocked, the released mapping node is merged into the conflict subgraph and the layout role occupancy status is updated. After each expansion, node candidate generation, candidate deletion, local mapping, and reverse relationship reconstruction are re-executed until the conflict residual is empty, or the conflict subgraph covers all mapped nodes, at which point the process transitions to full graph joint mapping.

Images