A method and system for visual layout of temporal event triples based on multi-objective optimization

Abstract

A kind of method and system for visual layout of time sequence event triple based on multi-objective optimization, the method of which comprises the following steps: (1) processing original text data, extracting SPO triple from original text data using HanLP technology to obtain structured data; (2) summarizing two optimization objectives according to the design requirements of time sequence event visualization, and constructing a multi-objective optimization model; (3) defining constraint conditions for the multi-objective optimization model; (4) implementing the multi-objective optimization model using a python program, and obtaining the optimal solution of the multi-objective optimization model by exploring the solution space using Gurobi.The present application increases the quantity and quality of information expression in story line visualization, and provides a clearer and more beautiful visual layout.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of time series event visualization, in particular a time series event triplets visual layout method and system based on multi-objective optimization. BACKGROUND

[0002] In text data such as novels, movie summaries, etc., there are stories composed of one or more events. A story is a collection of events that are related in time or causally. How to effectively show the time series events and the relationship between characters has always been a research hotspot in the field of visualization. Visualizing the triplets in time series events plays an important role in story understanding, character relationship analysis, character development, plot evolution and user engagement. It converts complex time series event information into intuitive and easy-to-understand forms through visualization, helping users better understand and analyze stories and gain deeper insights and emotional resonance.

[0003] Some existing visualization techniques perform well in specific analysis tasks, such as the evolution of plot or theme (flowchart), the connection between events or characters (node-link diagram), and the spatiotemporal information of story locations (scatter plot). However, these techniques often fail to provide comprehensive information.

[0004] Storyline visualization is one of the commonly used visualization techniques in time series visualization. Given the effectiveness and intuitiveness of storyline visualization in movie summaries, this visualization technique has also been applied to other data and fields, such as software evolution, meeting minutes, and group collaboration. The main techniques involved include data preprocessing, entity recognition, relationship extraction, text representation, visualization design and layout, etc., among which visualization layout optimization is a key point in storyline visualization. The common approach is to improve the position, size and shape of elements in the graph according to certain principles. In storyline visualization layout, the common design requirements are: (1) lines of the same set (relationship) should be close to each other; (2) lines need to be kept until the set (relationship) changes.

[0005] In many storyline visualization designs, the following design principles are used, i.e., from left to right representing the progress of time series events and using lines to encode entities. Although users can understand some behavior patterns of entities in the entire time series event through the aggregation and dispersion of lines, they are limited to relatively low-level behavior patterns such as "whether an entity participates in the same scene or location". This means that users cannot know the specific behavior between entities in a specific situation, such as in a certain event or at a certain time. Therefore, when showing the general trend of event sequences, the relationship between entities should also be considered.

[0006] Temporal event visualization is a time-based visualization technique that illustrates how events develop sequentially over time. Time axes for temporal event visualization have various forms of representation, such as straight lines, spirals, and concentric circles. For example, a time axis is represented by a straight line, and events are displayed on the timeline according to their occurrence time; a concentric circle ring of a circular time axis is used to visualize scenes and characters in a movie; a graph structure overview can help researchers grasp important nodes when analyzing temporal events, and so on.

[0007] Among various visualization techniques, story line visualization is widely praised for its effectiveness and practicality in providing an overview of story trends, revealing entity relationships, and simplifying visual communication. However, existing story line visualization methods cannot display specific relationships between entities, which can cause users to have difficulty understanding the plot and character relationships. This limitation can cause users to miss important interactions and conflicts in temporal events, as well as complex associations between characters. SUMMARY

[0008] The present invention overcomes the above-mentioned shortcomings of the prior art and provides a temporal event triple visual layout method and system based on multi-objective optimization for displaying specific relationships between entities to users while visualizing temporal events, and fully considering the simplicity and aesthetics of the layout.

[0009] The innovation of the present invention is to propose a temporal event triple visual layout method based on multi-objective optimization, which extracts entity relationships from text data using HanLP technology to form subject-predicate-object (SPO) triples, obtains structured data, then sets optimization goals according to design requirements, and models the layout problem with multi-objective optimization. Finally, the extracted SPO triples, time information, and event information are input into the optimization model to ensure that the visual layout is simple and easy to understand.

[0010] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0011] A temporal event triple visual layout method based on multi-objective optimization, the method comprising the following steps:

[0012] Step 1, extracting triplets. First, the original text data is cleaned, and then the triplets of <subject-predicate-object> (SPO) structure are extracted. HanLp technology is used to analyze the dependency relationship and semantic dependency of the sentence, and convert it into a dependency relationship tree. Among them, the dependency syntax tree identifies the syntactic components, and the semantic dependency tree identifies the semantic relationship. Nodes that meet the subject-predicate relationship (SBV) and the verb-object relationship (VOB) are selected from the dependency syntax tree, and nodes that get the semantic relationship type of "agent" or "patient" from the semantic dependency tree. The nodes obtained are compared, and if the nodes from the two syntax trees are the same, they are combined into SPO triplets.

[0013] Step 2, constructing a multi-objective optimization model. According to the design requirements proposed, two optimization objectives are summarized, namely reducing line intersection points and keeping related entities in the same event as close to each other as possible. The above two optimization objectives are expressed in mathematics, and a multi-objective optimization model is constructed.

[0014] The design requirements in step 2 specifically include the following points:

[0015] (1) Represent an entity with a line that extends from left to right over time.

[0016] (2) Use the aggregation and divergence of lines to represent whether an entity participates in the same event.

[0017] (3) Reduce the number of line intersections. Fewer line intersections can reduce the complexity of the view.

[0018] (4) Reduce the number of line swings. Excessive or unnecessary line swings not only make the line appear discontinuous in vision, but also increase the number of line intersections.

[0019] (5) Represent entities participating in the same event in an intuitive way.

[0020] (6) Keep entities in the same SPO triplet in the event as close as possible.

[0021] Step 2.1, optimization objective one: reducing line intersection points. First, define two entities i and j, y i，t represents the vertical coordinate of entity i at time t, y j，t represents the vertical coordinate of entity j at time t. If the two entities intersect between t and t+1, then (y i，t -y j，t ) (y i，t+1 -y j，t+1 ) < 0. To minimize the number of intersection points in the story visualization, the optimization objective one can be expressed as:

[0022]

[0023] To simplify the formula (1), 0-1 decision variable p i，j，t and o i，t，h are introduced. i，j，t p i，t =1 if y j，t < y i，j，t , otherwise p i，j，t =0. o i，t，h is used to determine the position of entity i in the queue at time t. If the position of entity i is h, then o i，t =1 if (y j，t -y i，t+1 )(y j，t+1 -y i，t，h ) < 0, otherwise o i，t，h =0, where h∈[1, H] and H represents the length of the queue. i，t，h The relationship between p i，t and o i，e can be described by the following formula.

[0024]

[0025] Therefore, the formula (1) can be simplified as:

[0026]

[0027] Step 2.2, optimization goal two: related entities in the same event are as close to each other as possible. This optimization goal can be described by time decomposition, that is, minimizing the difference in y-axis coordinates between two entities i and j. Formula (2) describes the y-axis coordinate of entity i at time t, so the difference in y-axis coordinates between two entities can be calculated by the following formula.

[0028]

[0029] When solving optimization goal two, two points need to be noted: first, entities i and j participate in event e at time t, and second, entities i and j have a relationship, so two 0-1 constants, C i，e and B e，t , are introduced. C i，e indicates that entity i participates in event e, and C i，e =1 if entity i participates in event e. B e，t indicates that event e occurs at time t, and B e，t =1 if event e occurs at time t, so optimization goal two can be represented as follows.

[0030]

[0031] Step 2.3, Multi-objective optimization model. The optimization objective "reduce line crossing points" described in step 2.1 and the optimization objective "as close as possible to each other for associated entities in the same event" described in step 2.2 constitute a multi-objective optimization model. The above two objective functions are converted by using weighted linear combination as follows.

[0032]

[0033] Because reducing line crossing is more important, here the parameters are set as a = 1, b = 0.1.

[0034] Step 3, Set constraints. First, ensure that there is a difference in position between entities i and j at time t, and exclude the case of taking only one entity, which can be converted into an "assignment problem". Second, to meet the requirement in step 2 that associated entities are as close to each other as possible in the same event, the position of the entity needs to be limited and it is ensured that there is no mixing of unrelated entities in the entity group participating in the same event, which can be converted into a "sliding window problem".

[0035] Step 3.1, Set constraint one: ensure that there is a difference in position between entities i and j at the same time t, and exclude the case of taking only one entity. For the problem of confirming the position of entity i in the queue at time t, it can be converted into an "assignment problem", assuming that there are H boxes that make up the queue, and at time t, entity i must and can only enter one box h, which can be expressed as the following formula:

[0036]

[0037]

[0038] wherein formula (7-1) ensures that there is and only one entity i at any time t and position h; formula (7-2) ensures that there is and only one position h for any entity i at time t.

[0039] The relationship between o i，t，h and p i，j，t can be expressed as the following formula:

[0040]

[0041]

[0042]

[0043] Through formula (8-2) and formula (8-3), it can be ensured that different entities i and j are located at different positions at time t, and the case of taking only one entity is excluded.

[0044] Step 3.2, setting constraint two: to achieve the optimization goal two in step 2.2, while determining the entity position, additional constraints are needed for the entities involved in the same event e. During the event e, the y-axis coordinate of the entity remains unchanged, which is expressed as follows:

[0045]

[0046] where T e represents the duration of event e.

[0047] In addition, restrictions need to be added in the y-axis direction to avoid mixing irrelevant entities into the entity group participating in the same event e. Here, the entity group is compared to a window with a length of H e , and the position adjustment of the entity group is considered as the sliding of the window, that is, the entity group is bound to move together. During the window sliding process, there are the following situations, that is, the starting point of the window is the starting point of the entity queue; the window is in the entity queue; the end point of the window is the end point of the entity queue. The sliding window can be described by formula (10), where d e，t，h is a 0-1 decision variable used to determine the starting point of the window composed of entities participating in event e at time t.

[0048]

[0049]

[0050]

[0051]

[0052] Formula (10-1) ensures that there is a starting point of the window when h∈[1,H-H e +1]; formula (10-2) ensures that there is no head of the window when h∈[H-H e +2,H]. Through formula (10-1) and formula (10-2), it can be ensured that when h∈[1,H], there is only one point as the starting point of the window. Formula (10-3) can ensure that the entity i participating in the event e cannot be arranged in front of the starting point h, and formula (10-4) can ensure that all entities participating in the event e are within the window.

[0053] Step 4, algorithm implementation. A python program is used to implement the multi-objective optimization model, and the optimal solution of the multi-objective optimization model is obtained by exploring the solution space through Gurobi. The minimum running time of the program is set to 30 minutes, and two conditions for terminating the solving program are set. The two conditions for terminating the solving program are that the relative gap between the best known solution and the best known bound is less than a given threshold and the set time frame is 12 hours. The experimental results are finally obtained according to the above conditions.

[0054] The second aspect of the application relates to a multi-objective optimization-based temporal event triple visual layout system, comprising:

[0055] A triple extraction module is used to first clean the original text data, and then extract triples of <subject-predicate-object> (SPO) structure. HanLP technology is used to analyze the dependency relationship and semantic dependency of the sentence, and convert it into a dependency relationship tree. The dependency syntax tree identifies the syntactic components, and the semantic dependency tree identifies the semantic relationship. Nodes that meet the subject-predicate relationship (SBV) and the verb-object relationship (VOB) are selected from the dependency syntax tree, and nodes that meet the semantic relationship type "agent" or "patient" are obtained from the semantic dependency tree. The obtained nodes are compared, and if the nodes from the two syntax trees are the same, they are combined into SPO triples.

[0056] A multi-objective optimization model construction module is used to summarize two optimization objectives according to the proposed design requirements, which are to reduce line intersections and to keep related entities in the same event as close to each other as possible. The two optimization objectives are expressed mathematically, and a multi-objective optimization model is constructed.

[0057] A constraint setting module is used to first ensure that at time t, there is a difference in the position of entities i and j, and to exclude the case of taking only one entity. This constraint can be converted into a "assignment problem". Second, to meet the requirement of the multi-objective optimization model construction module that related entities in the same event should be as close to each other as possible, the position of the entity needs to be limited and it is ensured that there is no unrelated entity mixed in the entity group participating in the same event. This can be converted into a "sliding window problem".

[0058] An algorithm implementation module uses a python program to implement the multi-objective optimization model, and obtains the optimal solution of the multi-objective optimization model by exploring the solution space through Gurobi. The minimum running time of the program is set to 30 minutes, and two conditions for terminating the solving program are set. The two conditions for terminating the solving program are that the relative gap between the best known solution and the best known bound is less than a given threshold and the set time frame is 12 hours. The results are finally obtained according to the above conditions.

[0059] The third aspect of the present application relates to a multi-objective optimization-based visual layout device of time sequence event triplets, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the multi-objective optimization-based visual layout method of time sequence event triplets.

[0060] The fourth aspect of the present application relates to a computer-readable storage medium, which stores a program, and the program is executed by a processor to implement the multi-objective optimization-based visual layout method of time sequence event triplets.

[0061] The fifth aspect of the present application relates to a computing device, comprising a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the multi-objective optimization-based visual layout method of time sequence event triplets.

[0062] The technical concept of the present application is that the present application provides a multi-objective optimization-based visual layout method of time sequence event triplets, aiming to solve the problem that the existing story line visualization method cannot display the specific relationship between entities. The method comprises extracting triplets, setting optimization objectives according to design requirements and modeling the layout problem with multi-objective optimization, setting constraint conditions for the multi-objective optimization model, and finally solving the model to obtain the result. The innovation of the present application is that the multi-objective optimization-based visual layout method of time sequence event triplets is proposed, so that the user can deeply understand the interaction between time sequence events and entities, and better grasp the internal logic and plot development of time sequence events.

[0063] The advantages of the present application are: 1) increasing the quantity and quality of information expression in story line visualization: the present application proposes a multi-objective optimization-based visual layout method of time sequence event triplets, which can display the relationship between events and entities and entities in plot visualization; 2) clearer and more beautiful visual layout: the present application proposes a multi-objective optimization model, which sets constraint conditions in combination with various design requirements to create a simple and intuitive story line visualization layout; 3) improving the quality and experience of understanding time sequence events: the present application proposes a multi-objective optimization-based visual layout method of time sequence event triplets, which can effectively help users understand the interaction between time sequence events and entities, so as to grasp the internal logic and plot development of the story. BRIEF DESCRIPTION OF DRAWINGS

[0064] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0065] Figure 1A flow chart of a method for visualizing a movie plot based on a multi-objective optimization-based temporal event triplets layout according to the present application;

[0066] Figure 2 A dependency tree diagram according to the present application;

[0067] Figure 3 A sliding window diagram according to the present application;

[0068] Figure 4 A movie plot visualization diagram according to the present application. Specific embodiments

[0069] Embodiment 1

[0070] Reference Figures 1 to 4 The present embodiment relates to a method for visualizing a movie plot based on a multi-objective optimization-based temporal event triplets layout according to the present application. Figure 1 The complete flow of the method is shown, and a movie synopsis data is selected as the original experimental data. The data processing stage includes extraction of SPO triplets, time axis and events, and the specific introduction of the extraction of triplets using HanLP technology; the optimization stage follows the design requirements, proposes optimization objectives and corresponding constraints, and further realizes the optimal solution of the multi-objective optimization model. It specifically includes the following steps:

[0071] Step 1, extract triplets. First, clean the original text data, and then extract the <subject-predicate-object>(SPO) structure triplets. Use HanLp technology to analyze the dependency relationship and semantic dependency of the sentence, and convert it into a dependency relationship tree, wherein the dependency syntax tree identifies the grammatical components, and the semantic dependency tree identifies the semantic relationship. Select the nodes that meet the subject-predicate relationship(SBV) and the verb-object relationship(VOB) from the dependency syntax tree, and get the nodes whose semantic relationship type is "agent" or "patient" from the semantic dependency tree. Compare the obtained nodes, if the nodes from the two syntax trees are the same, then combine them into SPO triplets.

[0072] Step 2, build a multi-objective optimization model. According to the proposed design requirements, two optimization objectives are summarized, which are to reduce the line intersection points and to keep the related entities in the same event as close to each other as possible. Express the above two optimization objectives in mathematics, and build a multi-objective optimization model.

[0073] The design requirements in step 2 specifically include the following points:

[0074] (1) Use a line that develops from left to right with time to represent an entity.

[0075] (2) Use the aggregation and divergence of the line to represent whether the entity participates in the same event.

[0076] (3) Reduce the number of line intersections. Fewer line intersections can reduce the complexity of the view.

[0077] (4) Reduce the number of line swings. Too many or unnecessary line swings will not only make the line appear discontinuous, but will also indirectly increase the number of line intersections.

[0078] (5) Represent entities participating in the same event in an intuitive way.

[0079] (6) Entities in the same SPO triplet in the event should be as close as possible to each other.

[0080] Step 2.1, Optimization Objective 1: Reduce line intersections. First, define two entities i and j, y i，t Let y represent the ordinate of entity i at time t. j，t Let represent the ordinate of entity j at time t. If two entities intersect between time t and time t+1, then (y... i，t -y j，t )(y i，t+1 -y j，t+1 ) < 0. To minimize the number of intersections in the storyline visualization, optimization objective one can be expressed as:

[0081]

[0082] Considering that formula (1) is rather cumbersome, a 0-1 decision variable p is introduced. i，j，t and o i，t，h p i，j，t Used to determine whether the i-th entity is below the j-th entity at time t. If y i，t <y j，t Then p i，j，t =1, otherwise p i，j，t =0. o i，t，h This is used to determine the position of entity i in the queue at time t. If the positions of entity i at times h intersect, then (y i，t -y j，t )(y i，t+1 -y j，t+1 If ) < 0, then o i，t，h =1, otherwise o i，t，h =0, where h∈[1,H], and H represents the length of the entity queue. i，t，h and y i，t The relationship between them can be described by the following formula.

[0083]

[0084] Therefore, formula (1) can be simplified to:

[0085]

[0086] Step 2.2, optimization goal two: related entities in the same event are as close to each other as possible. This optimization goal is decomposed by time, that is, it can be described as minimizing the difference in y-axis coordinates between two entities i and j. Equation (2) shows the y-axis coordinate of entity i at time t, so the difference in y-axis coordinates between the two entities can be calculated using the following equation.

[0087]

[0088] When solving optimization goal two, two points need to be noted: first, entity i and j participated in event e at time t, and second, there is a relationship between entities i and j, so two 0-1 constants are introduced here, C i，e and B e，t . Among them, C i，e indicates that entity i participated in event e, and if entity i participated in event e, then C i，e = 1. B e，t indicates that event e occurred at time t, and if event e occurred at time t, then B e，t = 1, so optimization goal two can be expressed as follows.

[0089]

[0090] Step 2.3, multi-objective optimization model. According to the optimization goal "reduce line intersections" described in step 2.1 and the optimization goal "related entities in the same event are as close to each other as possible" described in step 2.2, a multi-objective optimization model is formed. The above two objective functions are converted using weighted linear combination, as shown below.

[0091]

[0092] Because reducing line intersections is more important, here we set α = 1 and β = 0.1.

[0093] Step 3, set constraints. First, ensure that at time t, there is a difference in the positions of entities i and j, and exclude the case of taking only one entity, which can be converted into a "assignment problem". Second, to meet the requirement in step 2 that related entities in the same event are as close to each other as possible, the positions of entities need to be limited and it is necessary to ensure that there is no mixing of unrelated entities in the entity group participating in the same event, which can be converted into a "sliding window problem".

[0094] Step 3.1, setting constraint condition one: ensure that there is a difference in position between entities i and j at the same t time, and exclude the case of taking only one entity. For the position problem of entity i in the queue at t time, it can be converted into an "assignment problem", assuming that there are H boxes in the queue, at t time, entity i must and can only enter one box h, which can be expressed as the following formula:

[0095]

[0096]

[0097] Among them, formula (7-1) ensures that there is and only one entity i at any time t and position h; formula (7-2) ensures that there is and only one position h at t time for any entity i.

[0098] The relationship between o i，t，h and p i，j，t can be expressed as the following formula:

[0099]

[0100]

[0101]

[0102] Through formula (8-2) and formula (8-3), it can be ensured that different entities i and j are located at different positions at time t, and the case of taking only one entity is excluded.

[0103] Step 3.2, setting constraint condition two: to achieve optimization goal two in step 2.2, while determining the entity position, additional constraints need to be added to the entities involved in the same event e. During the event e, the y-axis coordinate of the entity remains unchanged, which can be expressed as the following formula:

[0104]

[0105] Where T e represents the duration of event e.

[0106] In addition, restrictions need to be added in the y-axis direction to avoid mixing irrelevant entities in the entity group participating in the same event e. Here, the entity group is compared to a window with a length of H e , and the position adjustment of the entity group is considered as the sliding of the window, that is, the entity group is bundled and moved together. During the window sliding process, there are the following situations, that is, the starting point of the window is the starting point of the entity queue; the window is in the entity queue; the end point of the window is the end point of the entity queue. The sliding window can be described by formula (10), where d e，t，his a 0-1 decision variable, which is used to determine the start point of the window composed of the entities participating in event e at time t.

[0107]

[0108]

[0109]

[0110]

[0111] Equation (10-1) ensures that there is a starting point of the window when h∈[1, H-H e +1]; Equation (10-2) ensures that there is no head of the window when h∈[H-H e +2, H]. Through Equation (10-1) and Equation (10-2), it can be ensured that there is and only one point as the starting point of the window when h∈[1, H]. Equation (10-3) can ensure that the entity i participating in event e cannot be arranged in front of the starting point h, and Equation (10-4) can ensure that all entities participating in event e are within the window.

[0112] Step 4, algorithm implementation. A python program is used to implement the multi-objective optimization model, and the optimal solution of the multi-objective optimization model is obtained by exploring the solution space through Gurobi. The minimum running time of the program is set to 30 minutes, and two conditions for terminating the solving program are set. The two conditions for terminating the solving program are that the relative gap between the best known solution and the best known bound is less than a given threshold and the set time frame is 12 hours. According to the above conditions, the experimental results are finally obtained, as shown in the following table:

[0113]

[0114] Step 5, visualization of movie plot. The timeline, triplets and events extracted in the data processing stage are input into the above model, and the relationship between entities is divided into three categories: neutral, friendly and malicious. By analyzing the relationship and behavior type between entities, the entities in the story are distinguished, and the role positioning of the entities in the plot is understood, as shown in Figure 4 the user can consider the character “Murphy” in the movie as a good person because the relationship type involving him is mostly benign.

[0115] Embodiment 2

[0116] The embodiment relates to a time sequence event triplet visual layout system based on multi-objective optimization, which is used to implement the method of embodiment 1, comprising:

[0117] The triple extraction module is used to firstly clean the original text data, and then extract the triple of <subject-predicate-object> (SPO) structure; the HanLP technology is used to analyze the dependency relationship and semantic dependency of the sentence, and convert it into a dependency relationship tree, wherein the dependency syntax tree identifies the syntax component, and the semantic dependency tree identifies the semantic relationship; nodes satisfying the subject-predicate relationship (SBV) and the verb-object relationship (VOB) are selected from the dependency syntax tree, and nodes whose semantic relationship type is "agent" or "patient" are obtained from the semantic dependency tree; the obtained nodes are compared, if the nodes from the two syntax trees are the same, the nodes are combined into an SPO triple;

[0118] The multi-objective optimization model construction module is used to summarize two optimization objectives according to the proposed design requirements, which are to reduce line intersection points and to make associated entities in the same event as close to each other as possible, to express the two optimization objectives in mathematics, and to construct a multi-objective optimization model.

[0119] The constraint condition setting module is used to firstly ensure that there is a difference in the positions of entities i and j at time t, and to exclude the case of taking only one entity, which can be converted into a "assignment problem"; secondly, to meet the multi-objective optimization model construction module, the associated entities in the same event need to be as close to each other as possible, which requires limiting the position of the entity and ensuring that there is no irrelevant entity mixed in the entity group participating in the same event, which can be converted into a "sliding window problem";

[0120] The algorithm implementation module uses a python program to implement the multi-objective optimization model, and obtains the optimal solution of the multi-objective optimization model by exploring the solution space through Gurobi; the minimum running time of the program is set to 30 minutes, and two conditions for terminating the solving program are set; the two conditions for terminating the solving program are that the relative gap between the best known solution and the best known boundary is less than a given threshold value, and the set time frame is 12 hours; the final result is obtained according to the above conditions.

[0121] Embodiment 3

[0122] This embodiment relates to a time sequence event triple visual layout device based on multi-objective optimization, comprising a memory and one or more processors, the memory stores executable code, and the one or more processors execute the executable code to implement the method of embodiment 1.

[0123] Embodiment 4

[0124] This embodiment relates to a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the method of embodiment 1.

[0125] Embodiment 5

[0126] An embodiment of the present disclosure relates to a computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method of embodiment 1.

Claims

1. A method for visual layout of temporal event triples based on multi-objective optimization, characterized in that, The method comprises the following steps: Step 1, extracting triplets; first, the original text data is cleaned, and then the triplets of <subject-predicate-object> (SPO) structure are extracted; HanLP technology is used to analyze the dependency relationship and semantic dependency of the sentence, and the dependency relationship tree is converted; the dependency syntax tree identifies the syntax component, and the semantic dependency tree identifies the semantic relationship; the nodes satisfying the subject-predicate relationship (SBV) and the verb-object relationship (VOB) are selected from the dependency syntax tree, and the nodes whose semantic relationship type is "agent" or "patient" are obtained from the semantic dependency tree; the obtained nodes are compared, if the nodes from the two syntax trees are the same, then they are combined into SPO triplets; Step 2, constructing a multi-objective optimization model; according to the design requirements proposed, two optimization objectives are summarized, which are to reduce line intersection points and to make associated entities in the same event as close to each other as possible; the two optimization objectives are expressed in mathematics, and a multi-objective optimization model is constructed; Step 3, setting constraints; first, ensure that there is a difference in the position of entity i and j at time t, and exclude the case of taking only one entity; this constraint can be converted into a "assignment problem"; second, to meet the requirement that associated entities in the same event are as close to each other as possible in step 2, the position of the entity needs to be limited and it is ensured that there is no irrelevant entity mixed in the entity group participating in the same event; this can be converted into a "sliding window problem"; Step 4, algorithm implementation; the multi-objective optimization model is realized by using a python program, and the optimal solution of the multi-objective optimization model is obtained by exploring the solution space through Gurobi; the minimum running time of the program is set to 30 minutes, and two conditions for terminating the solving program are set; the two conditions for terminating the solving program are that the relative gap between the best known solution and the best known boundary is less than a given threshold and the set time frame is 12 hours; the final result is obtained according to the above conditions.

2. The method of claim 1, wherein, The design requirements in step 2 specifically include the following points: (1) represent the entity with a line that develops from left to right over time; (2) use the aggregation and divergence of the line to represent whether the entity participates in the same event; (3) reduce the number of line intersection points, and fewer line intersections can reduce the complexity of the view; (4) reduce the number of line swings, too many or unnecessary line swings not only make the line discontinuous in vision, but also increase the number of line intersection points; (5) represent the entities participating in the same event in an intuitive way; (6) make the entities in the same SPO triplet in the event as close to each other as possible.

3. The method of claim 1, wherein, Step 2 specifically includes the following steps: Step 2.1, Optimization Objective 1: Reduce line intersections; First, define two entities i and j, y i，t Let y represent the ordinate of entity i at time t. j，t This represents the y-coordinate of entity j at time t; if two entities intersect between time t and time t+1, then (y i，t -y j，t )(y i，t+1 -y j，t+1 To minimize the number of intersections in the storyline visualization, the first optimization objective can be expressed as: To simplify the formula (1), 0-1 decision variable p i，j，t and o i，t，h are introduced i，j，t to determine whether the ith entity is below the jth entity at time t, if y i，t < y j，t , then p i，j，t = 1, otherwise p i，j，t = 0; o i，t，h is used to determine the position of entity i in the queue at time t, if the position of entity i is h, then o i，t = 1 if (y j，t - y i，t+1 )(y j，t+1 - y i，t，h ) < 0 between time h and h+1, otherwise o i，t，h = 0, where h ∈ [1, H], H represents the length of the entity queue; the relationship between o i，t，h and y i，t can be described by the following formula Therefore, formula (1) can be simplified as: Step 2.2, optimization objective two: associated entities in the same event are as close to each other as possible; this optimization objective can be described as minimizing the y-axis coordinate difference between two entities i and j by time decomposition; formula (2) shows that the y-axis coordinate of entity i at time t is represented, so the y-axis coordinate difference between the two entities can be calculated by the following formula: In solving the optimization goal two, two points need to be pointed out: one is that entities i and j participate in event e at time t, and two is that there is a connection between entities i and j, so two 0-1 constants are introduced here, which are C i，e and B e，t ; wherein, C i，e indicates that entity i participates in event e, and if entity i participates in event e, C i，e = 1; B e，t indicates that event e occurs at time t, and if event e occurs at time t, B e，t = 1, so the optimization goal two can be represented as follows; Step 2.3, multi-objective optimization model; the multi-objective optimization model is composed of the optimization objective "reducing line intersection points" in step 2.1 and the optimization objective "entities related to each other in the same event are as close to each other as possible" in step 2.2; the two objective functions are converted by using a weighted linear combination as follows: Because reducing line intersection points is more important, the parameters are set as α = 1 and β = 0.1 here.

4. The method of claim 1, wherein, The step 3 specifically comprises the following steps: Step 3.1, setting constraint condition one: ensuring that there is a difference in position between entities i and j at the same time t, and excluding the case of taking only one entity; for the problem of confirming the position of entity i in the queue at time t, it can be converted into a "assignment problem", assuming that there are H boxes to form a queue, at time t, entity i must and can only enter one box h, which can be expressed as the following formula: Wherein, formula (7-1) ensures that there is and only one entity i at any time t and position h; formula (7-2) ensures that there is and only one position h for any entity i at time t; The relationship between o i，t，h and p i，j，t can be expressed by the following equation: Through formula (8-2) and formula (8-3), it can be ensured that different entities i and j are located at different positions at time t, and the case of taking only one entity is excluded; Step 3.2, setting constraint condition two: to achieve the second optimization objective in step 2.2, while determining the position of the entity, an additional constraint is needed for the entities involved in the same event e; the y-axis coordinate of the entity remains unchanged during the event e, which is expressed as the following formula: where T e denotes the duration of event e; In addition, it is necessary to increase the restriction in the y-axis direction to avoid mixing irrelevant entities in the entity group participating in the same event e; here the entity group is compared to a window with length H e The position adjustment of the entity group is considered as the sliding of the window, that is, the entity group is bound to move together, and during the sliding of the window, there are the following cases, that is, the starting point of the window is the starting point of the entity queue; the window is in the entity queue; the end point of the window is the end point of the entity queue; the sliding window can be described by formula (10), wherein d e，t，h is a 0-1 decision variable, which is used to determine the starting point of the window formed by the entities participating in the event e at time t; Formula (10-1) guarantees that there is a position as the starting point of the window when h∈[1, H-H e +1]; Formula (10-2) guarantees that there is no head of the window when h∈[H-H e +2, H]; by Formula (10-1) and Formula (10-2), it can be guaranteed that there is only one point as the starting point of the window when h∈[1, H]; Formula (10-3) can ensure that the entity i participating in the event e cannot be arranged in front of the starting point h, and Formula (10-4) can ensure that all entities participating in the event e are within the window.

5. A multi-objective optimization based visual layout system for temporal event triples, characterized in that, It comprises: A triple extraction module is used to first clean the original text data, and then extract the triple of <subject-predicate-object> (SPO) structure; HanLP technology is used to analyze the dependency relationship and semantic dependency of the sentence, and convert it into a dependency relationship tree, wherein the dependency syntax tree identifies the syntax component, and the semantic dependency tree identifies the semantic relationship; the nodes satisfying the subject-predicate relationship (SBV) and the verb-object relationship (VOB) are selected from the dependency syntax tree, and the nodes whose semantic relationship type is "agent" or "patient" are obtained from the semantic dependency tree. The obtained nodes are compared, and if the nodes from the two syntax trees are the same, they are combined into an SPO triple; A multi-objective optimization model construction module is used to summarize two optimization objectives according to the design requirements proposed, which are reducing line intersection points and entities related to each other in the same event being as close to each other as possible, express the two optimization objectives in mathematics, and construct a multi-objective optimization model; A constraint condition setting module is used to first ensure that there is a difference in position between entities i and j at time t, and exclude the case of taking only one entity, which can be converted into an "assignment problem"; secondly, to meet the multi-objective optimization model construction module in the same event related to each other, the position of the entity needs to be limited and it is ensured that there is no unrelated entity mixed in the entity group participating in the same event, which can be converted into a "sliding window problem"; An algorithm implementation module is used to implement the multi-objective optimization model by using a python program, and the optimal solution of the multi-objective optimization model is obtained by exploring the solution space through Gurobi; the minimum running time of the program is set to 30 minutes, and two conditions for terminating the solving program are set; the two conditions for terminating the solving program are that the relative gap between the best known solution and the best known bound is less than a given threshold value and the set time frame is 12 hours; and finally the results are obtained according to the above conditions.

6. A visual layout device for temporal event triples based on multi-objective optimization, characterized in that, The memory stores executable code, and the one or more processors execute the executable code to implement the multi-objective optimization-based visual layout method of temporal event triplets.

7. A computer-readable storage medium, characterized in that, A program is stored thereon, and the program is executed by a processor to implement the method of any one of claims 1-4.

8. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and the processor executes the executable code to implement the method of any one of claims 1-4.

Images