A progressive timeline visual layout method based on integer linear programming and multi-objective optimization

Abstract

A kind of progressive timeline layout method based on mathematical optimization model, comprising: (1) obtaining certain time data, or analyzing and substituting time to non-time data, obtaining time evolution set, and representing time evolution set data;(2) in time evolution set, the original data is fixed, and new data point is inserted, the order of new point is determined by objective function, and harmony is as far as possible;(3) design optimization objective function, scale the coordinates of new point and calculate continuous coordinates, and design constraint condition to constrain the position of coordinates;(4) the objective function of model is processed by linear technology to adapt to Gurobi programming interface, and finally the model is solved by Gurobi.The application solves the problem of visual burden and confusion, and realizes a visual layout scheme of time evolution set.

Description

TECHNICAL FIELD

[0001] The present application relates to a progressive timeline visual layout method based on integer linear programming and multi-objective optimization. BACKGROUND

[0002] With the development of the information age, human production activities will generate a large amount of data, some of which are time-dependent, that is, time-evolving data. In order to analyze and explore time-evolving set data and obtain key information in dynamic high-dimensional information, data projection mode can be used for visual analysis of time-evolving set. Data projection is a widely used technique in data visualization, which can reduce data dimension, refine features and summarize data information. Scatter plot is the most commonly used visualization form to represent data distribution after data dimension reduction.

[0003] The timeline layout algorithm updates the element layout of the scatter plot through time advancement, which is an algorithm for effectively processing the set element layout of the time-evolving set. In recent years, many studies have involved time-evolving set data, such as the recognition model in pedestrian re-identification, which provides a ranking list of personnel meeting the user query. Limited by scalability, users only focus on the top K data points. In addition, due to changes in data and user queries, the set of data points will change greatly over time.

[0004] Timeline is a visualization form used to represent dynamic data in vertical or horizontal direction. Many researchers use timeline-based layout to visualize the progress of events and story layout. Story layout, as a typical timeline-based layout, has attracted the attention of many researchers, and most of their work has optimized the story layout from the aesthetic point of view.

[0005] When visualizing time-evolving sets based on timelines, it can effectively display the time-varying information of data in the horizontal direction in the set, but a large amount of time-evolving set data will cause visual burden and confusion when represented in a visual form. SUMMARY

[0006] The present application overcomes the above-mentioned shortcomings of the prior art and proposes a progressive timeline visual layout method based on integer linear programming and multi-objective optimization, which is used to determine the order and coordinates of elements in the inserted data set when inserting new data at the current layout timestamp, and design a coordinate scaling objective function to minimize curve swing and height from the aesthetic point of view.

[0007] To solve the above technical problems, the present application provides the following technical solutions:

[0008] A progressive timeline visual layout method based on integer linear programming and multi-objective optimization, the method comprising the following steps:

[0009] 1) Data description; the invention represents the time-evolving set as S t where t denotes the time stamp. S t is a dynamic set that changes over time, and needs to be distinguished between S t and the changed data subset and the fixed data subset at the previous time t-1. Define where denotes the set of original points in the point set, denotes the set of new points, and

[0010] (1-1) Coordinates of points; the invention represents the coordinates of points in S t in the layout algorithm, each point in S t is labeled as where is a continuous variable. The invention labels the order of each point in S t as where is a discrete variable greater than 0.

[0011] (1-2) Asymptotic timeline visual layout; the invention does not change the order and coordinates of points in the previous S t , S 0 , S 1 , S 2 ,..., S t-1 in the layout S t . The layout algorithm of the invention includes two steps: the first step is to determine the order of points in S t , and the second step is to scale the distance and determine the continuous coordinates of each point in S t .

[0012] 2) Determine the order of points in S t ; in discrete layout, the invention first keeps the order of points in S and inserts new points in S . While inserting new points, the goal is to make the original distance of new points as harmonious as possible, for which the following model is used as the layout requirement:

[0013]

[0014]

[0015] In model (1), the objective function contains an indicator function I{·}, which means that the function takes 1 if and only if the condition of the indicator function is true. In this model, the value of the range is between 1 and |S t |, dij is the distance between point i and point j projected onto the image. Note that when d ij ik and |x i j i k | are different, such as d ij ik > 0 and |x i j i k | < 0, the coordinate distance between i, j is larger than i, k, but the feature distance between i, j is smaller than i, k. This results in the imbalance between the coordinate distance and the feature distance between points, which is contrary to the order goal of the present invention. If the feature distance between i and j is larger than i and k, the model makes the order of points satisfy the feature distance order as much as possible.

[0016] The constraints in model (1) are designed to keep the order of the original points in , which only applies to the points in set to ensure that they have coordinates at time t-1.

[0017] Model (1) is a nonlinear programming model with a nonlinear objective function and linear constraints. Considering that the objective function in model (1) is complicated, the present invention uses linearization techniques and other decision variables to simplify the model and solve it through the Gurobi solver. When programming using the Gurobi interface, the present invention uses linearization techniques to help handle the indicator function. For example, to handle the absolute value in the objective function, the present invention defines two 0-1 decision variables p ij and q ijk . If the position of point i is higher than j, p ij is 1, otherwise 0. If the condition in the indicator function is true, q ijk is 1, otherwise q ijk is 0. By using p ij and q ijk , the objective function in the equation is rewritten as:

[0018]

[0019] where equation (2) is a linear function, avoiding the complicated form in equation (1), but new constraints are needed to describe the relationship between and p ij . The new constraints are represented as follows:

[0020]

[0021] ​​​​​​​​where M represents any chosen sufficiently large positive number. Equation (3) combines and p ij , and the constraint that and q ijk are represented as:

[0022]

[0023] The present application combines new constraints and a new objective function to obtain a new nonlinear programming model, and solves it through the Gurobi interface.

[0024] 3) Scaling the coordinates; after determining the order of the S t midpoints, the following three optimization objective functions are considered to scale and calculate the continuous coordinates of the points.

[0025] (3-1) Three optimization objective functions; respectively, the scale objective function, the boundary objective function, and the swing objective function.

[0026] The scale objective function is an extension of the objective function in discrete optimization. Given two points i and j, the ratio of the coordinate distance between them to the feature distance is represented as The goal is to make the scale between different points as close as possible, represented by δ t , which represents the scale range and minimizes it, and the objective function is represented as:

[0027]

[0028] where is the maximum value of the scale of all point pairs in the set, is the minimum value of the scale of all point pairs in the set. The smaller f1 is, the closer the distance between point pairs is to the distance in the projected coordinate plane.

[0029] The boundary objective function aims to avoid the range of point coordinates being too large. It makes the coordinate range of the points closer to

[0030]

[0031] where max{y i} is the maximum value of the y-coordinate of all points, and min{y i} is the minimum value of the y-coordinate of all points. The smaller f2 is, the closer the distance between point pairs is.

[0032] The swing objective function aims to avoid the line swing between the original points in The swing objective function is a measure of classic aesthetics in timeline layout. The present application describes the swing objective function as follows:

[0033]

[0034] Where ω i This is the weight of the original point, used to measure the importance of the point. A smaller f3 makes the distance between elements in adjacent sets closer.

[0035] In addition, the present invention takes into account several constraints, including gap constraints, boundary constraints and sequence constraints.

[0036] (3-2) Three constraints: gap constraint, boundary constraint, and sequence constraint.

[0037] The boundary objective function aims to bring points closer together. To avoid visual clutter and interference caused by overlapping points that are too close together, this invention establishes a gap constraint to limit the minimum distance between points, as shown below:

[0038]

[0039] Where D min This represents the minimum distance between two points.

[0040] Boundary constraints aim to place points along the height of a rectangular canvas, as shown below:

[0041]

[0042] The sequence constraints use the results of model (1). This invention utilizes sequence constraints to... Preserving the original order of points, this invention writes the order constraint as follows:

[0043]

[0044] Among the above constraints It has been determined by the discrete optimization model.

[0045] (3-3) Multi-objective programming model; The objective function and constraints are combined into a multi-objective programming model. Due to the complexity of solving multi-objective models, this invention uses a weighted linear combination to simplify the three objective functions, as shown below:

[0046] minα1f1+α2f2+α3f3 (11)

[0047] Where α1, α2, and α3 are the positive weights balancing the three objective functions. The proportional objective function has a smaller range than the other two objective functions. Finally, this invention yields a quadratic programming model with linear constraints. Similarly, the objective function is rewritten to fit the Gurobi interface, for example, equation (6) is rewritten as follows:

[0048] f2=z2-z1 (12)

[0049] where z1 and z2 are two variables, the present application uses them to limit y by the following constraints i The maximum and minimum values of y:

[0050]

[0051] The present application uses linearization techniques to handle the objective function of the model to adapt to the Gurobi programming interface, and then solves the model through Gurobi.

[0052] Preferably, in step (3-3), after multiple tests, the values of α1, α2, and α3 are 100, 0.01, and 0.1, respectively.

[0053] The technical concept of the present application is to insert new data points into the time evolution set and assign them with order and coordinate attributes. In order to make the distance of the new points and the distance in the original set as proportional as possible, a nonlinear objective function and model are designed to determine the order, and linearization techniques and other decision variables are used to simplify the model and then solved by the Gurobi solver. Then, from the perspective of coordinate scaling, combined with multi-objective optimization and constraint conditions, a new nonlinear programming model is obtained, and solved through the Gurobi interface.

[0054] The advantage of the present application is that the progressive timeline visual layout method represents an efficient method for solving the order of data points in timeline layout based on integer linear programming mathematical model, and designs a coordinate scaling objective function and constraint condition to minimize the curve swing and height from an aesthetic aspect for a large amount of time data, to a certain extent, solves the problem of visual burden and confusion, and realizes a visual layout scheme of time evolution set. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a flowchart of the progressive timeline visual layout of the present application;

[0056] Figure 2 is a timeline visual layout schematic diagram before the position of the midpoint in the layout S t of the present application;

[0057] Figure 3 is a timeline visual layout schematic diagram at the time of layout S t of the present application;

[0058] Figure 4 is a timeline visual layout schematic diagram at the time of layout S t+1 of the present application. DETAILED DESCRIPTION

[0059] The present application will be further described below in conjunction with the accompanying Figures 1-4

[0060] ​This embodiment describes a method for collecting pedestrian flow data in key areas of a park, applying the progressive timeline visual layout method based on integer linear programming and multi-objective optimization of the present invention. The method includes the following steps:

[0061] 1) such as Figure 1 As shown, the main contents of the implementation process include obtaining data from the database, processing relevant time data or non-time data into a time evolution set, determining the order of inserting data points at the current time, performing coordinate scaling operations after determining the order, and determining the coordinates of the data points at the current time.

[0062] This invention represents the time evolution set as S t , where t represents the timestamp. S t It is a dynamic collection that changes over time, and S needs to be distinguished during layout. t The subset of data that has changed compared to the previous time t-1 and the subset of data that remains constant. Definition in Denotes the set of original points in the point set. Let represent the set of new points, and

[0063] Specifically, the process involves acquiring visitor data for key areas of the park, dividing the data into segments based on time periods (e.g., by day), and obtaining visitor data for key areas in each time period, which is then used as a time evolution set.

[0064] As time changes, when the data on visitors to key areas of the park is updated, such as when new data is collected on a certain day, the progressive visual layout method provided by this invention can be used to determine the order and coordinates of the crowd points in the timeline layout of past periods without changing them, so as to carry out progressive visual layout.

[0065] (1-1) In the layout algorithm of this invention, S t Represent the coordinates of the midpoint, for S t The coordinates of each point are labeled as follows: in It is a continuous variable. This invention relates to S. t The order of each point in the data is labeled as follows: in It is a discrete variable greater than 0.

[0066] (1-2) In the layout S of the present invention t When the midpoint is in position, the previous S is not changed. 0 ,S 1 ,S 2 ,...,S t-1 The order and coordinates of the points in the data, such as Figure 2 As shown, S represents...0 S 1 S 2 ..., S 4 The layout method of the present invention includes two steps: the first step is to determine the order of the points in S t , and the second step is to scale the distances and determine the continuous coordinates of each point in S t .

[0067] In the crowd data of the key area of the park, it is shown that the newly added data is S t , where the points represent the crowd points in this period of time, S 0 , S 1 , S 2 ..., S t-1 represents the data set of the original period of time crowd points, represents the position order of the i-th crowd point in the newly added data set, a continuous variable represents the coordinates of the i-th crowd point in the newly added data set in the timeline visualization, and the order and coordinates of the points in S t in the timeline visualization are determined by the two steps of the present invention and

[0068] 2) In the discrete layout, the present invention first maintains the order of the points in S , and inserts new points in S . While inserting new points, the purpose is to make the original distances of the new points as harmonious as possible. As shown in Figure 3 , in the point set at time 6, the points connected with time 5 are the points in S , and the other points are new points inserted in S . After determining the order of the points in S t , at the next time t+1, the order of the points in S t+1 is determined, such as the order of the inserted points shown in time 7 of Figure 4 , until the order of the points of each time stamp in the progressive timeline visualization layout is determined. For this purpose, the following model is used as the layout requirement:

[0069]

[0070]

[0071] In model (1), the objective function contains an indicator function I{·}, which means that the function takes 1 when and only when the condition of the indicator function is true. In this model, the value ranges between 1 and |S t |, d ijis the distance between point i and point j projected onto the image. Note that when d ij -d ik and |x i -x j |-|x i -x k | are of different signs, such as d ij -d ik > 0 and |x i -x j |-|x i -x k | < 0, the coordinate distance of i, j is larger than that of i, k, but the feature distance of i, j is smaller than that of i, k. This results in an imbalance between the coordinate distance and the feature distance between points, which is contrary to the order goal of the points of the present application. If the feature distance of i and j is larger than that of i and k, the model makes the order of the points satisfy the feature distance order as much as possible.

[0072] The constraints in model (1) are designed to maintain the order of the original points in , which only applies to the points in set to ensure that they have coordinates at time t-1.

[0073] Model (1) is a nonlinear programming model with a nonlinear objective function and linear constraints. Considering that the objective function in model (1) is complicated, the present application uses linearization techniques and other decision variables to simplify the model and solve it through the Gurobi solver. When programming using the Gurobi interface, the present application uses linearization techniques to help handle the indicator function. For example, in order to handle the absolute value in the objective function, the present application defines two 0-1 decision variables p ij and q ijk . If the position of point i is higher than j, p ij is 1, otherwise it is 0. If the condition in the indicator function is true, q ijk is 1, otherwise q ijk is 0. By using p ij and q ijk , the objective function in the equation is rewritten as:

[0074]

[0075] where equation (2) is a linear function that avoids the complicated form in equation (1), but new constraints need to be added to describe the relationship between and p ij . The new constraints are expressed as follows:

[0076] p ij +p ji =1

[0077] x i +1-x j ≤M(1-p ji ) (3)

[0078] where M represents any chosen sufficiently large positive number. Equation (3) combines the constraints of and p ij , while the constraint combining and q ijk is expressed as:

[0079]

[0080] The present application combines new constraints and a new objective function to obtain a new nonlinear programming model, and solves it through the Gurobi interface.

[0081] 3) After determining the order of the S t midpoint, consider the following three optimization objective functions to scale and calculate the continuous coordinates of the points.

[0082] (3-1) Use the scale objective function, boundary objective function, and swing objective function to optimize the coordinates.

[0083] The scale objective function is an extension of the objective function in discrete optimization. Given two points i and j, the ratio of the coordinate distance between them to the feature distance is expressed as The goal is to make the scale between different points as close as possible. Use δ t to represent the range of the scale and minimize it, and the objective function is expressed as:

[0084]

[0085] where is the maximum value of the scale of all point pairs in the set, is the minimum value of the scale of all point pairs in the set. The smaller f1 is, the closer the distance between point pairs is to the distance in the projected coordinate plane.

[0086] The boundary objective function aims to avoid the range of point coordinates being too large. It makes the coordinate range of the points closer through the objective function of equation (6):

[0087]

[0088] where max{y i} is the maximum value of the y-coordinate of all points, and min{y i} is the minimum value of the y-coordinate of all points. The smaller f2 is, the closer the distance between point pairs is.

[0089] The swing objective function aims to avoid The line between the original points swings. The swing objective function is a measure of classic aesthetics in timeline layout. The invention describes the swing objective function as follows:

[0090]

[0091] where ω i is the weight of the original point, which measures the importance of the point. Smaller f3 makes the distance between elements in the adjacent set closer.

[0092] (3-2) Use gap constraints, boundary constraints, and sequence constraints to constrain coordinates.

[0093] The boundary objective function makes the distance between points closer. To avoid visual confusion and interference caused by too close points, the invention establishes a gap constraint to limit the minimum distance between points, as follows:

[0094]

[0095] where D min represents the minimum distance between two points.

[0096] The boundary constraint aims to layout points on the height of the rectangular canvas, as follows:

[0097]

[0098] The sequence constraint uses the results of model (1). The invention uses the sequence constraint to preserve the original order of the points by The invention writes the sequence constraint as follows:

[0099]

[0100] where the in the above constraints have been determined by the discrete optimization model.

[0101] (3-3) Combine the objective functions and constraints into a multi-objective programming model. Due to the complexity of solving multi-objective models, the invention uses a weighted linear combination to simplify the three objective functions, as follows:

[0102] minα1f1+α2f2+α3f3 (11)

[0103] where α1, α2, α3 are positive weights that balance the three objective functions. After multiple tests, 100, 0.01, and 0.1 are chosen as their values. Because the scale objective function has a smaller range than the other two objective functions. Finally, the invention obtains a quadratic programming model with linear constraints. Rewrite the objective function to adapt to the Gurobi interface. For example, the invention rewrites equation (6) as follows:

[0104] f2 = z2 - z1 (12)

[0105] where z1 and z2 are two variables, the present application uses them to limit y by the following constraints i the maximum and minimum values of y:

[0106]

[0107] The present application processes the objective function of the model using linearization technology to adapt to the Gurobi programming interface, and then solves the model through Gurobi.

[0108] (4) After obtaining the coordinates of the data points, the elements of the newly added set are drawn in the timeline visualization through the visualization toolkit D3.js. The comparison of different set elements in the timeline can assist staff in analysis, and according to the change of the compared set elements, the park management personnel can make decisions to strengthen the management of the key areas. Moreover, as time changes, when the data of the crowd visiting the park key area is updated (that is, a new period of data is added), the method of the present application is used to determine the coordinate position of the crowd point in the new period, and the timeline visualization is updated, which can help staff to observe the differences of park visitors in real time (such as changes in traffic), and help park staff to guide visitors.

Claims

1. A progressive timeline visual layout method based on integer linear programming and multi-objective optimization, comprising the following steps: 1) processing time-dependent data into time-evolving set data; representing time-evolving set as S t where t denotes the time stamp; S t is a dynamic set that changes over time, and needs to be distinguished between S t the changing data subset and the fixed data subset at the previous time t-1; defining where denotes the set of original points in the point set, denotes the set of new points, and coordinates of the points (1-1); in the layout algorithm, the coordinates of the points S t midpoint are denoted as t coordinates of each point in S where is a continuous variable; the order of each point in S t is denoted as where is a discrete variable greater than 0; (1-2) Progressive timeline visual layout; in layout S t When the midpoint is in position, the previous S is not changed. 0 ,S 1 ,S 2 ,...,S t-1 The layout method for determining the order and coordinates of points in S involves two steps: the first step is to determine the order and coordinates of points in S. t The order of midpoints, the second step is to scale the distance and determine S. t The continuous coordinates of each point in the array; 2) determine S t the order of the midpoints; in a discrete layout, first in the order of the points is maintained, and in new points are inserted; while inserting new points, the aim is to make the original distances of the new points as harmonious as possible, for which the following model is used as a layout requirement: In model (1), the objective function includes an indicator function I{·}, which means that the function takes the value 1 if and only if the condition of the indicator function is true; in this model, The value ranges from 1 to |S t |between, d ij It is the distance between points i and j projected onto the image; note that when d ij -d ik and |x i -x j |-|x i -x k When the symbols of | are different, for example, d ij -d ik >0 and |x i -x j |-|x i -x k If |<0, the coordinate distance between i and j is greater than that between i and k, but the feature distance between i and j is smaller than that between i and k. This leads to an imbalance between the coordinate distance and feature distance between points, which is the opposite of the point order objective. If the feature distance between i and j is greater than that between i and k, the model will make the point order satisfy the feature distance order as much as possible. The constraints in model (1) aim at preserving the order of the original points, which only applies to the points in the set to ensure that they have coordinates at time t-1; Model (1) is a nonlinear programming model with a nonlinear objective function and linear constraints; considering that the objective function in model (1) is relatively complicated, linearization techniques and other decision variables are used to simplify the model, and it is solved by Gurobi solver; when using Gurobi interface programming, linearization techniques are used to help handle the indicator function; for example, in order to handle the absolute value in the objective function, two 0-1 decision variables p ij and q ijk are defined; if the position of point i is higher than j, p ij is 1, otherwise 0; if the condition in the indicator function is true, q ijk is 1, otherwise q ijk is 0; by using p ij and q ijk , the objective function in the equation is rewritten: where formula (2) is a linear function, which avoids the cumbersome form of formula (1), but requires the addition of new constraints to describe the relationship between p and p and p ij ; the new constraints are expressed as follows: p ij +p ji =1 x i +1-x j ≤M(1-p ji ) (3) where M represents any chosen sufficiently large positive number; equation (3) combines and p ij while the constraint on and q ijk is expressed as: A new nonlinear programming model is obtained by combining new constraints and new objective functions, and is solved through the Gurobi interface. 3) Scaling the coordinates; after determining the order of the midpoints t The scaling is performed using an optimization objective function and the successive coordinates of the points are calculated, and the relevant constraints on the coordinates of the points are proposed to optimize the proposed model, implementing a gradual timeline layout algorithm.

2. The progressive timeline visual layout method based on integer linear programming and multi-objective optimization according to claim 1, wherein the process of step 3) is as follows: (3-1) Three optimization objective functions; respectively, the proportional objective function, the boundary objective function, and the swing objective function; The proportional objective function is an extension of the objective function in discrete optimization; Given two points i and j, the ratio of the coordinate distance between them to the feature distance is written as The goal is to make the ratios between different points as close as possible, using δ t to represent the range of ratios and minimizing it, the objective function is expressed as: wherein is the maximum value of the ratio of all pairs of points in the set, is the minimum value of the ratio of all pairs of points in the set; the smaller f1, the closer the distance between pairs of points is to the distance on the projection coordinate plane; The boundary objective function aims to avoid too large point coordinate range; it makes the point coordinate range closer through the formula (6) objective function: where max {y i} is the maximum value of the ordinate of all points, and min {y i} is the minimum value of the ordinate of all points; The smaller f2 is, the closer the distance between points is; The wiggle objective function aims to avoid wiggling between the original points; the wiggle objective function is a measure of classic aesthetics in timeline layout; the wiggle objective function is described as follows: where ω i is the weight of the original point, which measures the importance of the point; a smaller f3 makes the distance between elements in the adjacent set closer; In addition, consider these constraints, including gap constraints, boundary constraints, and sequence constraints; (3-2) Three constraints; respectively, gap constraints, boundary constraints, and sequence constraints; The boundary objective function makes the points closer; in order to avoid visual confusion and interference caused by too close points overlapping, gap constraints are established to limit the minimum distance between points, as shown below: where D min represents the minimum value of the distance between two points; The boundary constraint aims to layout points on the height of the rectangular canvas, as shown below: The results of the sequential constraint usage model (1) are used; with sequential constraints by Preserving the original order of points, the sequential constraint is written as follows: wherein the above constraints are have been determined by a discrete optimization model; (3-3) Multi-objective programming model; combine the objective function and the constraint into a multi-objective programming model, due to the complexity of solving the multi-objective model, use weighted linear combination to simplify the three objective functions, as follows: min α1f1+α2f2+α3f3 (11) Where α1, α2, α3 are positive weights that balance the three objective functions, the proportional objective function has a smaller range than the other two objective functions; finally, a quadratic programming model with linear constraints is obtained; similarly, rewrite the objective function to adapt to the Gurobi interface, for example, rewrite formula (6) as follows: f2=z2-z1 (12) where z1 and z2 are two variables that are used to restrict y by the following constraints i max and min of y: Use linear techniques to process the objective function of the model to adapt to the Gurobi programming interface, and then solve the model through Gurobi.

3. A progressive timeline visual layout method based on integer linear programming and multi-objective optimization as claimed in claim 2, characterized in that: The values of α1, α2, α3 in step (3-3) are 100, 0.01, and 0.1, respectively.

Images