Method and system for automatically generating legend for visualization chart

Abstract

The present invention belongs to the field of visualization, and relates to a method and system for automatically generating a legend for a visualization chart. The method comprises: by means of performing geometric shape recognition and classification for visualization, obtaining different shape sets, and extracting a representative icon symbol and a mapping channel; on the basis of the extracted representative icon symbol and mapping channel, searching for a visualization legend in a high-dimensional legend space by means of a legend model agent; and, on the basis of a feedback model, scoring the searched visualization legend. By using the method disclosed in the present invention, for one visualization, user interactive adjustment and dynamic updating of a feedback model are implemented in a human-computer interaction framework according to different user preferences, so that a legend conforming to an evaluation measurement standard is generated; this is helpful for clear visual data interpretation, and lays a reliable foundation for the development of chart visualization.

Description

A method and system for automatically generating legends for visual charts Technical Field

[0001] The present invention belongs to the field of visualization, and in particular relates to a method and system for automatically generating legends for visualization charts. Background Art

[0002] Visualization is an important means of expressing data in the data age. Legends are important text components in visualizations, building a bridge between data and visualization elements. However, many visualizations suffer from missing or inappropriate legends, which prevent users from interpreting the data or lead to deviations in their interpretation, thus reducing the quality of visualization.

[0003] Summary of the Invention

[0004] In response to the defects existing in the prior art, the purpose of the present invention is to provide a method and system for automatically generating legends for visual charts. Taking a visualization as input, the system automatically generates an interactive legend in combination with user feedback. Taking into account that different users may have different preferences, preference measurement is also performed. User interaction adjustment and dynamic updating of the feedback model are implemented within the framework of human-computer interaction, thereby generating a legend that meets the evaluation metric standards.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] In a first aspect, a method for automatically generating a legend for a visual chart comprises the following steps:

[0007] S1. Extract representative icon symbols and mapping channels for visualization;

[0008] S2. Based on the extracted representative icon symbols and mapping channels, the legend model agent searches for visual legends in the high-dimensional legend space;

[0009] S3. Score the search visualization legend based on the feedback model.

[0010] Furthermore, extracting representative iconic symbols in step S1 is used for visualization, geometric shape recognition and classification, and obtaining different shape sets to extract representative iconic symbols. Extracting representative iconic symbols in step S1 includes the following sub-steps:

[0011] S111, performing precise shape matching on the visualization to classify shapes with identical shapes in the visualization and obtain corresponding shape sets;

[0012] S112, performing transformed shape matching on the visualization to classify the same shapes that have been transformed in the visualization and obtain a corresponding shape set;

[0013] S113, performing fuzzy shape clustering for visualization to cluster shapes with slight differences and obtain corresponding shape sets;

[0014] S114. Identify a representative icon symbol for each shape set.

[0015] Furthermore, in step S111 , precise shape matching is performed by comparing geometric contours to obtain a set of shapes with exactly the same shape in the visualization.

[0016] Furthermore, the shape transformation in step S112 includes translation, rotation and resizing. The centroid-vertex vector is used to represent each shape in step S112. Step S112 includes finding the center of gravity of the visualization primitive, calculating the distance between the center of gravity and the primitive vertices, and sequentially forming a vector with the distances between the center of gravity and each vertex to obtain the center of gravity-vertex vector of the primitive. Each center of gravity-vertex vector of the primitive is regularized by dividing it by the vector value with the largest distance to obtain a normalized vector, and the normalized vectors of each primitive are compared to perform shape transformation matching on the visualization.

[0017] Furthermore, in step S113 , in the clustering results, each cluster represents a shape set, and a shape is randomly selected from each cluster as a representative icon symbol of the corresponding shape set.

[0018] Furthermore, extracting mapping channels in step S1 is used to group the shape sets obtained when extracting representative iconic symbols according to color attributes. Extracting mapping channels in step S1 includes the following sub-steps:

[0019] S121, performing color clustering on each identified shape set to obtain a color grouping for each shape set;

[0020] S122, sorting the color groups of each shape set to obtain a corresponding color sequence;

[0021] S123 . Perform color interpolation on the sorted color sequence to obtain a color sequence suitable for generating a legend.

[0022] Furthermore, in step S121 , all visualized RGB colors are converted into CIELAB space and normalized, and clustering is performed using the DBSCAN algorithm in the normalized CIELAB space.

[0023] Furthermore, in step S2, a genetic algorithm is used to simulate the visualization legend search process as a gene mutation process, and the user feedback received from the feedback model is used to adjust and optimize the search results.

[0024] Furthermore, in step S3, the feedback model receives the evaluation metric index input by the user and scores the searched visualization legend through a multi-layer neural network.

[0025] In a second aspect, a system for automatically generating legends for visual charts adopts a method for automatically generating legends for visual charts as described in the first aspect of the present invention and any optional embodiment thereof to automatically generate legends for visualization.

[0026] The beneficial technical effect of the present invention is that: a method and system for automatically generating legends for visual charts disclosed in the present invention is adopted, with a visualization as input, by extracting icon symbols and mapping channels, finding matching legends in a high-dimensional legend space, and scoring the matching legends based on a feedback model, to generate legends that meet the evaluation metric standards. The feedback model receives the evaluation metric as input and scores the legends through a multi-layer neural network. The search network is based on a genetic algorithm and supports searching for solutions in a mixed space containing discrete spaces (such as arrangement direction, symbol layout, text layout) and continuous spaces (such as position). Taking into account that different users may have different preferences, the reward model also includes preference metrics (such as horizontal, vertical, center or edge preferences). Under the framework of human-computer interaction, user interaction adjustment and dynamic updating of the feedback model are realized, laying a solid foundation for the development of chart visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 shows corresponding legends generated for discrete, continuous and multi-channel visualization using a system for automatically generating legends for visual charts disclosed in a second embodiment of the present invention;

[0028] FIG2 shows corresponding legends generated for visualizations of different shapes using a system for automatically generating legends for visualization charts disclosed in a second embodiment of the present invention;

[0029] FIG3 is a legend generated for the stacked bar chart in FIG1( a ) by using a system for automatically generating legends for visual charts disclosed in the second embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1

[0032] An embodiment of the present invention provides a method for automatically generating a legend for a visual chart, the method comprising the following steps:

[0033] S1. For visualization, extract representative icon symbols and mapping channels.

[0034] For visualization, geometric shape recognition and classification are performed to obtain different shape sets to extract representative iconic symbols, and the recognized shape sets are further classified according to color attributes.

[0035] Extracting representative iconic symbols and mapping channels is used to identify the most representative visual elements and mapping relationships in the visualization. Extracting representative iconic symbols is used to identify and classify the geometric features of the visualization symbols. Extracting mapping channels is used to group shape elements according to color attributes. This process uses the shape sets identified in the previous step of extracting representative iconic symbols as the starting point.

[0036] Extracting representative iconic symbols in step S1 includes the following sub-steps:

[0037] S111. Perform precise shape matching on the visualization to classify shapes with exactly the same shape in the visualization and obtain a corresponding shape set.

[0038] In an alternative embodiment, symbols with identical shapes in the visualization are identified by directly comparing their geometric outlines.

[0039] For encoding uniform geometric shapes (such as rectangles, circles, or other identical shapes), the visualization effect achieved by performing exact shape matching is very good.

[0040] Because path elements are divided into two categories: closed paths and open paths, closed paths have the same start and end points, while open paths have different start and end points. Path elements are more complex and diverse in the information they encode, and many visualization effects use path elements to display basic geometric shapes. Therefore, the next two steps focus on processing path elements.

[0041] S112 : Perform transformed shape matching on the visualization to classify the transformed identical shapes in the visualization and obtain a corresponding shape set.

[0042] Transformation shape matching is performed to extract geometrically identical shaped elements that have been translated, rotated, and resized.

[0043] Each shape is represented using a centroid-vertex vector. The specific process is to find the center of gravity of the visualization primitive, then calculate the distance between the center of gravity and the primitive's vertices. The distances between the center of gravity and each vertex are sequentially constructed into a vector to obtain the primitive's centroid-vertex vector. Each centroid-vertex vector of the primitive is normalized by dividing it by the vector value with the largest distance to obtain a normalized vector. The normalized vectors of each primitive are compared to match the transformed visualization shape. The centroid-vertex vector is stable to changes caused by rotation and scaling, and the same transformed shape has the same number of centroid-vertex vectors.

[0044] S113. Perform fuzzy shape clustering for visualization to cluster shapes with slight differences and obtain corresponding shape sets.

[0045] For many shapes with only slight differences, it is necessary to mine their implicit common patterns. In an optional embodiment, shapes are clustered using area and aspect ratio in two-dimensional space. In an embodiment of the present invention, a density-based clustering algorithm, DBSCAN, is used to cluster shapes, and outliers that deviate are deleted. In the clustering results, each cluster represents a shape pattern, and a shape is randomly selected from each cluster as the representative of each cluster. This selected shape can be used to represent the common pattern in visualization.

[0046] S114. Identify a representative icon symbol for each shape set.

[0047] In step S112, any shape in the same shape set that has undergone shape transformation can be used as a representative icon symbol of the corresponding shape set. In step S113, in the clustering results, each cluster represents a shape set, and a shape randomly selected from each cluster is the representative icon symbol of the corresponding shape set.

[0048] Extracting the mapping channel in step S1 includes the following sub-steps:

[0049] S121 . Perform color clustering on the identified geometric shape sets to obtain color groups for each shape set.

[0050] In visualization, the same geometric shape set may have multiple colors. In this example, the colors of the same geometric shape set are clustered into several groups, including single-color continuous colors, multi-color continuous colors, divergent continuous colors, ordinal colors, and categorical colors. To perform color clustering, all RGB colors are converted to CIELAB space and normalized. Then, the DBSCAN algorithm is used to cluster these points in the normalized CIELAB space.

[0051] S122 , sorting the colors of the color groups of each shape set to obtain a corresponding color sequence.

[0052] Disordered color samples cannot be interpolated to form a continuous color gradient, and the colors should be rearranged into a reasonable order that conforms to human perception. This problem can be viewed as a traveling salesman problem, where the goal is to find the shortest path that passes through all color points and passes through them exactly once. To this end, in this embodiment, an approximation algorithm is adopted. First, the minimum adjacency matrix is ​​constructed using the nearest neighbor algorithm. If the number of colors exceeds 100, the minimum neighbor count is increased to 3 to avoid a disconnected adjacency graph. Since the starting and ending colors are unknown, a minimum spanning tree (MST) is constructed starting from each node, and a series of color points and their corresponding costs are obtained and recorded according to the pre-order traversal. The sequence with the minimum distance is extracted as the final color sequence.

[0053] S123 . Perform color interpolation on the sorted color sequence to obtain a color sequence suitable for generating a legend.

[0054] Consecutive color sampling points can be viewed as samples of a curve in color space. Color interpolation is performed on these consecutive sampling points to obtain a color sequence suitable for generating legends. In this embodiment, interpolation is performed using a cubic Bezier curve in the CIELAB color space. First, a smooth curve is fitted, and then points are sampled evenly along the curve. Because interpolation is performed in the CIELAB color space, this method produces results that are more consistent with human perception of color.

[0055] S2. Search for visualization legends in the high-dimensional legend space based on the extracted representative iconic symbols and mapped channels.

[0056] A legend model agent searches for visual legends in a mixed high-dimensional legend space containing both discrete and continuous dimensions. Spatial position is a continuous dimension, while other dimensions, such as arrangement direction, symbol layout, and text layout, are discrete dimensions. Searching for visual legends in a high-dimensional legend space is a typical optimization problem in a mixed combinatorial space. In an embodiment of the present invention, a genetic algorithm is used to simulate visual legend selection as a genetic mutation process. By receiving user feedback from a feedback model, the legend model agent can gradually generate better visual legends over time. The feedback model plays a crucial role in optimizing search results.

[0057] S3. Score the search visualization legend based on the feedback model.

[0058] The feedback model receives the evaluation metric as input and scores the searched visualization legend through a multi-layer neural network.

[0059] The Legend Model agent provides feedback on the exploration legends it generates based on multiple predefined metrics and their default weights. These weights are updated and adjusted in real time based on user feedback and applied directly to the Legend Model agent. The feedback model is a lightweight multi-layer perceptron, which can be viewed as a two-layer fully connected network. The weights of different metrics and measures are adjusted based on direct user edits to the legend, allowing for embedding expertise and personalization. The efficient combination of the feedback model and the Legend Model agent allows for rapid response to user needs for creating, adjusting, and personalizing recommended visualizations.

[0060] Users can modify legend settings directly through the interactive interface, including changing the legend symbol, text arrangement, and position. Legend setting modifications represent user knowledge or preferences. Legend setting modification records are collected and represented as partially ordered relational tuples representing user preferences. Backpropagation is performed on these tuples to the feedback model. Because the feedback model is small, it can be trained in real time. The updated feedback model is then used to continuously adapt to user preferences.

[0061] A feedback model is used to propagate user preferences. This feedback model is an online learning feedback model. After users edit chart positions and selections on the interactive interface, the backend collects and stores edit records. Based on these edit records, the results after the user edits are generally considered better than the results before the edits. Using these partially ordered binary tuples, the feedback model is trained in real time to rapidly update its parameters. Subsequent legend calculations use the updated feedback model.

[0062] Example 2

[0063] An embodiment of the present invention provides a system for automatically generating legends for visual charts. The system uses a method for automatically generating legends for visual charts as described in Example 1 of the present invention and any optional implementation manner thereof to automatically generate legends for visualization.

[0064] This section presents the titles of charts generated using the system for automatically generating legends for visualization charts disclosed in an embodiment of the present invention. This section demonstrates legends generated for six visualization types, including bar charts, node-link diagrams, maps, area charts, heat maps, and bubble charts. These visualizations cover a variety of visual channels and attributes; the generated legends include both single-channel and multi-channel legends.

[0065] As shown in FIG1 , a system for automatically generating legends for visualization charts disclosed in an embodiment of the present invention can generate legends for discrete, continuous, and multi-channel visualizations.

[0066] Figures 1(a) and 1(b) are single-channel color channels with classification and ordering, respectively. These types of charts occupy an important part of the visualization, and corresponding legends can be generated for them and placed in appropriate locations. Figure 1(c) shows a visualization of elements with multiple different labels; a system for automatically generating legends for visual charts disclosed in an embodiment of the present invention can generate different legends for circles, rectangles, and lines, and the generated legends also provide corresponding interactions. The legend maps node colors to different categories and maps the line thickness of the connecting edge to the connection of the node. Users can interact with the legend to highlight nodes or connecting edges according to specific attributes.

[0067] As shown in Figure 2, a system for automatically generating legends for visualization charts disclosed in an embodiment of the present invention is used to generate legends for various types of visualizations. Figure 2(d) shows a visualization with both size and color attribute elements. Two separate legends are generated by extracting two visual encoding channels from the visualization, representing the categorized color and size of the circles, respectively. The generated legends also support user interaction. As shown in Figure 2(e), the user can reconfigure the area chart to a line chart by setting the stroke color and setting the fill color to transparent. The visualization in Figure 2(f) has a quantitative single color channel, and the visualization is redirected by using a new color scheme.

[0068] A system for automatically generating legends for visual charts disclosed in an embodiment of the present invention can generate multiple legend options for a single visualization. As shown in FIG3 , for the stacked bar chart in FIG1(a), different types of legends are generated for the user to choose from. In addition to generating legends, a system for automatically generating legends for visual charts disclosed in an embodiment of the present invention can also support users to interact with legends in three ways: obtain corresponding values ​​through visualization, highlight corresponding content in visualization through legends, or redirect visualization through legends to change its style. For example, a user can interact with the legend to select a focused category, which will highlight the corresponding bar in the chart.

[0069] Through the above embodiments, it can be seen that the present invention discloses a method and system for automatically generating legends for visual charts. By performing geometric shape recognition and classification on the visualization, different shape sets are obtained, and representative icon symbols and mapping channels are extracted. Based on the extracted representative icons and mapping channels, a visual legend is searched in a high-dimensional legend space by a legend model agent. The searched visual legend is adjusted and scored based on a feedback model. Using the method disclosed in the present invention, for a visual input, according to different user preferences, dynamic updates of the user interaction adjustment and feedback model are implemented within the framework of human-computer interaction, thereby generating a legend that meets the evaluation metric standard, which helps to interpret data clearly and intuitively, and lays a solid foundation for the development of chart visualization.

[0070] The method and system of the present invention are not limited to the embodiments described in the specific implementation manner. Those skilled in the art may derive other implementation manners based on the technical solution of the present invention, which also fall within the scope of the technical innovation of the present invention.

Claims

1. A method for automatically generating a legend for a visualization chart, the method comprising the following steps: S1. Extract representative icon symbols and mapping channels for visualization; S2, based on the representative icon symbols and mapping channels extracted from the visualization, the legend model agent searches for visualization legends in the high-dimensional legend space; S3. Score the search visualization legend based on the feedback model.

2. A method for automatically generating legends for a visual chart as claimed in claim 1, characterized in that: Extracting representative iconic symbols in step S1 is used for visualization, geometric shape recognition and classification, and obtaining different shape sets to extract representative iconic symbols. Extracting representative iconic symbols in step S1 includes the following sub-steps: S111, performing precise shape matching for visualization to classify shapes with exactly the same shape in the visualization and obtain a corresponding shape set; S112, performing transformed shape matching for visualization, so as to classify the same shapes that have been transformed in the visualization and obtain a corresponding shape set; S113, performing fuzzy shape clustering for visualization, so as to cluster different shapes with slight differences and obtain corresponding shape sets; S114. Identify a representative icon symbol for each shape set.

3. A method for automatically generating legends for a visual chart as claimed in claim 2, characterized in that: In step S111, precise shape matching is performed by comparing geometric contours to obtain a set of shapes with exactly the same shape in the visualization.

4. A method for automatically generating legends for a visual chart as claimed in claim 2, characterized in that: The shape transformation in step S112 includes translation, rotation and resizing. The centroid-vertex vector is used to represent each shape in step S112. Step S112 includes finding the centroid of the visualization primitive, calculating the distance between the centroid and the primitive vertices, and sequentially constructing a vector from the distances between the centroid and each vertex to obtain the centroid-vertex vector of the primitive. Each centroid-vertex vector of the primitive is regularized by dividing the vector value with the largest distance to obtain a normalized vector, and the visualization is transformed and matched by comparing the normalized vectors of each primitive.

5. A method for automatically generating legends for a visual chart as claimed in claim 2, characterized in that: In step S113, in the clustering results, each cluster represents a shape set, and a shape is randomly selected from each cluster as a representative icon symbol of the corresponding shape set.

6. A method for automatically generating legends for a visual chart according to claim 1, characterized in that: Extracting the mapping channel in step S1 is used to group the shape set obtained when extracting the representative icon symbol according to the color attribute. Extracting the mapping channel in step S1 includes the following sub-steps: S121, performing color clustering for each identified shape set to obtain a color grouping for each shape set; S122, sorting the colors of each shape set to obtain a corresponding color sequence; S123: Perform color interpolation on the sorted color sequence to obtain a color sequence suitable for generating a legend.

7. A method for automatically generating legends for a visual chart as claimed in claim 6, characterized in that: In step S121, all visualized RGB colors are converted into CIELAB space and normalized, and clustering is performed using the DBSCAN algorithm in the normalized LAB space.

8. A method for automatically generating legends for a visualization chart as claimed in claim 1, characterized in that: In step S2, a genetic algorithm is used to simulate the visualization legend search process as a gene mutation process, and the user feedback received from the feedback model is used to adjust and optimize the search results.

9. A method for automatically generating legends for a visualization chart as claimed in claim 1, characterized in that: In step S3, the feedback model receives the evaluation metric index input by the user and scores the searched visualization legend through a multi-layer neural network.

10. A system for automatically generating legends for visual charts, characterized by: A method for automatically generating legends for a visualization chart as described in any one of claims 1 to 9 is used to automatically generate legends for visualization.

Images