A dynamic visualization recommendation method based on user tasks

Abstract

The application discloses a kind of dynamic visualization recommendation methods based on user task, based on the task and preference data obtained by analysis during user exploration data process, design and realize the visualization recommendation model with interactive ability that can dynamically change with user exploration process;First, by analyzing data features and operation sequence, generate data facts, analysis task facts and field preference facts, then use these facts to guide the generation of recommended visualization for user selection.The application not only can integrate the task and field preference of user into the recommendation process, but also supports the interactive assignment of user to task and field preference, and can learn online based on user analysis process, continuously improve the model effect, can help data analysts to quickly analyze data and create multi-view visualization panel, and then improve data analysis and display efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information visualization and visual analytics, and particularly to a dynamic visualization recommendation method based on user tasks. BACKGROUND

[0002] In recent years, with the increasing application of visualization and visual analytics technology, people's demand for visualization tools is also increasing. For experienced visualization experts, the process of designing and using visualization is usually as follows: defining the visual analytics task, performing data analysis and processing, designing / choosing a visualization scheme, and defining the visualization visual mapping. However, this is not easy for ordinary users. Users can usually clearly define the analysis task, but cannot determine which data analysis method should be used, and it is even more difficult to determine the visualization scheme according to the characteristics of the data. Researchers have designed many visualization design guidelines based on their own experience, but these guidelines are complex and highly experienced, and are difficult to promote.

[0003] Automatic visualization methods provide a solution to the above problems: through the mixed driving of data and experience, high-quality insights are automatically discovered from data, and appropriate visualizations are selected for presentation. By providing assistance in data analysis, visualization scheme selection, and visual mapping, automatic visualization methods greatly reduce the threshold for users to create visualizations, enabling ordinary users to create visualizations comparable to professionals.

[0004] Existing automatic visualization methods usually take data as input and use the statistical characteristics of the calculated data as the basis for generating visualization charts. However, this data feature-based method lacks modeling of user tasks, resulting in poor recommendation efficiency and the need for repeated iterations. At the same time, the recommendation process of traditional methods lacks user control, making the recommended results prone to show monotony and repetition. Therefore, in recent years, researchers have tried to find new input data for automatic visualization methods. For example, taking the combination of user-created existing views and data as input can integrate the user's analysis process into the recommendation process, improving the accuracy of the recommendation effect. However, these methods cannot explicitly integrate the user's analysis task into the recommendation process, resulting in insufficient controllability of the recommendation process and a lack of sufficient explanation of the recommended results, making users lack confidence in the recommended effect. In summary, existing automatic visualization creation tools have the shortcomings of inaccurate recommendation effect, monotonous and repetitive recommended results, and poor controllability of the recommendation process. SUMMARY

[0005] In view of the above problems, the purpose of the present application is to provide a dynamic visualization recommendation method based on user tasks, based on the task and preference data obtained by analyzing the user exploration data process, design and implement a dynamic visualization recommendation model with interactive ability that can change with the user exploration process. The present application not only integrates the user's task and field preference into the recommendation process, but also supports the user's interactive assignment of tasks and field preferences, and can learn online based on the user analysis process, and continuously improve the model effect. The present application can help data analysts to quickly analyze data and create multi-view visualization panels, thereby improving the efficiency of data analysis and display. The specific technical solutions are as follows:

[0006] A dynamic visualization recommendation method based on user tasks, comprising the following steps:

[0007] S1: data acquisition and processing

[0008] Statistical analysis is performed on the user input data set to obtain the statistical characteristics and data types of the data; through the event bus and the publisher-subscriber model, the user operation behavior is recorded in the interactive analysis interface, and the high-frequency events are subjected to anti-jitter operation to ensure the relative balance of the frequencies of various events; the event type is mapped to the input format of the task perception model;

[0009] S2: analysis task and field preference generation

[0010] The task and field preference prediction results of the current time slice are read out from the task perception model; for the task prediction result, the historical data is used to smooth the prediction result to reduce the prediction result jitter; for the field preference prediction result, statistical test method is used to compare the existing field preference with the equal probability model to determine whether to modify the field preference parameters of the model;

[0011] S3: visualization view generation and processing

[0012] Based on the data characteristics and data types obtained in S1, data definition facts are generated; based on the analysis task and field preference obtained in S2, analysis task facts and field preference facts are generated; based on the visualization design experience, general visualization rule sets, task related rule sets and field preference rule sets are designed; based on the above facts and rule sets, answer set programming and linear optimization methods are used to search the visualization design space to find the optimal visualization design scheme; based on the visual similarity, a visualization deduplication strategy is designed to filter similar recommendation results and optimize the recommendation effect;

[0013] S4: interactive design

[0014] The algorithm control panel is designed to output the predicted user task and field preference characteristics, and to show the changes in the algorithm prediction results; based on a dynamic ranking column chart, the algorithm key inputs of the current time slice are displayed, and based on the mask and penalty value, the editing operation of the task is defined.

[0015] Further, in step S1, the data acquisition and processing are specifically as follows:

[0016] S11: five types of data are defined, namely, Boolean, integer, number, date and string; integer and number types belong to number type fields, and Boolean, date and string belong to string type fields; a corresponding detection function is set for each type of data, so as to determine the data type of each field;

[0017] S12: three statistical variables, namely, mean, variance and cardinality, are defined; for number type fields, the mean, variance and cardinality of the field are counted; for string type fields, only the cardinality of the field is counted and saved;

[0018] S13: based on the mitt transaction bus, the user behavior event name is defined; the event is published in the user interaction control and subscribed in the service component;

[0019] S14: a de-bouncing algorithm is used to remove duplicates for the sliding filtering operation, to avoid imbalance of behavior categories;

[0020] S15: a behavior-number mapping table is constructed to map the behavior event to a digital number, which is sent to the backend prediction model through the network.

[0021] Further, in step S2, the analysis of task and field preference is specifically as follows:

[0022] S21: the backend prediction model receives the encoded behavior sequence, and predicts the output probability of the current existing task and the preference degree of each field;

[0023] S22: historical information is used to smooth the model prediction results; the final obtained probability of each task is calculated based on the following formula:

[0024] S task =α*H task +(1-α)*T task

[0025] In the formula, S task represents the probability of task task, H task represents the last used task probability, T task represents the task probability output by the model, T task ∈(0,1), and alpha represents the fusion degree, 0≤alpha≤1; therefore, S task∈ [0, 1]; and in the initial state, H task = 0;

[0026] After the end of a calculation, S task is assigned to H task :

[0027] H task = S task , H task ∈ [0, 1]

[0028] S23: using a statistical test method, test the field preference degree and the equal probability distribution for comparison; if the detection finds that there is a significant difference between the two, the user field preference is used; otherwise, the user field preference is not enabled.

[0029] Further, the statistical test method is specifically:

[0030] S231: construct a statistical test hypothesis, construct the following hypothesis:

[0031] H0: the user's field preference has no significant difference compared with the existing data;

[0032] H1: the user's field preference has a significant difference compared with the existing data;

[0033] S232: acquisition of existing data: first, execute a hash algorithm on the data set to obtain a fixed-length id string; retrieve the record corresponding to the string in the system database, if it exists, the record is regarded as the existing field preference data of the data set; if it does not exist, it is regarded as the equal probability of each field selection, assuming that there are n fields fields, the theoretical preference probability of each field is

[0034] S233: sample size test: the number of user's selection for each field is A f , the total number of user's selection for the field is n records ; set the conditions of the total number of user's field preference records, the number of selection records of each field, and the theoretical frequency of each field, and perform sample size test:

[0035] If the sample size test is passed, go to step S234, otherwise, it is considered that the statistical test is not passed, and the user's field preference is rejected;

[0036] S234: correlation test: calculate X 2 by the following formula:

[0037]

[0038] Wherein, X 2 is the value of chi-square test; nfields is the total number of fields; n f is the access frequency of field f, p f is the theoretical preference probability of field f;

[0039] based on the degree of freedom n fields -1, find the cumulative distribution function corresponding to the chi-square distribution, based on the cumulative distribution function and the obtained X 2 , that is, the probability p of the H0 hypothesis being true is obtained, as follows:

[0040] p = 1 - F cdf (X 2 , n fields -1)

[0041] where F cdf is the cumulative distribution function of the chi-square distribution;

[0042] If p < 0.1, it is considered that the H0 hypothesis is not true, the H1 hypothesis is accepted, and the user preference is significantly different from the existing data, and the user field preference is used to replace the field preference of the existing data.

[0043] Further, in step S3, the specific process of the visualization generation and processing is as follows:

[0044] S31: Generate data definition facts: use the string template-based method to convert the data types determined in S1 into data type fact defined in the answer set programming;

[0045] S32: Generate analysis task facts: use the string template-based method to convert the analysis tasks determined in S2 into analysis task facts defined in the answer set programming;

[0046] S33: Generate field preference facts: use the string template-based method to convert the field preferences determined in S2 into field preference facts defined in the answer set programming;

[0047] S34: Design rule set: the rule set includes a general visualization rule set, a task-related rule set, and a field preference rule set; the general visualization rule set encodes general visualization aesthetic knowledge and general visualization knowledge based on data facts, which includes hard constraints and soft constraints; the task-related rule set encodes task-specific visualization knowledge; the field preference rule set encodes visualization knowledge under the presence of field preference constraints;

[0048] S35: Soft constraint weight design: when the visualization violates a soft constraint, a loss consistent with the weight is imposed; the loss of the visualization on all soft constraints is used to measure the pros and cons of the visualization; use the RankSVM-based method to generate weights for soft constraints, and count the total number of soft constraints as N softThe weights corresponding to soft constraints are expressed as follows:

[0049]

[0050] Among them, w i This represents the weight of the i-th soft constraint;

[0051] S36: Generation of candidate visualizations: Data definition facts, analysis task facts, field preference facts, and visualization rules are combined to form the input of the answer set programming solver. The answer set programming solver is based on a linear optimization algorithm to find the top few visualizations with the smallest soft constraint loss among all visualizations that do not violate hard constraints, thus forming candidate visualizations.

[0052] Furthermore, in step S35, the method for generating soft constraint weights is as follows:

[0053] S351: Randomly generate candidate visualization pairs: Based on predefined visualization variables and the candidate values ​​that can be selected for each variable, randomly generate a visualization A; randomly select one of the visualization variables and modify it to a different candidate value to obtain a similar visualization A′; A and A′ form a visualization pair, and the above steps are repeated until N is obtained. pair Each visualization pair is randomly assigned a task label.

[0054] S352: Comparison of Visualization Pairs: Users are required to select the visualization from several pairs that best accomplishes a given task, ultimately obtaining N... pair A visualization showing the comparison results on the corresponding task;

[0055] S353: Soft Constraint Weight Training: Solve for the number of times that visualizations A and A′ in a visualization pair violate each soft constraint, forming N. soft 3D eigenvector F A and F A′ ; F A and F A′ Subtracting them yields two visually distinct feature vectors F. diff =F A -F A′ And construct the comparison result y of A and A′:

[0056]

[0057] Each F diff As the input feature of the RankSVM algorithm, y is used as the output label. The hinge loss function is used to optimize the weights W to obtain the optimal soft constraint loss that satisfies the given comparison relationship. The optimization objective is:

[0058]

[0059] W * = argmin W L

[0060] wherein L is the value of loss; n is the number of input samples; k is the number of soft constraint weights; y i is the label of corresponding sample; λ is the regularization weight; ‖·‖2 represents L2 regularization method; argmin W L represents the value of W that minimizes L; W * is the optimized weight value W.

[0061] Further, in step S4, the interaction design is specifically as follows:

[0062] S41: Dynamic display of user tasks: the dynamic display of user tasks is in the form of a dynamically sorted column chart, each rectangle in the column chart represents an analysis task, and different filling colors are used between rectangles corresponding to different tasks as a distinction; the horizontal width width task = S task × 100%, when S task changes, the width and order of the rectangle are adjusted in the form of a tween animation to ensure that the width of the rectangle in the column chart decreases from top to bottom; S task represents the probability of task task;

[0063] S42: Dynamic display of field preferences: the dynamic display of field preferences is in the form of a dynamically sorted column chart, each rectangle in the column chart represents a field in the data set, and the horizontal width width

[0064] For a field whose preference is not identified, the corresponding rectangle is filled with a transparency n records is the total number of selections of the field by the user, n fields is the total number of fields;

[0065] For a field whose preference is identified, the corresponding rectangle is filled with a transparency β = 100%, and is displayed above the rectangle whose preference is not identified;

[0066] When n field and n records change, the width and order of the rectangle are adjusted in the form of a tween animation to ensure that the width of the rectangle in the column chart decreases from top to bottom;

[0067] S43: Assignment of tasks: when the user is not satisfied with the task prediction result, the assignment function is used to add tasks from the candidate tasks to improve the recommendation effect of subsequent visualization;

[0068] S44: Removal of tasks: When the user is not satisfied with the task prediction result, the task is removed from the identified tasks through the removal function to improve the recommended effect of subsequent visualization;

[0069] S45: Temporary masking of tasks: When the user needs to compare the recommended results under different task combinations, the temporary masking function is used to temporarily mask the tasks from the identified tasks to compare the differences between the recommended results.

[0070] The beneficial effects of the present application are:

[0071] 1) The present application makes up for the deficiency of the personalized factor in the existing automatic visualization. The existing automatic visualization method completes the recommendation task according to the general visualization knowledge, or automatically learns from high-quality visualization charts. Such recommendation lacks consideration of the personalized factors of the user itself, so the recommendation effect is difficult to optimize. In order to solve this problem, the present method defines the user intention as the analysis task and the preference field, and respectively designs the analysis task and the preference field specific rules to integrate the user intention, so that the visualization recommendation can better meet the actual needs of the user.

[0072] 2) The present application makes up for the deficiency of the configurability in the existing automatic visualization. The existing automatic visualization method is divided into end-to-end method and multi-stage method, the former directly generates visualization through deep learning model, which is limited by the black box characteristics of deep learning model, and is usually difficult to configure and expand. The latter generates visualization through multi-stage heuristic algorithm such as feature extraction, rule matching and sorting, although it has expandability, but the coupling between the stages is strong, and usually needs to be configured by experts in specific fields, which is difficult for ordinary users to operate. The present method models the "intention" as the user task and the field preference, so that ordinary users can adjust the core parameters in the visualization recommendation process in a way they are familiar with, thereby reducing the configuration threshold of the system; on this basis, the present method uses the answer set programming based method to store the visualization rules, which reduces the coupling between different rules, so that the maintenance personnel only need a small amount of prior knowledge to maintain and expand the knowledge base of the system. BRIEF DESCRIPTION OF DRAWINGS

[0073] Figure 1 is the overall method flow framework schematic diagram of the present application based on automatic identification of user behavior analysis task and field preference, and generating visualization recommendation.

[0074] Figure 2 is the overall user interface structure and layout schematic diagram of the present application.

[0075] Figure 3 is the model control panel part of the user interface schematic diagram of the present application.

[0076] Figure 4It is the interaction diagram of the user manually removing the previously set preference field in the interaction control method of the preference field of the application.

[0077] Figure 5 It is the interaction diagram of the user manually removing the previously set preference field in the interaction control method of the preference field of the application.

[0078] Figure 6 It is the interaction diagram of the user manually setting the preference field in the interaction control method of the preference field of the application.

[0079] Figure 7 It is the automatic adjustment diagram of the preference field in the interaction control method of the preference field of the application. DETAILED DESCRIPTION

[0080] The application realizes automatic perception of user tasks and field preferences and supports visual recommendation through effective automatic visualization methods, combined with statistical learning methods and answer set programming means. On this basis, a convenient interactive interface is also provided for analysts to manually assign preferences to adjust the recommendation effect, ultimately helping analysts to discover data facts in visual data exploration, thereby improving decision-making and visualization creation efficiency. The technical scheme includes data acquisition and processing, analysis task and field preference generation, visual view generation and processing, and interaction design. The overall method flow framework diagram is shown in Figure 1 The specific steps are as follows:

[0081] Step 1: Data acquisition and processing

[0082] Statistical analysis is performed on the user input data set to obtain the statistical characteristics and data types of the data; through the event bus and the publisher-subscriber model, user operation behaviors are recorded in the interactive analysis interface, and high-frequency events are subjected to anti-jitter operation to ensure the relative balance of the frequencies of various events; and the event types are mapped to the input format of the task perception model. The specific process is as follows:

[0083] S11: Define five data types, namely Boolean, integer, number, date and string; integer and number types belong to number type fields, and Boolean, date and string belong to string type fields; set corresponding detection functions for each data type to determine the data type of each field;

[0084] S12: Define three statistical variables, namely mean, variance and cardinality; for number type fields, the mean, variance and cardinality of the field are counted; for string type fields, only the cardinality of the field is counted and saved;

[0085] S13: Define user behavior event name based on mitt transaction bus; publish event in user interaction control and subscribe in service component;

[0086] S14: Use anti-shake algorithm to remove duplicate in sliding filtering operation, and avoid behavior category imbalance;

[0087] S15: Build behavior-number mapping table to map behavior event to digital number, and send to backend prediction model through network.

[0088] The specific implementation is as follows:

[0089] A dataset feature extraction system and a user operation record and conversion system are designed and implemented. The dataset feature extraction system can read the CSV or JSON format data uploaded by the user, automatically analyze the statistical characteristics, and generate data facts; the user operation record and conversion system can receive the operation log from the user interface, perform de-duplication, anti-shake and other operations, and generate user operation sequences.

[0090] 1. Design and implement a dataset feature extraction system.

[0091] (1) Determine the file format and parse: for the data uploaded by the user, first determine whether the file is a JSON file or a CSV file by the extension, and select the parser according to the file type to parse the data file into an object array.

[0092] (2) Infer field name: if the uploaded file is in CSV format, the field names in the dataset can be directly inferred from the first row. If the uploaded file is in JSON format, the first record can be used to determine the field names.

[0093] (3) Infer field type: take the first valid record as the standard to determine the data type of each field. During the judgment process, according to the judgment order in Table 1, the record value is judged whether it meets the judgment basis. If it meets, the record belongs to this data type, otherwise, it is judged whether it meets the next data type.

[0094] Table 1 Field data type judgment order and judgment basis

[0095] Judgment sequence Data type Judgment basis 1 Boolean Value is equal to "true" or "false" 2 Integer Value can be judged as true by Number.isInteger function 3 Number Value can be judged as false by isNaN function, and is not a date 4 Date Value can be parsed as a date 5 String None of the above judgment basis is satisfied

[0096] (4) Extract statistical variables: according to the field type extracted in (3), the fields are divided into two categories: numerical fields and string fields, where integer and numerical types belong to numerical fields, and boolean, date and string belong to string fields. For numerical fields, the average, variance and cardinality of the field are calculated. The cardinality refers to the number of unique values in the field. For string fields, only the cardinality of the field is calculated.

[0097] 2. Design and implement a user operation recording and conversion system.

[0098] (1) Capture user interaction operations: Introduce a mitt transaction bus globally in the user interface, and add event sending code in the interaction control callback function that needs to be recorded. The event information should include event name, event category, occurrence time, and self-incremented event serial number.

[0099] (2) Return user interaction operations: Introduce a global listener in the user interface to listen to all events in the mitt event bus. For a large number of repetitive events such as dragging and mouse movement, use the debounce algorithm to prevent jittering of events, and retain the first and last events within the debounce interval. After processing, the user interaction event information is transmitted to the backend server through network request.

[0100] (3) Convert user interaction operations: In order to represent discrete user interactions as vectors suitable for deep learning processing, user operations need to be converted to one-hot encoded form. According to all operation types supported by the system, define a continuous and unique serial number for each operation. For each operation, the corresponding number position is 1 and other positions are 0, which can obtain the one-hot encoded form of user operation.

[0101] Step two: Analysis of task and field preference generation

[0102] Read the task and field preference prediction results of the current time slice from the task perception model; for task prediction results, use historical data to smooth the prediction results to reduce prediction result jitter; for field preference prediction results, use statistical test methods to compare the existing field preference with the equal probability model to determine whether to modify the field preference parameters of the model.

[0103] S21: The backend prediction model receives the encoded behavior sequence and predicts the probability of the existence of a task and the preference degree of each field;

[0104] S22: Use historical information to smooth the model prediction results; the final task probability is calculated based on the following formula:

[0105] S task =α*H task +(1-α)*T task

[0106] In the formula: S task represents the probability of task task, H task represents the last used task probability, T task represents the task probability output by the model, and Ttask ∈(0,1), a represents the degree of fusion, 0≤a≤1; thus, S task ∈[0,1]; and in the initial state, H task =0;

[0107] After the end of a calculation, S task is assigned to H task :

[0108] H task =S task , H task ∈[0,1]

[0109] S23: using statistical test methods, test the field preference degree and compare the equal probability distribution; if the detection finds that there is a significant difference (p<0.1) between the two, then use the user field preference; otherwise, do not enable the user field preference.

[0110] The statistical test method is specifically:

[0111] S231: construct a statistical test hypothesis, construct the following hypothesis:

[0112] H0: the user's field preference has no significant difference compared with the existing data;

[0113] H1: the user's field preference has a significant difference compared with the existing data;

[0114] S232: acquisition of existing data: first, execute a hash algorithm on the data set to obtain a fixed-length id string. Retrieve the record corresponding to the string in the system database, and if it exists, consider the record as the existing field preference data of the data set; if it does not exist, consider that the selection probability of each field is equal, and suppose that there are n fields fields, then the theoretical preference probability of each field is

[0115] S233: sample size test. The number of times the user selects each field is A f , and the total number of times the user selects the field is n records . If the following conditions are met at the same time, the sample size test is passed:

[0116] ①n records ≥40. That is, the total number of times the user's field preference record is required to be greater than 40;

[0117] ②min(n f )≥5. That is, the selection record of each field is not less than 5 times;

[0118] ③ That is, the theoretical frequency of each field is not less than 5.

[0119] In this step, if the sample size test is passed, go to step S234, otherwise, it is considered that the statistical test is not passed, and the user's field preference is rejected.

[0120] S234: Correlation test. Calculate X 2 :

[0121]

[0122] Based on the degrees of freedom n fields -1, find the cumulative distribution function (CDF) corresponding to the chi-square distribution, based on the cumulative distribution function and the calculated X 2 , the probability p that the H0 hypothesis is established can be obtained, as follows:

[0123] p=1-F cdf (X 2 ,n fields -1)

[0124] If p<0.1, it is considered that the H0 hypothesis is not established, the H1 hypothesis is accepted, the user's preference is significantly different from the existing data, and the user's field preference is used instead of the field preference of the existing data.

[0125] The specific implementation is as follows:

[0126] After obtaining the user interaction operation, further generate the analysis task and the field preference through the machine learning model and the statistical learning method, as the personalized parameter guiding the visual generation.

[0127] 1. Operation sequence to analysis task prediction model: design a sequence-multiclass machine learning model (such as a decision tree), which can output the multi-classification result of the user's task and its confidence under the condition of a specified number of one-hot encoded user operations. Specifically, the analysis tasks supported by the present application are shown in Table 2.

[0128] Table 2: Analysis tasks and examples supported by the system

[0129] Task Example Trend How does the annual average temperature change? Correlation Is there a relationship between precipitation and wind speed? Comparison Is there a difference between the temperature of New York and Seattle? Confirm the value What is the temperature of New York on January 1, 2012? Data conversion What is the average temperature of New York in January? Others Is there an outlier in the temperature of New York in 2012?

[0130] 2. Obtain field preference from operation sequence: the user's operation may contain selection operation on visual encoding (such as setting of the data field corresponding to the X axis), which is considered as an access to the corresponding field. Whenever the back-end server receives a new user operation, it will first check whether it is an operation containing visual encoding selection. If so, the access frequency of the corresponding field in the counter will be incremented by one.

[0131] 3. Smoothing the prediction result with historical information: Since the input of the model is fixed length, it cannot consider long-term historical information, which leads to fluctuations in the prediction result. Therefore, historical information is used to smooth the prediction result. For each task, its final score S task = a x H task + (1-a) x T task , where H task is the historical score of the task, T task is the task score output by the model at present, and a is the smoothing weight, which is set to 0.5. At the same time, after obtaining the score S task , the historical score H task is updated as H task .

[0132] 4. Test the sample size of the user access field, with the following requirements:

[0133] ① The total number of user access to the field is greater than 40 times;

[0134] ② The access frequency of each data field is greater than 5 times;

[0135] If the user's access to the field meets the above two requirements, it is considered to pass the sample size test, otherwise it is considered not to pass the test, and the preference for the access field is empty, directly entering step three.

[0136] 5. Use statistical test method (chi-square test) to test whether the field preference degree is significantly different from the equal probability model: If the user's access frequency to the field is significantly different from the equal probability model (p>0.1), it is considered to pass the statistical test, and the user's access frequency to the field is input into the recommender as the field preference.

[0137] Step three: Visualization view generation and processing

[0138] Based on the data characteristics and data types obtained in step one, generate data definition facts; based on the analysis tasks and field preferences obtained in step two, generate analysis task facts and field preference facts; based on the visualization design experience, design a general visualization rule set, a task-related rule set, and a field preference rule set; based on the above facts and rule sets, use answer set programming and linear optimization methods to search the visualization design space to find the optimal visualization design scheme; based on visual similarity, design a visualization deduplication strategy to filter similar recommended results and optimize the recommendation effect.

[0139] After obtaining the user's analysis tasks and field preferences, these information can be used to guide the generation of visualization recommendations.

[0140] 1. Generation of data definition facts: Using string template-based method, the field types and statistical variables obtained in step 1 are converted into data definition facts in Answer Set Programming (ASP). For example, if the data type of field F A is Number, then its corresponding ASP fact is :-type(FA, Number). Where type(<field name>, <data type>) represents the type relationship of the field.

[0141] 2. Generation of analysis task facts: Using string template-based method, the analysis tasks obtained in step 2 are converted into analysis task facts in Answer Set Programming (ASP). For example, if the user has analysis task T B , then its corresponding ASP fact is :-task(TB). Where task(<task name>) represents that the user is engaged in a certain task.

[0142] 3. Generation of field preference facts: Using string template-based method, the field preference facts obtained in step 2 are converted into field preference facts in Answer Set Programming (ASP). For example, if the user's access frequency to field F A is 0.4, then its corresponding ASP fact is :-preference(FA, 4). Where preference(<field name>, <access frequency>) represents the user's preference for a certain field. Since Answer Set Programming does not support floating-point numbers, the converted access frequency is ten times the actual access frequency and is rounded off.

[0143] 4. Design of rule set: To generate visualization through Answer Set Programming, it is necessary to pre-set visualization rules to limit the scope of the solver and optimize the solving effect.

[0144] The rule set includes general visualization rule set, task-related rule set, and field preference rule set; the general visualization rule set encodes general visualization aesthetic knowledge and data fact-based general visualization knowledge; the task-related rule set encodes task-specific visualization knowledge; the field preference rule set encodes visualization knowledge under the constraint of field preference.

[0145] (1) User interviews and literature research: Through organizing seminars, one-on-one interviews, and other forms, communicate with potential users of the system and visualization system design experts, and record the corresponding visualization rules one by one. Through research and summary of literature on domestic and international visualization or graphics-related conferences and journals, usable visualization rules can also be obtained.

[0146] (2) Hard and soft constraint division: From the perspective of the importance, accuracy and purpose of the rules, the collected visualization rules are manually divided into hard and soft constraints. The division criteria are shown in Table 3:

[0147] Table 3 Differences and examples of hard and soft constraints

[0148] Features Hard constraints Soft constraints Importance Must be met Can be violated Accuracy Accurate, from visualization experts or authoritative literature Not necessarily accurate, from user requirements Purpose Ensure the basic usability of the generated visualization Improve the display effect of the visualization Example Cannot calculate the average value of string type data Trend type task should use line chart

[0149] 5. Soft constraint weight design: When a visualization violates a soft constraint, a loss consistent with the weight is applied; the loss of the visualization on all soft constraints is used to measure the pros and cons of the visualization; a RankSVM-based method is used to generate weights for soft constraints, and the total number of soft constraints is counted as N soft , the weight corresponding to the soft constraint is represented as:

[0150]

[0151] where w i represents the weight of the i-th soft constraint.

[0152] (1) Generate candidate visualization pairs: adjusting the soft constraint weight using RankSVM requires the construction of comparable visualization pairs. Visualization pairs are randomly enumerated based on the visualization variables and candidate values defined in Table 4. Each time a combination of visualization variables is randomly selected, then one of the variables is modified to a random candidate value, thereby generating a visualization pair that differs only in one visualization variable. A total of 1000 visualization pairs are generated in this step, each corresponding to a random task.

[0153] Table 4 Supported visualization variables and candidate values

[0154]

[0155]

[0156] (2) Visualization pair comparison: recruit about 100 users and ask each user to watch 30-40 visualization pairs and select a visualization that can better complete a given task. Among them, the order of the visualizations watched by the user is randomly balanced. For conflicting comparison results of the same visualization pair, the more results are used as the criterion. From this, the comparison results of these visualization pairs on the corresponding tasks can be obtained.

[0157] (3) Soft constraint weight training: for each visualization pair (A, A'), first record the number of times they violate each soft constraint to obtain the feature vector pair (F A , F A′ ). Further, F A and F A′the difference vector F diff = F A - F A′ The comparison results of A and A' are then constructed:

[0158]

[0159] The soft constraint weight w is optimized using a hinge loss function, and the optimization objective is:

[0160]

[0161] W * = argmin W L

[0162] where L is the value of the loss; n is the number of input samples; k is the number of soft constraint weights; y i is the label corresponding to the sample; λ is the regularization weight; ‖·‖2 represents the L2 regularization method; argmin W L represents the value of W that minimizes L; W * is the optimized weight value W.

[0163] 6. Generation of candidate visualizations: The data facts, analysis task facts, field preference facts, and visualization rules are spliced together to form a complete answer set programming program input. The above program input answer set programmer (such as Clingo) is used to solve the k candidate visualizations that satisfy the hard constraints and have the minimum total weight (referred to as loss) of violating the soft constraints.

[0164] Candidate visualization deduplication: Since each candidate visualization is independently solved, the candidate visualizations generated by the solver may include a large number of visually similar visualizations. These visually similar candidate visualizations increase the cognitive burden of the user. For each visualization, the unordered set of fields involved in the encoding is extracted. For candidate visualizations with the same field set, only the candidate visualization with the minimum loss is retained.

[0165] Step four: system interaction design

[0166] The algorithm control panel is designed to output the user task and field preference characteristics predicted in S2, and to show the changes in the algorithm prediction results. Based on the dynamic ranking bar chart, the algorithm key inputs of the current time slice are displayed, and based on the mask and penalty value, three editing operations of the task, namely "assign", "remove", and "temporary mask", are defined.

[0167] Designing front-end user interface and interaction control panel to allow users to interact with the system for interactive data exploration and create multiple visualizations; allow users to view visualization recommendation results and apply in real time; allow users to adjust and manually assign analysis tasks and field preferences; allow users to enable or disable related modules to adapt to different analysis needs.

[0168] S41: Dynamic display of user tasks. User task dynamic display is presented in the form of a dynamically sorted column chart, where each rectangle in the column chart represents an analysis task. The horizontal width of the column chart is 100%, and the horizontal width of each rectangle is width task = S task × 100%. Different filling colors are used to distinguish between different task rectangles. When S task changes, the width and order of the rectangles are adjusted in the form of a tween animation to ensure that the width of the rectangles in the column chart decreases from top to bottom.

[0169] S42: Dynamic display of field preferences. Field preference dynamic display is presented in the form of a dynamically sorted column chart, where each rectangle in the column chart represents a field in the dataset. The horizontal width of the column chart is 100%, and the horizontal width of each rectangle is width For fields that have not been identified as preferences, their corresponding rectangles have a fill transparency For fields that have been identified as preferences, their corresponding rectangles have a fill transparency β = 100%, and are displayed above the rectangles that have not been identified as preferences. When n field and n records change, the width and order of the rectangles are adjusted in the form of a tween animation to ensure that the width of the rectangles in the column chart decreases from top to bottom.

[0170] S43: Task assignment. When the user is not satisfied with the task prediction results, the assignment function can be used to add tasks from the candidate tasks to improve the recommendation effect of subsequent visualizations. The S task of tasks assigned by the user is always 1.

[0171] S44: Task removal. When the user is not satisfied with the task prediction results, the removal function can be used to delete tasks from the identified tasks to improve the recommendation effect of subsequent visualizations. The S task of tasks deleted by the user is always 0.

[0172] S45: Temporary masking of tasks. When the user needs to compare the recommendation results of different task combinations, the temporary masking function can be used to temporarily mask tasks from the identified tasks to compare the differences between the visualization recommendation results. The S task of tasks temporarily masked by the user will be considered as 0, but can be manually or automatically restored to the original value.

[0173] 1. Overall interface: The overall user interface of the method includes three main parts, namely the interactive control panel, the recommended result viewing bar and the grid canvas. Among them, the interactive control panel can be further divided into a visualization control panel and a model control panel. The user interface structure and layout diagram is shown in Figure 2 .

[0174] (1) Visualization control panel: This panel controls the visual settings and data settings of the selected chart. The user can enable or disable a certain visual channel, select the visualization type, use the aggregation function for the visual channel, filter and bin. For numerical fields, the user can select a continuous interval to limit the display of data within the interval; for string fields, the user can select only the checked values by checking.

[0175] (2) Model control panel: The user interface diagram of the model control panel is shown in Figure 3 . This panel displays the user's analysis tasks and field preferences and allows interactive adjustment. The user's analysis tasks are presented in the form of a line chart, where the X-axis encodes the number of operation steps, and the maximum of 15 recent operation steps are displayed; the Y-axis encodes the confidence of the model output, with a range of 0-1; the color encodes the type of task, and the supported analysis tasks are shown in Table 2. The model infers the task the user is likely to be performing in the form of a dynamically sorted bar chart, with the length encoding representing the confidence and the color encoding representing the task type. The user's field preferences are also displayed in the form of a dynamically sorted bar chart, where the model-judged preferred fields and the user-specified preferred fields are located at the top and distinguished from each other by color; the remaining fields are located below, with both length and transparency encoding representing access frequency. A dividing line is used to distinguish between the two areas. In particular, the correspondence between transparency and access frequency is shown as follows:

[0176]

[0177] where β is the transparency of the rectangle, freq f is the access frequency of field f.

[0178] (3) Recommended result viewing bar: The recommended result viewing bar displays the candidate visualizations generated by the visualization recommendation module, with the loss increasing from top to bottom. The user can click on the satisfactory recommended result to add it to the canvas.

[0179] (4) Grid canvas: The grid canvas uses a grid layout to position each visualization, i.e. the four corners of each visualization must fall on the grid points. The number of rows of the grid is not limited, and the number of columns is not more than 12 to facilitate user alignment of charts.

[0180] 2. Interactive design: Users can control the parameters of the recommendation system through interactive means. Parameter control can be divided into task control, preference field control and module switch.

[0181] (1) Task control: Users can click the rectangle in the dynamic sorting column chart to temporarily mask the corresponding task. The rectangle corresponding to the temporarily masked task is filled with light gray to distinguish it. Clicking the rectangle again can cancel the masking effect. A delete button is drawn on the right side of each rectangle, and users can click the button to permanently delete the corresponding task. The deleted task will no longer be displayed in the dynamic sorting column chart, unless manually added by the user. Users can also manually add the corresponding task by adding a button. The confidence of the manually added task is always 1.

[0182] (2) Preference field control: Users can click the dynamic sorting column chart to adjust the preference of the field. The schematic diagram of the interactive control method for the preference field is shown in Figure 4-7 , and the operation method is as follows:

[0183] ① If the rectangle corresponding to the field is a field that the model judges to have a preference, this click is considered to forcibly remove the preference of the field. The rectangle corresponding to the field is considered to have no special preference and is moved to the area below the remaining fields. The transparency and length mapping method still follows the access frequency; as shown in Figure 4 .

[0184] ② If the rectangle corresponding to the field is a field that the user has specified as a preference field, this click is considered to cancel the specification of the preference of the field. The preference of the field will be restored to the true value. The rectangle corresponding to the field is moved to the area below the remaining fields. The transparency and length mapping method is consistent with the true access frequency; as shown in Figure 5 .

[0185] ③ If the rectangle corresponding to the field is not a field that the model or the user judges as a preference field, this click is considered to specify the preference of the field. The access frequency of the field is considered to be 1. The rectangle corresponding to the field is moved to the area above the preference fields. The transparency and length mapping method is consistent with the current frequency; as shown in Figure 6 . As the access rate changes, the column chart changes the sorting; as shown in Figure 7 .

[0186] (3) Module switch: the user can specify the task preference, field preference switch through the switch, and can turn off or turn on the visual recommendation, or reset the model state. Turning off the task preference means that the model no longer generates task facts; turning off the field preference means that the model no longer generates field preference facts; after turning off the visual recommendation, the "recommended result viewing column" is no longer displayed; resetting the model will delete the existing operation sequence and access frequency, and restore the task prediction model to the initial state.

Claims

1. A dynamic visualization recommendation method based on user tasks, characterized in that, The method comprises the following steps: S1: data acquisition and processing statistical analysis is performed on the user input data set to obtain statistical characteristics and data types of the data; through an event bus and a publisher-subscriber model, user operation behaviors are recorded in an interactive analysis interface, and high-frequency events are subjected to anti-jitter operation to ensure the relative balance of the frequencies of the events; and event types are mapped to an input format of a task perception model; S2: analysis task and field preference generation task and field preference prediction results of a current time slice are read from the task perception model; for the task prediction results, historical data is used to smooth the prediction results to reduce jitter of the prediction results; for the field preference prediction results, statistical test methods are used to compare existing field preferences with an equal probability model to determine whether to modify field preference parameters of the model; S3: visual view generation and processing data definition facts are generated based on the data characteristics and data types obtained in S1; analysis task facts and field preference facts are generated based on the analysis tasks and field preferences obtained in S2; based on visual design experience, general visual rules, task-related rules and field preference rules are designed; based on the above facts and rule sets, answer set programming and linear optimization methods are used to search the visual design space to find an optimal visual design scheme; based on visual similarity, a visual deduplication strategy is designed to filter similar recommended results and optimize the recommendation effect; S4: interactive design an algorithm control panel is designed to output predicted user tasks and field preference characteristics and display changes in the algorithm prediction results; based on a dynamic ranking column chart, algorithm key inputs of the current time slice are displayed, and based on masks and penalty values, editing operations of the tasks are defined; In step S3, the specific process of the visual generation and processing is as follows: S31: generating data definition facts: using a string template-based method, the data types determined in S1 are converted into data type facts defined in answer set programming; S32: generating analysis task facts: using a string template-based method, the analysis tasks determined in S2 are converted into analysis task facts defined in answer set programming; S33: generating field preference facts: using a string template-based method, the field preferences determined in S2 are converted into field preference facts defined in answer set programming; S34: designing rule sets: the rule sets include general visual rules, task-related rules and field preference rules; the general visual rules encode general visual aesthetic knowledge and general visual knowledge based on data facts, and include hard constraints and soft constraints; the task-related rules encode task-specific visual knowledge; the field preference rules encode visual knowledge under the presence of field preference constraints; S35: soft constraint weight design: when a visual violates a soft constraint, a loss consistent with the weight is applied; visualize the loss on all soft constraints and to measure the quality of the visualization; use a RankSVM based approach to generate weights for the soft constraints, counting the total number of soft constraints The weight corresponding to a soft constraint is then represented as: ； wherein, represents the weight of the th soft constraint; S36: Generation of candidate visualizations: data definition facts, analysis task facts, field preference facts, visualization rule combinations, form the input of the answer set programming solver, the answer set programming solver based on linear optimization algorithm, find the first few visualizations in all visualizations that do not violate the hard constraints, the soft constraint loss is minimum, form the candidate visualizations. 2.The method of claim 1, wherein, In step S1, the data acquisition and processing are specifically: S11: Define five data types, namely: Boolean, integer, number, date and string; integer and number type belong to number type field, Boolean, date and string belong to string type field; set the corresponding detection function for each data type, so as to judge the data type of each field; S12: Define three statistical variables, namely mean, variance and cardinality; for number type field, calculate the mean, variance and cardinality of the field; for string type field, only calculate the cardinality of the field and record it; S13: Based on mitt transaction bus, define user behavior event name; publish events in user interaction control and subscribe in service component; S14: Use anti-shake algorithm to remove duplicate data for sliding filtering operation to avoid imbalance of behavior categories; S15: Build behavior-number mapping table to map behavior events to digital number and send to backend prediction model through network. 3.The method of claim 2, wherein, In step S2, the analysis task and field preference generation are specifically: S21: The backend prediction model receives the encoded behavior sequence and predicts the probability of the current existence of a task and the preference degree of each field; S22: Use historical information to smooth the model prediction result; the final task probability is calculated based on the following formula: ； wherein: represents the probability of a task , represents the probability of a task used last time represents the probability of a task output by the model , represents the degree of fusion ; therefore ; and in the initial state ; After the end of one calculation, the values are set to :​ ； S23: Use statistical test method to test the field preference degree and equal probability distribution; if the test finds that there is a significant difference between the two, use the user field preference; otherwise, do not enable the user field preference.

4. The method of claim 3, wherein, The statistical test method is specifically: S231: Construct statistical test hypothesis, construct the following hypothesis: H0: There is no significant difference between the user's field preference and the existing data; H1: There is a significant difference between the user's field preference and the existing data; S232: Obtain existing data: first, perform a hash algorithm on the data set to obtain a fixed-length id string; search the system database for the record corresponding to the string, if it exists, consider the record as the existing data set field preference data; if it does not exist, consider the selection probability of each field to be equal, assuming there are a total of fields , then the theoretical preference probability of each field is ; S233: Sample size test: the number of times that the user selects each field is , the total number of times that the user selects the field is ; conditions are set for the total number of times that the user's field preference record is recorded, the number of times that each kind of field is selected, and the theoretical frequency of each field, and a sample size test is performed: If the sample size test is passed, go to step S234, otherwise, it is considered that the statistical test is not passed and the user's field preference is not used; S234: Correlation test: Calculate by the following formula : ； wherein, is a value for a chi-square test; is a total number of fields; is a field f access frequency, is a field f theoretical preference probability; Based on the degrees of freedom , find the cumulative distribution function corresponding to the chi-square distribution, based on the cumulative distribution function and the obtained , the probability that the H0 hypothesis is true is obtained , as follows: ； wherein is the cumulative distribution function of the chi-square distribution; If H0 hypothesis is not established, H1 hypothesis is accepted, the user preference is significantly different from the existing data, and the user field preference is used instead of the field preference of the existing data.

5. The method of claim 3, wherein, In step S35, the soft constraint weight generation method is specifically as follows: S351: randomly generate a candidate visualization pair: randomly generate a visualization based on the predefined visualization variables and candidate values selectable for each variable ; randomly select one of the visualization variables to be modified to a different candidate value, resulting in a similar visualization ; and constitute a visualization pair, repeat the steps of generating a visualization pair until a number of visualization pairs is obtained visualization pairs, randomly assign a task label to each visualization pair; S352: Comparison of pairs of visualizations: the user is asked to select the visualization of a number of pairs of visualizations that best accomplishes a given task, resulting in a comparison of the result of the pair of visualizations on the corresponding task; S353: soft constraint weight training: solving visualization in pair and the number of violations of each soft constraint, forming a feature vector of dimension and ; subtracting and to obtain a difference feature vector of the two visualizations ; and constructing and a comparison result : ； Each of the plurality of images is classified into one of the plurality of classes As input features for the RankSVM algorithm, As output labels, the hinge loss function is used to optimize the weights to obtain an optimal soft-constrained loss that satisfies a given comparison relationship, and the optimization objective is ； ； wherein, is a value of loss; is a number of input samples; is a number of soft constraint weights; is a label of a corresponding sample; is a regularization weight; denotes an L2 regularization method; denotes a minimization of L takes a value; is an optimized weight value ; is an i-th difference feature in a difference feature vector ; and is an i-th difference feature.

6. The method of claim 1, wherein, In step S4, the interaction design is specifically: S41: Dynamic display of user tasks: the dynamic display of user tasks is in the form of a dynamically sorted column chart, and each rectangle in the column chart represents an analysis task. Different filling colors are used to distinguish between different tasks. The horizontal width of each rectangle changes, the width and order of the rectangles are adjusted in the form of in- between animation to ensure that the width of the rectangles in the column chart decreases from top to bottom; represents the probability of the task .​ S42: Dynamic display of field preference: the dynamic display of field preference is presented in the form of a dynamically sorted column chart, each rectangle in the column chart represents a field in the data set, and the horizontal width of each rectangle represents the proportion of the field in the data set ; For the field whose preference is not identified, its corresponding rectangle fills the transparency ; is the total selection times of the user to the field, is the total number of fields; For the field for which a preference has been identified, its corresponding rectangle is filled with transparency , and is displayed above the rectangle for which no preference has been identified; When and When the height changes, the width and order of the rectangles are adjusted in the form of tweening to ensure that the width of the rectangles in the column chart decreases from top to bottom. S43: Task assignment: when the user is not satisfied with the task prediction result, increase the task from the candidate task through the assignment function to improve the recommendation effect of subsequent visualization; S44: Task removal: when the user is not satisfied with the task prediction result, delete the task from the identified task through the removal function to improve the recommendation effect of subsequent visualization; S45: Temporary masking of task: when the user needs to compare the recommended results under different task combinations, temporarily mask the task from the identified task through the temporary masking function to compare the differences between the visualization recommendation results.

Images